Wildcard SNIs not being matched

Close! Certificate management never blocks startup, it’s always in the background of the main goroutine (unless you set manage_sync to true, but don’t do that unless you have a very good reason). The only difference is that Caddy will obtain “on-demand” certificates, well, on-demand… i.e. at the first handshake that requires it, rather than in the background at config load.

Sure. Say you run a SaaS / web service, and your customers bring their own domain names. You tell them “point your domains to our server” but you have no idea when or if that will ever actually happen. If you started getting a cert for that domain right away, it would fail, since you don’t know when the customer has updated their DNS. Sure, you could poll, but DNS is all a matter of perspective anyway, so it’s unreliable and expensive to do that.

All you can do is whitelist their domain when they sign up for your service, and whenever your site starts seeing handshakes for that server name, then you know it’s time to get a certificate.

1 Like

To clarify the “whitelist” is with using the "ask" option https://caddyserver.com/docs/json/apps/tls/automation/on_demand/

Caddy would “ask” the SaaS whether a domain is okay to use. You’d probably have a DB table that has all the domains you allow your customers to use to point to your service, and reply with HTTP 200 if it’s in the list, 4xx otherwise.

1 Like

Oh man, that’s the innovation right there! That’s amazing stuff! Kudos to you for coming up with that idea!

One more reason why you deserve more love in terms of sponsorship. You’ll be the first person that I sponsor when I can actually afford it.

And you’re really nice and welcoming which is another thing ( other than fast response times ) that’s not seen in other open source projects that I’ve put my hands in.

2 Likes

No problem. And thanks.

I’ll spend a little time to make sure the policy matching works correctly.

What is happening in your case is with the automation policies. Those matchers are taken literally, since you can actually have a certificate with a subject of *.shine.caddy.test.shinenelson.xyz (a wildcard certificate), vs. "all certificates for subdomains of shine.caddy.test.shinenelson.xyz".

I suppose we could try lumping them together, i.e. doing an exact match first, and if not, then trying a wildcard match… not sure if that would break anything though… probably not?

@shine I think this commit will do what you want, however, where an automation policy can match a non-wildcard name with a wildcard subject in its list: https://github.com/caddyserver/caddy/commit/c87f82f0ce58ec714b3e13fbe69c322a0d612c67

If you could download the build artifact from that commit and try it out, that’d be much appreciated!

Hey, I’m sorry I missed this notification, I was away from the keyboard.

I’m not sure what this change is supposed to address. Is it the thing about generating “on-demand” TLS certificates even for wildcard domains like my actual intention?

I thought that that was a confusion on my part. Did you just make that a feature? :smiley:

Should I chuck the wildcard automation stuff out of the config again?

To clarify, – and summarize, – this config:

"tls" : {
      "automation" : {
        "policies" : [
          {
            "subjects" : [
              "shine.caddy.test.shinenelson.xyz",
              "*.shine.caddy.test.shinenelson.xyz"
            ],
...

used to do exact matching only. Now it matches on wildcards like TLS connection policies, even for non-wildcard certificates.

I originally assumed you were trying to do wildcard certs on-demand because that’s how the automation policy selection worked. I’ve changed it though so that now wildcards allow selecting on non-exact matches.

Wow man, this is precisely what I was asking for! This was the functionality that I had in my head when I read about the on-demand feature and the “arbitrary domains” use-case.
I love you what you did there. You just converted an out-of-scope thing into a feature.

However, I thought you said that this would have rate limit implications on Let’s Encrypt’s side right? So, would that be a good idea putting it out there?

Also, what happens to domains that would actually want a proper wildcard domain like you said?

Now, this would be good if it was limited to the internal ( SmallStep ) CA or some other provider without a rate limitation ( or something that is very, very lenient ).

My change had nothing to do with the Let’s Encrypt rate limits. Any use of on-demand is subject to abuse unless you put in protections; that’s why it’s not on by default. And if enabled, make sure to make wise use of it.

They should still be matched to the correct automation policy.

The issuer that’s used is also irrelevant, Caddy’s cert management will behave the same regardless of where it gets a cert from.

How? I don’t get it.

My scenario was that I specify *.shine.caddy.test.shinenelson.xyz in my automation policy and then when I request a.shine.caddy.test.shinenelson.xyz or b.shine.caddy.test.shinenelson.xyz, I’d get 2 separate TLS certificates per sub-domain that was being requested. And with this change, that’s exactly what it did as well.

Now, my question is - what if someone else wanted a single TLS certificate with *.shine.caddy.test.shinenelson.xyz as the SNI?

Oh, I think I get it now. My scenario would work only if on_demand is enabled right?

Otherwise, it would provision the wildcard TLS certificate during the server initialization itself ( of course, in the background! ) so then any subsequent wildcard-matching requests would pick the wildcard TLS certificate.

Good - that’s my intention as well.

But before the change, an automation policy for *.shine.caddy.test.shinenelson.xyz was literal, and would only match for those wildcard certificates.

Then they would configure Caddy’s tls app to manage a certificate for that name (or do that implicitly through auto-HTTPS in the http app).

Yes, exactly. If an exact match is not available to answer a TLS handshake, then a matching wildcard will be used instead.

Understood. This is amazing stuff man.

Now, my only wish ( I know it’s too much to ask. Hence, I’m not even asking. ) is that there are pre-built binaries in package repositories for GNU/Linux distributions ( including helper binaries for different external “modules” ), then it’d be so much more easier to integrate into automated server initialization process for thousands of servers at scale. I know there are multiple ongoing discussions on that topic as well - Packaging Caddy, V2: Packaging for Debian/Ubuntu

NB : I think this discussion has digressed away from the original topic which was for the tls_connection_policies block. Do you think we should break this post away at post 12 into a separate topic? Maybe title it to - on_demand’ wildcard TLS certificates.