How to tell caddy to use a wildcard cert instead of requesting a new cert?

1. Caddy version (caddy version):

latest

2. How I run Caddy:

In a docker container

a. System environment:

linux

b. Command:

paste command here

c. Service/unit/compose file:

paste full file contents here

d. My complete Caddyfile or JSON config:

I have a very large caddy file. (Over 80 SNIs are specified…)

    http_port: 8080
    https_port: 8443
    servers:
      srv0:
        listen:
          - :8443
        routes:
          - match:
              - host:
                  - accounts-customer.secondlife.{{$.Env.HOST}}
                  - accounts-customer-sl.{{$.Env.HOST}}

(many others omitted with very different configs for many of the services...)

3. The problem I’m having:

I’m wanting to get a real real certs to make our life easier. However, we have so many host names we set up on our sandbox environments we quickly blow past Let’sEncrypt rate limits.

However, ALL the SNIs are subdomains of the developers sandbox machine. So we would like to get a wildcard cert. The part of the config doing that is here:

  tls:
    automation:
      policies:
        - subjects:
            - '*.{{$.Env.HOST}}.dev.tilia-inc.com'
          issuers:
            - module: internal

The current set up is just asking for an internal cert while I figure this out… eventually this will use route53.

However, I do not understand how to tell caddy to use the wildcard cert for a particular SNI.

If I manually load the certs - like this:

    certificates:
      load_files:
        - certificate: /certs/accounts-customer-sl.{{$.Env.HOST}}.crt
          key: /certs/accounts-customer-sl.{{$.Env.HOST}}.key
          tags:
            - cert2

thien I can use a policy to tie it to a that cert. However, it does not appear there is a way to tag managed certs?

4. Error messages and/or full log output:

Currently it does not match the wildcard cert and instead - attempts to get a cert from LE.

5. What I already tried:

Tried to do this in a Caddyfile until I found out it will never work there…

Moving to yaml. But now stuck again… really not sure what to try next…

I think all I need is a way to have managed certs to have tags. But I failed to see any mention of tags in the docs for managed certs…

6. Links to relevant resources:

Note: I should mention SOME of the SNIs we configure also use a client_auth cert as well.

        tls_connection_policies:
          - match:
              sni:
                - accounts-customer.secondlife.{{$.Env.HOST}}
                - accounts-customer-sl.{{$.Env.HOST}}
            certificate_selection:
              any_tag:
                - cert2
            client_authentication:
              trusted_ca_certs_pem_files:
                - /certs/ca.crt
              mode: verify_if_given

I will still need to do that as well.

Maybe I’m misunderstanding, but does a pattern like this not work for you?

*.example.com {
	tls internal

	@foo host foo.example.com
	handle @foo {
		respond "foo"
	}

	handle {
		respond "unknown"
	}
}

I don’t think that will work for me. Wouldn’t your version only create one tls_connection_policy?

Unless this somehow worked:

*.example.com {
	tls internal

	@foo host foo.example.com
	handle @foo {
        # Some some sites also need client certs, others do not
        tls /certs/foo.crt /certs/foo.key {
            client_auth {
                mode verify_if_given
                trusted_ca_cert_file /certs/ca.crt
            }
        }
		respond "foo"
	}

	@bar host bar.example.com
	handle @bar {
		respond "bar"
	}

	handle {
		respond "unknown"
	}
}

Because I have some subdomains that need to use specific certs for client_auth, and others that should not have it.

I.e. I think my tls_connection_policies need to look like this:

        tls_connection_policies:
          - match:
              sni:
                - foo.example.com
                - foo2.example.com
            certificate_selection:
              any_tag:
                - wildcard_cert
            client_authentication:
              trusted_ca_certs_pem_files:
                - /certs/ca.crt
              mode: verify_if_given
          - match:
              sni:
                - bar.example.com
                - other.example.com
            certificate_selection:
              any_tag:
                - wildcerd_cert

@matt you had suggested to me in a different thread that I needed to use JSON config for this. Can you weigh in on this?

I know have my config in json - but I do not see how to associate specific site configs to the managed wildcard ceert. I totally see how I could do it with tags for an unmanaged wildcard cert I loaded. But there is no tags for managed certs…

I basically getting the same behavior as in the Caddyfile version. A given SNI does not match the wildcard cert and instead it attempts to request one from LE. (Which is a problem due to how many I have…)

@francislavoie I replied why I did not think your solution would not work for my use case. Do you have any thoughts?

The only solution I can think of right now is to use a different caddy server to get the managed wildcard cert. Then somehow export that cert out (though I do not know how at the moment) and then load it manually in my real config.

Unfortunately I don’t. I’ll need to let @matt chime in here. Even I have trouble with configuring the TLS app via JSON. I’m more of a Caddyfile expert :slight_smile:

@matt I found a different forum thread where you talk about how you added the code to match to the wildcard cert. So it seems I should not NEED tags as I mentioned above. Here is the relevant part of my json.config (with route53 config):

        tls_connection_policies:
          - match:
              sni:
                - discord-accounts-ops-api.rayj2.dev.tilia-inc.com
                - subscriptions-ops-api.rayj2.dev.tilia-inc.com
                - registration-ops-api.rayj2.dev.tilia-inc.com
                - invoicing-ops-api.rayj2.dev.tilia-inc.com
                - subscriptions-api.rayj2.dev.tilia-inc.com
                - accounts-ops-api.rayj2.dev.tilia-inc.com
                - email-bouncelist.rayj2.dev.tilia-inc.com
                - exchange-ops-api.rayj2.dev.tilia-inc.com
                - payments-ops-api.rayj2.dev.tilia-inc.com
                - personas-ops-api.rayj2.dev.tilia-inc.com
                - registration-api.rayj2.dev.tilia-inc.com
                - graphql-ops-api.rayj2.dev.tilia-inc.com
                - wallets-ops-api.rayj2.dev.tilia-inc.com
                - email-service.rayj2.dev.tilia-inc.com
                - fraud-ops-api.rayj2.dev.tilia-inc.com
                - invoicing-api.rayj2.dev.tilia-inc.com
                - proxy-service.rayj2.dev.tilia-inc.com
                - tools-ops-api.rayj2.dev.tilia-inc.com
                - transfers-api.rayj2.dev.tilia-inc.com
                - accounts-api.rayj2.dev.tilia-inc.com
                - exchange-api.rayj2.dev.tilia-inc.com
                - payments-api.rayj2.dev.tilia-inc.com
                - personas-api.rayj2.dev.tilia-inc.com
                - pii-ops-api.rayj2.dev.tilia-inc.com
                - wallets-api.rayj2.dev.tilia-inc.com
                - fraud-api.rayj2.dev.tilia-inc.com
                - tools-api.rayj2.dev.tilia-inc.com
                - tools-web.rayj2.dev.tilia-inc.com
                - pii-api.rayj2.dev.tilia-inc.com
                - nonce.rayj2.dev.tilia-inc.com
                - auth.rayj2.dev.tilia-inc.com
            client_authentication:
              trusted_ca_certs_pem_files:
                - /certs/ca.crt
              mode: verify_if_given
              # mode: require_and_verify
          - match:
              sni:
                - kinesis-lambda.rayj2.dev.tilia-inc.com
                - sl-login.rayj2.dev.tilia-inc.com
                - return-generator.rayj2.dev.tilia-inc.com
                - moov-ofac-admin.rayj2.dev.tilia-inc.com
                - moov-fed-admin.rayj2.dev.tilia-inc.com
                - moov-accounts-admin.rayj2.dev.tilia-inc.com
                - moov-paygate-admin.rayj2.dev.tilia-inc.com
                - moov-ach-admin.rayj2.dev.tilia-inc.com
                - moov-accounts.rayj2.dev.tilia-inc.com
                - moov-paygate.rayj2.dev.tilia-inc.com
                - moov-ofac.rayj2.dev.tilia-inc.com
                - moov-ach.rayj2.dev.tilia-inc.com
                - moov-fed.rayj2.dev.tilia-inc.com
                - accounts-customer.secondlife.rayj2.dev.tilia-inc.com
                - accounts-customer-sl.rayj2.dev.tilia-inc.com
                - kinesis-bouncelist-lambda.rayj2.dev.tilia-inc.com
                - fake-promise-integrator.rayj2.dev.tilia-inc.com
                - accounts-customer-web.rayj2.dev.tilia-inc.com
                - accounts-customer.rayj2.dev.tilia-inc.com
                - exchange.rayj2.dev.tilia-inc.com
                - exchange-frontend.rayj2.dev.tilia-inc.com
                - fake-integrator-web.rayj2.dev.tilia-inc.com
                - fake-integrator2.rayj2.dev.tilia-inc.com
                - fake-integrator.rayj2.dev.tilia-inc.com
                - lambda-server.rayj2.dev.tilia-inc.com
                - elasticsearch.rayj2.dev.tilia-inc.com
                - localstack.rayj2.dev.tilia-inc.com
                - tools.rayj2.dev.tilia-inc.com
                - login.rayj2.dev.tilia-inc.com
                - login.sansar.com
                - logs.rayj2.dev.tilia-inc.com
                - www.rayj2.dev.tilia-inc.com
                - web.rayj2.dev.tilia-inc.com
          - {}
  tls:
    automation:
      policies:
        - subjects:
            - '*.rayj2.dev.tilia-inc.com'
          issuers:
            - ca: https://acme-staging-v02.api.letsencrypt.org/directory
              challenges:
                dns:
                  provider:
                    name: route53
              module: acme

However, in the logs. It sure looks like it trying to get new certs for most of these domains:

{"level":"debug","ts":1611329609.6023097,"logger":"http","msg":"starting server loop","address":"[::]:8080","http3":false,"tls":false}
{"level":"info","ts":1611329609.6071563,"logger":"http","msg":"enabling automatic TLS certificate management","domains":["pii-api.rayj2.dev.tilia-inc.com","fake-integrator2.rayj2.dev.tilia-inc.com","email-service.rayj2.dev.tilia-inc.com","accounts-customer-sl.rayj2.dev.tilia-inc.com","tools-ops-api.rayj2.dev.tilia-inc.com","accounts-api.rayj2.dev.tilia-inc.com","fake-promise-integrator.rayj2.dev.tilia-inc.com","exchange-ops-api.rayj2.dev.tilia-inc.com","tools-api.rayj2.dev.tilia-inc.com","login.rayj2.dev.tilia-inc.com","kinesis-bouncelist-lambda.rayj2.dev.tilia-inc.com","auth.rayj2.dev.tilia-inc.com","invoicing-api.rayj2.dev.tilia-inc.com","personas-ops-api.rayj2.dev.tilia-inc.com","accounts-ops-api.rayj2.dev.tilia-inc.com","moov-fed-admin.rayj2.dev.tilia-inc.com","graphql-ops-api.rayj2.dev.tilia-inc.com","www.rayj2.dev.tilia-inc.com","moov-ofac.rayj2.dev.tilia-inc.com","accounts-customer.secondlife.rayj2.dev.tilia-inc.com","fake-integrator-web.rayj2.dev.tilia-inc.com","tools-web.rayj2.dev.tilia-inc.com","return-generator.rayj2.dev.tilia-inc.com","fraud-ops-api.rayj2.dev.tilia-inc.com","registration-api.rayj2.dev.tilia-inc.com","moov-ach.rayj2.dev.tilia-inc.com","payments-api.rayj2.dev.tilia-inc.com","registration-ops-api.rayj2.dev.tilia-inc.com","lambda-server.rayj2.dev.tilia-inc.com","moov-paygate.rayj2.dev.tilia-inc.com","subscriptions-ops-api.rayj2.dev.tilia-inc.com","logs.rayj2.dev.tilia-inc.com","kinesis-lambda.rayj2.dev.tilia-inc.com","wallets-api.rayj2.dev.tilia-inc.com","transfers-api.rayj2.dev.tilia-inc.com","proxy-service.rayj2.dev.tilia-inc.com","personas-api.rayj2.dev.tilia-inc.com","email-bouncelist.rayj2.dev.tilia-inc.com","login.sansar.com","moov-ofac-admin.rayj2.dev.tilia-inc.com","moov-accounts-admin.rayj2.dev.tilia-inc.com","web.rayj2.dev.tilia-inc.com","payments-ops-api.rayj2.dev.tilia-inc.com","exchange-api.rayj2.dev.tilia-inc.com","elasticsearch.rayj2.dev.tilia-inc.com","tools.rayj2.dev.tilia-inc.com","accounts-customer.rayj2.dev.tilia-inc.com","exchange.rayj2.dev.tilia-inc.com","moov-ach-admin.rayj2.dev.tilia-inc.com","invoicing-ops-api.rayj2.dev.tilia-inc.com","moov-fed.rayj2.dev.tilia-inc.com","localstack.rayj2.dev.tilia-inc.com","moov-paygate-admin.rayj2.dev.tilia-inc.com","moov-accounts.rayj2.dev.tilia-inc.com","pii-ops-api.rayj2.dev.tilia-inc.com","discord-accounts-ops-api.rayj2.dev.tilia-inc.com","wallets-ops-api.rayj2.dev.tilia-inc.com","sl-login.rayj2.dev.tilia-inc.com","exchange-frontend.rayj2.dev.tilia-inc.com","nonce.rayj2.dev.tilia-inc.com","fake-integrator.rayj2.dev.tilia-inc.com","fraud-api.rayj2.dev.tilia-inc.com","subscriptions-api.rayj2.dev.tilia-inc.com","accounts-customer-web.rayj2.dev.tilia-inc.com"]}
{"level":"info","ts":1611329609.618157,"logger":"tls.obtain","msg":"acquiring lock","identifier":"pii-api.rayj2.dev.tilia-inc.com"}
{"level":"info","ts":1611329609.618968,"logger":"tls.obtain","msg":"lock acquired","identifier":"pii-api.rayj2.dev.tilia-inc.com"}
...

Eventually, I’m getting entries like this:

{"level":"debug","ts":1611329613.0725164,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"HEAD","url":"https://acme-staging-v02.api.letsencrypt.org/acme/new-nonce","headers":{"User-Agent":["Caddy/2.3.0 CertMagic acmez (linux; amd64)"]},"status_code":200,"response_headers":{"Cache-Control":["public, max-age=0, no-cache"],"Date":["Fri, 22 Jan 2021 15:33:33 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["0004YkOGPd0j_kCsqoxEwhFmuZdO0OZnJekha8ueNo0r5Vk"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]}}
{"level":"debug","ts":1611329613.0856571,"logger":"http.stdlib","msg":"http: TLS handshake error from 10.132.6.97:57896: no certificate available for 'fraud-ops-api.rayj2.dev.tilia-inc.com'"}
{"level":"debug","ts":1611329613.2287264,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-staging-v02.api.letsencrypt.org/acme/new-order","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.3.0 CertMagic acmez (linux; amd64)"]},"status_code":201,"response_headers":{"Boulder-Requester":["17628814"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["374"],"Content-Type":["application/json"],"Date":["Fri, 22 Jan 2021 15:33:33 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Location":["https://acme-staging-v02.api.letsencrypt.org/acme/order/17628814/226610735"],"Replay-Nonce":["0004OFaVwtjcdf2YhSSrILJSed3aI6YUq3MYmcraSqVmUVw"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]}}

Which sure looks like it trying to get certs from LE for each SNI.

The log is rather large. But I do not even see it attempting to acquire the wildcard cert…

Do you have a really basic JSON config that reproduces the behavior you’re seeing? Without any templating or extra stuff. Just the most basic possible. I’ll take a look into it when I have a chance.

Sure - here is a basic config with what I’m trying to do:

logging:
  logs:
    default:
      level: DEBUG
apps:
  http:
    http_port: 80
    https_port: 443
    servers:
      srv0:
        listen:
          - :8443
        routes:
          - match:
              - host:
                  - foo.rayj2.dev.tilia-inc.com
            handle:
              - handler: subroute
                routes:
                  - handle:
                      - handler: reverse_proxy
                        upstreams:
                          - dial: foo:80
            terminal: true
          - match:
              - host:
                  - bar.rayj2.dev.tilia-inc.com
            handle:
              - handler: subroute
                routes:
                  - handle:
                      - handler: reverse_proxy
                        upstreams:
                          - dial: bar:80
            terminal: true
        tls_connection_policies:
          - match:
              sni:
                - foo.rayj2.dev.tilia-inc.com
            client_authentication:
              trusted_ca_certs_pem_files:
                - /certs/ca.crt
              mode: require_and_verify
          - match:
              sni:
                - bar.rayj2.dev.tilia-inc.com
          - {}
  tls:
    automation:
      policies:
        - subjects:
            - '*.rayj2.dev.tilia-inc.com'
          issuers:
            - module: acme
              ca: https://acme-staging-v02.api.letsencrypt.org/directory
              challenges:
                dns:
                  provider:
                    name: route53
        - subjects:
            - 'login.sansar.com'
          issuers:
            - module: internal

And here is the log output from caddy:

root@5c24d5340a96:/site# caddy run --config /config.yml --adapter yaml
2021/01/24 08:49:51.330	INFO	using provided configuration	{"config_file": "/config.yml", "config_adapter": "yaml"}
2021/01/24 08:49:51.332	INFO	admin	admin endpoint started	{"address": "tcp/localhost:2019", "enforce_origin": false, "origins": ["localhost:2019", "[::1]:2019", "127.0.0.1:2019"]}
2021/01/24 08:49:51.332	INFO	tls.cache.maintenance	started background certificate maintenance	{"cache": "0xc00017e620"}
2021/01/24 08:49:51.344	INFO	http	enabling automatic HTTP->HTTPS redirects	{"server_name": "srv0"}
2021/01/24 08:49:51.345	INFO	http	enabling strict SNI-Host matching because TLS client auth is configured	{"server_name": "srv0"}
2021/01/24 08:49:51.379	WARN	pki.ca.local	installing root certificate (you might be prompted for password)	{"path": "storage:pki/authorities/local/root.crt"}
2021/01/24 08:49:51 Warning: "certutil" is not available, install "certutil" with "apt install libnss3-tools" or "yum install nss-tools" and try again
2021/01/24 08:49:51 define JAVA_HOME environment variable to use the Java trust
2021/01/24 08:49:51 certificate installed properly in linux trusts
2021/01/24 08:49:51.983	DEBUG	http	starting server loop	{"address": "[::]:8443", "http3": false, "tls": true}
2021/01/24 08:49:51.983	DEBUG	http	starting server loop	{"address": "[::]:80", "http3": false, "tls": false}
2021/01/24 08:49:51.983	INFO	http	enabling automatic TLS certificate management	{"domains": ["foo.rayj2.dev.tilia-inc.com", "bar.rayj2.dev.tilia-inc.com"]}
2021/01/24 08:49:51.984	INFO	tls	cleaned up storage units
2021/01/24 08:49:51.984	INFO	tls.obtain	acquiring lock	{"identifier": "foo.rayj2.dev.tilia-inc.com"}
2021/01/24 08:49:51.986	INFO	tls.obtain	lock acquired	{"identifier": "foo.rayj2.dev.tilia-inc.com"}
2021/01/24 08:49:51.987	INFO	tls.obtain	acquiring lock	{"identifier": "bar.rayj2.dev.tilia-inc.com"}
2021/01/24 08:49:51.987	INFO	tls.obtain	lock acquired	{"identifier": "bar.rayj2.dev.tilia-inc.com"}
2021/01/24 08:49:51.986	INFO	autosaved config	{"file": "/root/.config/caddy/autosave.json"}
2021/01/24 08:49:51.989	INFO	serving initial configuration
2021/01/24 08:49:52.162	DEBUG	tls.issuance.acme.acme_client	http request	{"method": "GET", "url": "https://acme-staging-v02.api.letsencrypt.org/directory", "headers": {"User-Agent":["Caddy/2.3.0 CertMagic acmez (linux; amd64)"]}, "status_code": 200, "response_headers": {"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["724"],"Content-Type":["application/json"],"Date":["Sun, 24 Jan 2021 08:49:52 GMT"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]}}
2021/01/24 08:49:52.212	DEBUG	tls.issuance.acme.acme_client	http request	{"method": "HEAD", "url": "https://acme-staging-v02.api.letsencrypt.org/acme/new-nonce", "headers": {"User-Agent":["Caddy/2.3.0 CertMagic acmez (linux; amd64)"]}, "status_code": 200, "response_headers": {"Cache-Control":["public, max-age=0, no-cache"],"Date":["Sun, 24 Jan 2021 08:49:52 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["0003xTBQee7kPGn5hgXL19BQOikYixGkWmNkXGNAniuK1c4"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]}}
2021/01/24 08:49:52.276	DEBUG	tls.issuance.acme.acme_client	http request	{"method": "POST", "url": "https://acme-staging-v02.api.letsencrypt.org/acme/new-acct", "headers": {"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.3.0 CertMagic acmez (linux; amd64)"]}, "status_code": 201, "response_headers": {"Boulder-Requester":["17685901"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["285"],"Content-Type":["application/json"],"Date":["Sun, 24 Jan 2021 08:49:52 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\"","<https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf>;rel=\"terms-of-service\""],"Location":["https://acme-staging-v02.api.letsencrypt.org/acme/acct/17685901"],"Replay-Nonce":["0004AHPNHGi32gGVVo_p-m9tm_O20AJZzDC1_mXfiOo7dXg"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]}}
2021/01/24 08:49:52.277	INFO	tls.issuance.acme	waiting on internal rate limiter	{"identifiers": ["foo.rayj2.dev.tilia-inc.com"]}
2021/01/24 08:49:52.277	INFO	tls.issuance.acme	done waiting on internal rate limiter	{"identifiers": ["foo.rayj2.dev.tilia-inc.com"]}
2021/01/24 08:49:52.337	DEBUG	tls.issuance.acme.acme_client	http request	{"method": "HEAD", "url": "https://acme-staging-v02.api.letsencrypt.org/acme/new-nonce", "headers": {"User-Agent":["Caddy/2.3.0 CertMagic acmez (linux; amd64)"]}, "status_code": 200, "response_headers": {"Cache-Control":["public, max-age=0, no-cache"],"Date":["Sun, 24 Jan 2021 08:49:52 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["0004p5j8vwULmD8emd1hii6jK5KCfJjPUqKxrvBDduVMLb8"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]}}
2021/01/24 08:49:52.342	DEBUG	tls.issuance.acme.acme_client	http request	{"method": "POST", "url": "https://acme-staging-v02.api.letsencrypt.org/acme/new-order", "headers": {"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.3.0 CertMagic acmez (linux; amd64)"]}, "status_code": 201, "response_headers": {"Boulder-Requester":["17685901"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["361"],"Content-Type":["application/json"],"Date":["Sun, 24 Jan 2021 08:49:52 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Location":["https://acme-staging-v02.api.letsencrypt.org/acme/order/17685901/227637346"],"Replay-Nonce":["0003ZIV1v4diIM00K4uQGNUwpF-HSCEud_PHYDSVspfQfwo"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]}}
2021/01/24 08:49:52.396	DEBUG	tls.issuance.acme.acme_client	http request	{"method": "POST", "url": "https://acme-staging-v02.api.letsencrypt.org/acme/new-acct", "headers": {"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.3.0 CertMagic acmez (linux; amd64)"]}, "status_code": 201, "response_headers": {"Boulder-Requester":["17685902"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["285"],"Content-Type":["application/json"],"Date":["Sun, 24 Jan 2021 08:49:52 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\"","<https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf>;rel=\"terms-of-service\""],"Location":["https://acme-staging-v02.api.letsencrypt.org/acme/acct/17685902"],"Replay-Nonce":["00047DRhs7To1XoPTLgyOjDxgYeAKJeja2YSlFrx_8F-Hw8"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]}}
2021/01/24 08:49:52.396	INFO	tls.issuance.acme	waiting on internal rate limiter	{"identifiers": ["bar.rayj2.dev.tilia-inc.com"]}
2021/01/24 08:49:52.397	INFO	tls.issuance.acme	done waiting on internal rate limiter	{"identifiers": ["bar.rayj2.dev.tilia-inc.com"]}
2021/01/24 08:49:52.398	DEBUG	tls.issuance.acme.acme_client	http request	{"method": "POST", "url": "https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/197501389", "headers": {"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.3.0 CertMagic acmez (linux; amd64)"]}, "status_code": 200, "response_headers": {"Boulder-Requester":["17685901"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["826"],"Content-Type":["application/json"],"Date":["Sun, 24 Jan 2021 08:49:52 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["0003PR0mIYSNHeny5qSbtvz2SFPre8Hgkz-jM0VISm2Ml-4"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]}}
2021/01/24 08:49:52.398	INFO	tls.issuance.acme.acme_client	trying to solve challenge	{"identifier": "foo.rayj2.dev.tilia-inc.com", "challenge_type": "dns-01", "ca": "https://acme-staging-v02.api.letsencrypt.org/directory"}
2021/01/24 08:49:52.466	DEBUG	tls.issuance.acme.acme_client	http request	{"method": "POST", "url": "https://acme-staging-v02.api.letsencrypt.org/acme/new-order", "headers": {"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.3.0 CertMagic acmez (linux; amd64)"]}, "status_code": 201, "response_headers": {"Boulder-Requester":["17685902"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["361"],"Content-Type":["application/json"],"Date":["Sun, 24 Jan 2021 08:49:52 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Location":["https://acme-staging-v02.api.letsencrypt.org/acme/order/17685902/227637347"],"Replay-Nonce":["0004JuYj94Uy3bB8jfeGeEA3afLhtm0N3nWJk5zJsScWBa4"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]}}
2021/01/24 08:49:52.513	DEBUG	tls.issuance.acme.acme_client	http request	{"method": "POST", "url": "https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/197501390", "headers": {"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.3.0 CertMagic acmez (linux; amd64)"]}, "status_code": 200, "response_headers": {"Boulder-Requester":["17685902"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["826"],"Content-Type":["application/json"],"Date":["Sun, 24 Jan 2021 08:49:52 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["0004kscA4NFftc5J6FtB603qoZWFDKkNE-23wMNyMdkF0ug"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]}}
2021/01/24 08:49:52.513	INFO	tls.issuance.acme.acme_client	trying to solve challenge	{"identifier": "bar.rayj2.dev.tilia-inc.com", "challenge_type": "dns-01", "ca": "https://acme-staging-v02.api.letsencrypt.org/directory"}
2021/01/24 08:49:56.347	ERROR	tls.issuance.acme.acme_client	cleaning up solver	{"identifier": "foo.rayj2.dev.tilia-inc.com", "challenge_type": "dns-01", "error": "no memory of presenting a DNS record for foo.rayj2.dev.tilia-inc.com (probably OK if presenting failed)"}
2021/01/24 08:49:56.395	DEBUG	tls.issuance.acme.acme_client	http request	{"method": "POST", "url": "https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/197501389", "headers": {"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.3.0 CertMagic acmez (linux; amd64)"]}, "status_code": 200, "response_headers": {"Boulder-Requester":["17685901"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["830"],"Content-Type":["application/json"],"Date":["Sun, 24 Jan 2021 08:49:56 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["0003zWjRHKdHOvM-Z3OygFGWMOx-bvKZ2yQgLgc0xj8uDYA"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]}}
2021/01/24 08:49:56.395	ERROR	tls.obtain	will retry	{"error": "[foo.rayj2.dev.tilia-inc.com] Obtain: [foo.rayj2.dev.tilia-inc.com] solving challenges: presenting for challenge: adding temporary record for zone dev.tilia-inc.com.: InvalidChangeBatch: InvalidChangeBatch: [Tried to create resource record set [name='_acme-challenge.foo.rayj2.dev.tilia-inc.com.', type='TXT'] but it already exists]\n\tstatus code: 400, request id: 4d186c0c-ac7a-4f7d-be48-cf5914cc972f (order=https://acme-staging-v02.api.letsencrypt.org/acme/order/17685901/227637346) (ca=https://acme-staging-v02.api.letsencrypt.org/directory)", "attempt": 1, "retrying_in": 60, "elapsed": 4.409548965, "max_duration": 2592000}
2021/01/24 08:49:56.413	ERROR	tls.issuance.acme.acme_client	cleaning up solver	{"identifier": "bar.rayj2.dev.tilia-inc.com", "challenge_type": "dns-01", "error": "no memory of presenting a DNS record for bar.rayj2.dev.tilia-inc.com (probably OK if presenting failed)"}
2021/01/24 08:49:56.461	DEBUG	tls.issuance.acme.acme_client	http request	{"method": "POST", "url": "https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/197501390", "headers": {"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.3.0 CertMagic acmez (linux; amd64)"]}, "status_code": 200, "response_headers": {"Boulder-Requester":["17685902"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["830"],"Content-Type":["application/json"],"Date":["Sun, 24 Jan 2021 08:49:56 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["0004PzsuZkjlwXzsTN2vb7_7ZIuCdZ2SaSwgyDaKrgDugzY"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]}}
2021/01/24 08:49:56.462	ERROR	tls.obtain	will retry	{"error": "[bar.rayj2.dev.tilia-inc.com] Obtain: [bar.rayj2.dev.tilia-inc.com] solving challenges: presenting for challenge: adding temporary record for zone dev.tilia-inc.com.: InvalidChangeBatch: InvalidChangeBatch: [Tried to create resource record set [name='_acme-challenge.bar.rayj2.dev.tilia-inc.com.', type='TXT'] but it already exists]\n\tstatus code: 400, request id: 03dc86e4-35b6-41e9-9e3a-73552783e870 (order=https://acme-staging-v02.api.letsencrypt.org/acme/order/17685902/227637347) (ca=https://acme-staging-v02.api.letsencrypt.org/directory)", "attempt": 1, "retrying_in": 60, "elapsed": 4.473865753, "max_duration": 2592000}
2021/01/24 08:50:56.579	DEBUG	tls.issuance.acme.acme_client	http request	{"method": "HEAD", "url": "https://acme-staging-v02.api.letsencrypt.org/acme/new-nonce", "headers": {"User-Agent":["Caddy/2.3.0 CertMagic acmez (linux; amd64)"]}, "status_code": 200, "response_headers": {"Cache-Control":["public, max-age=0, no-cache"],"Date":["Sun, 24 Jan 2021 08:50:56 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["0004eFIq0KTrF8PYh5Z_KZILlDP3-RBNMl9u-lgAcpk0CYA"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]}}
2021/01/24 08:50:56.645	DEBUG	tls.issuance.acme.acme_client	http request	{"method": "POST", "url": "https://acme-staging-v02.api.letsencrypt.org/acme/new-order", "headers": {"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.3.0 CertMagic acmez (linux; amd64)"]}, "status_code": 201, "response_headers": {"Boulder-Requester":["17685902"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["361"],"Content-Type":["application/json"],"Date":["Sun, 24 Jan 2021 08:50:56 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Location":["https://acme-staging-v02.api.letsencrypt.org/acme/order/17685902/227637851"],"Replay-Nonce":["0004Lm-Nem_QezDRTCRVPK6D9Qzdwr2nLZwEW6ZzLMlZcZg"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]}}
2021/01/24 08:50:56.673	DEBUG	tls.issuance.acme.acme_client	http request	{"method": "HEAD", "url": "https://acme-staging-v02.api.letsencrypt.org/acme/new-nonce", "headers": {"User-Agent":["Caddy/2.3.0 CertMagic acmez (linux; amd64)"]}, "status_code": 200, "response_headers": {"Cache-Control":["public, max-age=0, no-cache"],"Date":["Sun, 24 Jan 2021 08:50:56 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["0004m_EtJY5s0yQMvWBgIjzlcmUoIagA1EiCcHSqY7caFUo"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]}}
2021/01/24 08:50:56.726	DEBUG	tls.issuance.acme.acme_client	http request	{"method": "POST", "url": "https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/197501829", "headers": {"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.3.0 CertMagic acmez (linux; amd64)"]}, "status_code": 200, "response_headers": {"Boulder-Requester":["17685902"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["826"],"Content-Type":["application/json"],"Date":["Sun, 24 Jan 2021 08:50:56 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["0004pjKFoJuYSLgfgdsCbGxst4HG60nBW_m9RqyDNYuplRw"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]}}
2021/01/24 08:50:56.726	INFO	tls.issuance.acme.acme_client	trying to solve challenge	{"identifier": "foo.rayj2.dev.tilia-inc.com", "challenge_type": "dns-01", "ca": "https://acme-staging-v02.api.letsencrypt.org/directory"}
2021/01/24 08:50:56.798	DEBUG	tls.issuance.acme.acme_client	http request	{"method": "POST", "url": "https://acme-staging-v02.api.letsencrypt.org/acme/new-order", "headers": {"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.3.0 CertMagic acmez (linux; amd64)"]}, "status_code": 201, "response_headers": {"Boulder-Requester":["17685902"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["361"],"Content-Type":["application/json"],"Date":["Sun, 24 Jan 2021 08:50:56 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Location":["https://acme-staging-v02.api.letsencrypt.org/acme/order/17685902/227637852"],"Replay-Nonce":["00043LybTHKBo2mayzXwVOxlEqCybUV3bmlFfhJfNY32nWA"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]}}
2021/01/24 08:50:56.847	DEBUG	tls.issuance.acme.acme_client	http request	{"method": "POST", "url": "https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/197501830", "headers": {"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.3.0 CertMagic acmez (linux; amd64)"]}, "status_code": 200, "response_headers": {"Boulder-Requester":["17685902"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["826"],"Content-Type":["application/json"],"Date":["Sun, 24 Jan 2021 08:50:56 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["0004Kq2Xc3UjLAMfsa-TI0y3vP0BGIe5jSnotCzeB176AVg"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]}}
2021/01/24 08:50:56.847	INFO	tls.issuance.acme.acme_client	trying to solve challenge	{"identifier": "bar.rayj2.dev.tilia-inc.com", "challenge_type": "dns-01", "ca": "https://acme-staging-v02.api.letsencrypt.org/directory"}
2021/01/24 08:50:57.457	ERROR	tls.issuance.acme.acme_client	cleaning up solver	{"identifier": "foo.rayj2.dev.tilia-inc.com", "challenge_type": "dns-01", "error": "no memory of presenting a DNS record for foo.rayj2.dev.tilia-inc.com (probably OK if presenting failed)"}
2021/01/24 08:50:57.511	DEBUG	tls.issuance.acme.acme_client	http request	{"method": "POST", "url": "https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/197501829", "headers": {"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.3.0 CertMagic acmez (linux; amd64)"]}, "status_code": 200, "response_headers": {"Boulder-Requester":["17685902"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["830"],"Content-Type":["application/json"],"Date":["Sun, 24 Jan 2021 08:50:57 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["0003qqUEX79S4ByvFSozwRynljn_d2qAGPRD_9PNXrgCIT0"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]}}
2021/01/24 08:50:57.511	ERROR	tls.obtain	will retry	{"error": "[foo.rayj2.dev.tilia-inc.com] Obtain: [foo.rayj2.dev.tilia-inc.com] solving challenges: presenting for challenge: adding temporary record for zone dev.tilia-inc.com.: InvalidChangeBatch: InvalidChangeBatch: [Tried to create resource record set [name='_acme-challenge.foo.rayj2.dev.tilia-inc.com.', type='TXT'] but it already exists]\n\tstatus code: 400, request id: e7ca206d-72b3-423c-a379-eaabeddd269a (order=https://acme-staging-v02.api.letsencrypt.org/acme/order/17685902/227637851) (ca=https://acme-staging-v02.api.letsencrypt.org/directory)", "attempt": 2, "retrying_in": 120, "elapsed": 65.525519769, "max_duration": 2592000}
2021/01/24 08:50:57.569	ERROR	tls.issuance.acme.acme_client	cleaning up solver	{"identifier": "bar.rayj2.dev.tilia-inc.com", "challenge_type": "dns-01", "error": "no memory of presenting a DNS record for bar.rayj2.dev.tilia-inc.com (probably OK if presenting failed)"}
2021/01/24 08:50:57.619	DEBUG	tls.issuance.acme.acme_client	http request	{"method": "POST", "url": "https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/197501830", "headers": {"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.3.0 CertMagic acmez (linux; amd64)"]}, "status_code": 200, "response_headers": {"Boulder-Requester":["17685902"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["830"],"Content-Type":["application/json"],"Date":["Sun, 24 Jan 2021 08:50:57 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["0003p0rQS9pHIABmKNI0A4Dv1MtX4ET0RVtuAknSFX9ut7Y"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]}}
2021/01/24 08:50:57.619	ERROR	tls.obtain	will retry	{"error": "[bar.rayj2.dev.tilia-inc.com] Obtain: [bar.rayj2.dev.tilia-inc.com] solving challenges: presenting for challenge: adding temporary record for zone dev.tilia-inc.com.: InvalidChangeBatch: InvalidChangeBatch: [Tried to create resource record set [name='_acme-challenge.bar.rayj2.dev.tilia-inc.com.', type='TXT'] but it already exists]\n\tstatus code: 400, request id: c19c9838-50ed-4899-8fc7-85e57aea744f (order=https://acme-staging-v02.api.letsencrypt.org/acme/order/17685902/227637852) (ca=https://acme-staging-v02.api.letsencrypt.org/directory)", "attempt": 2, "retrying_in": 120, "elapsed": 65.631656277, "max_duration": 2592000}

From those logs it sure looks like it is trying to allocate foo.rayj2.dev.tilia-inc.com, bar.rayj2.dev.tilia-inc.com - but never *.rayj2.dev.tilia-inc.com

Thanks – that’s helpful.

I suppose this section in autohttps.go could be modified to support wildcard matching, rather than just exact string comparison:

L258 should probably be something like:

if certmagic.MatchWildcard(d, apHost) {

In fact, if you have a few moments today and can make that change and try it out, that would be really awesome!

1 Like

Ok - I tried it out. Did not seem to make a difference.

I’m wondering if maybe that doesn’t work because the wildcard cert itself has not been provisioned by the time it is trying to match?

Hmm, no, because it’s in the automation policies; shouldn’t need to be in the cache yet. I’ll take a closer look at this today.

Okay, I was wrong, as usual. :slight_smile:

Now that I’ve got my head on straight and it’s not early in the morning, here’s an explanation of what’s going on:

The reason your config isn’t getting a cert for *.rayj2.dev.tilia-inc.com is simply because you didn’t tell it to. :man_shrugging: An automation policy just tells Caddy how to manage a certificate, not to manage it.

You can do this a couple of ways, for example, adding it to your hosts matcher somewhere, or adding certificates to your tls app config:

"certificates": {
	"automate": ["*.rayj2.dev.tilia-inc.com"]
},

(I guess in your case, you’d want the YAML equivalent.)

Caddy will still try to get certs for foo.rayj2.dev.tilia-inc.com because you have automatic HTTPS enabled. I suppose you’ll want to simply disable it entirely. You don’t need to enumerate all the domains. Set disabled: true here: JSON Config Structure - Caddy Documentation

You’ll just want to set up the redirects yourself, but that’s really easy.

Yes! With those changes I got everything working. Thank you!!!

BTW, I still have the code change in there. I still need to try things without it. Do you believe the code change is still needed? If so, any idea when a release will happen with that in it?

I am pretty sure the code change is not needed.

This topic was automatically closed after 30 days. New replies are no longer allowed.