Unable to retrieve wildcard cert for second-level subdomain

1. The problem I’m having:

I am trying to set up a wildcard certificate for a second level subdomain. However, caddy is unable to retrieve a certificate.

Specifically, I want my DNS server (AdGuard Home) to be able to resolve this using a DNS rewrite, but I don’t want a record in my upstream resolver, Cloudflare. I want to have something like foo.local.zenvoserver.com, which would only be accessible in my own network. I know I can get the same functionality if add the record to Cloudflare and block external IPs from connecting, but I wanted the extra layer of security.

I don’t think this is the same issue as multi-SAN support.

2. Error messages and/or full log output:

2025-04-14T23:41:38.256080153Z {"level":"error","ts":1744674098.2560134,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"*.local.zenvoserver.com","issuer":"acme-v02.api.letsencrypt.org-directory","error":"[*.local.zenvoserver.com] solving challenges: waiting for solver certmagic.solverWrapper to be ready: timed out waiting for record to fully propagate; verify DNS provider configuration is correct - last error: <nil> (order=https://acme-v02.api.letsencrypt.org/acme/order/2324093527/374233011677) (ca=https://acme-v02.api.letsencrypt.org/directory)"}
2025-04-14T23:41:38.256152489Z {"level":"debug","ts":1744674098.2560446,"logger":"events","msg":"event","name":"cert_failed","id":"2a0a0ba0-9a16-43c9-abe0-c893f9cdf9e1","origin":"tls","data":{"error":{},"identifier":"*.local.zenvoserver.com","issuers":["acme-v02.api.letsencrypt.org-directory"],"renewal":false}}
2025-04-14T23:41:38.256166696Z {"level":"error","ts":1744674098.2560906,"logger":"tls.obtain","msg":"will retry","error":"[*.local.zenvoserver.com] Obtain: [*.local.zenvoserver.com] solving challenges: waiting for solver certmagic.solverWrapper to be ready: timed out waiting for record to fully propagate; verify DNS provider configuration is correct - last error: <nil> (order=https://acme-v02.api.letsencrypt.org/acme/order/2324093527/374233011677) (ca=https://acme-v02.api.letsencrypt.org/directory)","attempt":1,"retrying_in":60,"elapsed":121.81695712,"max_duration":2592000}

3. Caddy version:

v2.9.1

4. How I installed and ran Caddy:

Docker Compose

a. System environment:

Unraid 6.12.14 (no Unraid templates used)

b. Command:

I use dockge which just does docker compose up

c. Service/unit/compose file:

services:
  caddy:
    container_name: caddy
    build:
      dockerfile_inline: |
        FROM caddy:2-builder AS builder

        RUN xcaddy build \
            --with github.com/mholt/caddy-l4 \
            --with github.com/caddyserver/transform-encoder \
            --with github.com/hslatman/caddy-crowdsec-bouncer/http@main \
            --with github.com/hslatman/caddy-crowdsec-bouncer/layer4@main \
            --with github.com/ueffel/caddy-brotli \
            --with github.com/caddy-dns/cloudflare \
            --with github.com/mholt/caddy-ratelimit \
            --with github.com/jasonlovesdoggo/caddy-defender

        FROM caddy:2

        COPY --from=builder /usr/bin/caddy /usr/bin/caddy
    ports:
      - 80:80
      - 443:443
      - 443:443/udp
    environment:
      - CROWDSEC_API_KEY=${CROWDSEC_API_KEY}
      - CLOUDFLARE_API_TOKEN=${CLOUDFLARE_API_TOKEN}
    restart: unless-stopped
    volumes:
      - /various-mounts-here
    networks:
      - caddy
      - crowdsec
    security_opt:
      - no-new-privileges=true
  crowdsec:
    image: docker.io/crowdsecurity/crowdsec:latest
    container_name: crowdsec
    environment:
      - GID=100
      - COLLECTIONS=crowdsecurity/caddy crowdsecurity/http-cve
        crowdsecurity/whitelist-good-actors
      - BOUNCER_KEY_CADDY=${CROWDSEC_API_KEY}
    volumes:
      - /various-mounts-here
    networks:
      - crowdsec
    restart: unless-stopped
    security_opt:
      - no-new-privileges=true
networks:
  caddy:
    external: true
  crowdsec:
    external: true

d. My complete Caddy config:

{
	debug

	crowdsec {
		api_url http://crowdsec:8080
		api_key {env.CROWDSEC_API_KEY}
	}
}

(block_world) {
	@block not remote_ip private_ranges
	abort @block
}

(cloudflare_whitelist) {
	"ranges":
	[
	"173.245.48.0/20,
			103.21.244.0/22,
			103.22.200.0/22,
			103.31.4.0/22,
			141.101.64.0/18,
			108.162.192.0/18,
			190.93.240.0/20,
			188.114.96.0/20,
			197.234.240.0/22,
			198.41.128.0/17,
			162.158.0.0/15,
			104.16.0.0/13,
			104.24.0.0/14,
			172.64.0.0/13,
			131.0.72.0/22"
	]
}

(cloudflare_only) {
	@block not remote_ip private_ranges cloudflare_whitelist
}

(security_headers) {
	header {
		Strict-Transport-Security "max-age=31536000; includeSubDomains;"
		X-Frame-Options "SAMEORIGIN"
		X-Content-Type-Options "nosniff"
		Referrer-Policy "strict-origin"
		X-Robots-Tag "noindex, nofollow, nosnippet, noarchive"
	}
}

(log_settings) {
	log {
		output file /config/logs/access.log
		level WARN
	}
}

(dns_challenge) {
	tls {
		dns cloudflare {env.CLOUDFLARE_API_TOKEN}
		resolvers 1.1.1.1 1.0.0.1
	}
}

(crowdsec_bouncer) {
	route {
		crowdsec
	}
	log {
		##Logs needed for crowdsec
		output file /var/log/caddy/access.log
	}
}

*.local.zenvoserver.com {
	import log_settings
	import security_headers
	import dns_challenge
	import block_world

	@foo host foo.local.zenvoserver.com
	handle @foo {
		reverse_proxy http://192.168.1.254:10022
	}

	handle {
		abort
	}
}

*.zenvoserver.com {
	import log_settings
	import security_headers
	import dns_challenge
	import cloudflare_only
	import crowdsec_bouncer

	@bar host bar.zenvoserver.com
	handle @bar {
		reverse_proxy http://192.168.1.254:20010
	}

	handle {
		abort
	}
}

5. Links to relevant resources:

sorry this post was missed. Can you retry with v2.10 and report the result?

Sure, though I see this docker image PR hasn’t been merged yet. I can wait for that and report back.

1 Like

Update, docker hub now has 2.10.0. But for some reason I needed to pin my caddy version to 2.10.0 in order for it to pull, instead of just 2. (Maybe I did something wrong? Tried force-rebuild and --no-cache.)

dockerfile_inline: |
        FROM caddy:2.10.0-builder AS builder

        RUN xcaddy build \
            --with github.com/mholt/caddy-l4 \
            --with github.com/caddyserver/transform-encoder \
            --with github.com/hslatman/caddy-crowdsec-bouncer/http@main \
            --with github.com/hslatman/caddy-crowdsec-bouncer/layer4@main \
            --with github.com/ueffel/caddy-brotli \
            --with github.com/caddy-dns/cloudflare \
            --with github.com/mholt/caddy-ratelimit \
            --with github.com/jasonlovesdoggo/caddy-defender

        FROM caddy:2.10.0

        COPY --from=builder /usr/bin/caddy /usr/bin/caddy
    ports:

Anyway, I’m still seeing the same error:

2025-04-22T01:36:49.007539893Z {"level":"error","ts":1745285809.0074468,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"*.local.zenvoserver.com","issuer":"acme-v02.api.letsencrypt.org-directory","error":"[*.local.zenvoserver.com] solving challenges: waiting for solver certmagic.solverWrapper to be ready: timed out waiting for record to fully propagate; verify DNS provider configuration is correct - last error: <nil> (order=https://acme-v02.api.letsencrypt.org/acme/order/2324093527/376614802167) (ca=https://acme-v02.api.letsencrypt.org/directory)"}
2025-04-22T01:36:49.007602461Z {"level":"debug","ts":1745285809.0074766,"logger":"events","msg":"event","name":"cert_failed","id":"e501f7cd-dcb5-4258-994d-5c43d321b357","origin":"tls","data":{"error":{},"identifier":"*.local.zenvoserver.com","issuers":["acme-v02.api.letsencrypt.org-directory"],"renewal":false}}
2025-04-22T01:36:49.007666823Z {"level":"error","ts":1745285809.007577,"logger":"tls.obtain","msg":"will retry","error":"[*.local.zenvoserver.com] Obtain: [*.local.zenvoserver.com] solving challenges: waiting for solver certmagic.solverWrapper to be ready: timed out waiting for record to fully propagate; verify DNS provider configuration is correct - last error: <nil> (order=https://acme-v02.api.letsencrypt.org/acme/order/2324093527/376614802167) (ca=https://acme-v02.api.letsencrypt.org/directory)","attempt":1,"retrying_in":60,"elapsed":121.735237096,"max_duration":2592000}

LetsEncrypt needs to be able to look up the DNS challenge results - you have to have those records (eg _acme-challenge.foo.local.zenoserver.com) visible publicly, Just in your local DNS will not suffice.

I found the issue - user error. I thought I disabled DNS hijacking on my router… turned out I didn’t. I saw it was making the acme records on cloudflare but was timing out with that error, because I guess caddy was looking for the record in my local DNS.

I happened to stumble upon this github issue that was very similar.

Thanks for the ideas! It put me on the right track. :slightly_smiling_face:

2 Likes