Failure to Renew Certification - Wildcard Subdomain - Porkbun DNS

Help
1

1. The problem I’m having:

I have an already running server at *.team.bytezone.dev. The automatic certificate renew process is failing due to “unknown domain”?

2. Error messages and/or full log output:

caddy-1  | {
  "level": "error",
  "ts": 1733165308.495665,
  "logger": "tls.renew",
  "msg": "could not get certificate from issuer",
  "identifier": "*.team.bytezone.dev",
  "issuer": "acme-v02.api.letsencrypt.org-directory",
  "error": "[*.team.bytezone.dev] solving challenges: presenting for challenge: adding temporary record for zone \"team.bytezone.dev.\": Invalid http response status, {\"status\":\"ERROR\",\"message\":\"Invalid domain.\"} (order=https://acme-v02.api.letsencrypt.org/acme/order/1920392906/329140245707) (ca=https://acme-v02.api.letsencrypt.org/directory)"
}
caddy-1  | {
  "level": "error",
  "ts": 1733165308.4957364,
  "logger": "tls.renew",
  "msg": "will retry",
  "error": "[*.team.bytezone.dev] Renew: [*.team.bytezone.dev] solving challenges: presenting for challenge: adding temporary record for zone \"team.bytezone.dev.\": Invalid http response status, {\"status\":\"ERROR\",\"message\":\"Invalid domain.\"} (order=https://acme-v02.api.letsencrypt.org/acme/order/1920392906/329140245707) (ca=https://acme-v02.api.letsencrypt.org/directory)",
  "attempt": 1,
  "retrying_in": 60,
  "elapsed": 1.21061175,
  "max_duration": 2592000
}

3. Caddy version:

v2.8.4 h1:q3pe0wpBj1OcHFZ3n/1nl4V4bxBrYoSoab7rL9BMYNk=

Build Info:

$ caddy build-info | grep dns
dep     github.com/caddy-dns/porkbun    v0.2.1  h1:/+5TYIbEo/2MI2Vq16HIpBEMA2dGclPpq8oEXjoK5Nw=
dep     github.com/libdns/libdns        v0.2.2  h1:O6ws7bAfRPaBsgAYt8MDe2HcNBGC29hkZ9MX2eUSX3s=
dep     github.com/libdns/porkbun       v0.2.0  h1:oa2F0doE93RiJkauVVbM+P2AZ7jovDSiH6u3aaQezvQ=
dep     github.com/mholt/caddy-dynamicdns       v0.0.0-20241025234131-7c818ab3fc34      h1:y6kb4RLCopL/onHljtBi+5wHJinh+RMp4HBNFLmlvaE=
dep     github.com/miekg/dns    v1.1.59 h1:C9EXc/UToRwKLhK5wKU/I4QVsBUc8kE6MkHBkeypWZs=

4. How I installed and ran Caddy:

Via docker compose.

a. System environment:

OS: Debian Bookworm
Kernel: 6.1.0-26-could-amd64
Docker Version:

Client: Docker Engine - Community
 Version:           27.3.1
 API version:       1.47
 Go version:        go1.22.7
 Git commit:        ce12230
 Built:             Fri Sep 20 11:41:11 2024
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          27.3.1
  API version:      1.47 (minimum version 1.24)
  Go version:       go1.22.7
  Git commit:       41ca978
  Built:            Fri Sep 20 11:41:11 2024
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.7.23
  GitCommit:        57f17b0a6295a39009d861b89e3b3b87b005ca27
 runc:
  Version:          1.1.14
  GitCommit:        v1.1.14-0-g2c9f560
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

b. Command:

docker compose up -d

c. Service/unit/compose file:

Compose.yml file

services:
  caddy:
    image: registry.team.bytezone.dev/bytezone/caddy:${CADDY_VERSION}-alpine-porkbun
    restart: always
    pull_policy: build
    build:
      args:
        - VERSION=${CADDY_VERSION}
      context: .
      dockerfile: Dockerfile
    environment:
      - PORKBUN_API_KEY=${PORKBUN_API_KEY}
      - PORKBUN_API_SECRET_KEY=${PORKBUN_API_SECRET_KEY}
    network_mode: service:wg
    volumes:
      - ./caddy.d/Caddyfile:/etc/caddy/Caddyfile
      - ./certs:/certs:ro
      - caddy-config:/config
      - caddy-data:/data

volumes:
  caddy-config:
    driver: local
  caddy-data:
    driver: local

Dockerfile

ARG VERSION=2
FROM caddy:${VERSION}-builder-alpine AS builder

RUN xcaddy build \
    --with github.com/caddy-dns/porkbun \
    --with github.com/mholt/caddy-dynamicdns

FROM caddy:${VERSION}-alpine AS runtime

COPY --from=builder /usr/bin/caddy /usr/bin/caddy

d. My complete Caddy config:

{
	dynamic_dns {
		provider porkbun {
			api_key {$PORKBUN_API_KEY}
			api_secret_key {$PORKBUN_API_SECRET_KEY}
		}

		domains {
			bytezone.dev team
		}

		check_interval 20m
		versions ipv4
		ttl 6h
	}
}

*.team.bytezone.dev {
	tls {
		dns porkbun {
			api_key {$PORKBUN_API_KEY}
			api_secret_key {$PORKBUN_API_SECRET_KEY}
		}
	}

	# RocketChat UI
	@chat host chat.team.bytezone.dev
	handle @chat {
		reverse_proxy /* {
			header_up Host {http.request.host}
			header_up X-Real-IP {http.request.remote}
			header_up X-Forwarded-Port {http.request.port}
			header_up Connection {http.request.header.Connection}
			header_up Upgrade {http.request.header.Upgrade}

			to http://172.18.1.4:3000
		}
	}

	# Ollama UI
	@ollama host ollama.team.bytezone.dev
	handle @ollama {
		reverse_proxy /* {
			to http://10.20.24.5:8080
		}
	}

	# Ollama API
	@ollama-api host ollama-api.team.bytezone.dev
	handle @ollama-api {
		reverse_proxy /* {
			to http://10.20.24.5:11434
		}
	}

	# Docker registry UI
	@registry host registry.team.bytezone.dev
	handle @registry {
		reverse_proxy /* {
			to http://172.18.1.6:80
		}
	}

	# Soft-Serve http Endpoint
	@git host git.team.bytezone.dev
	handle @git {
		reverse_proxy /* {
			to http://172.18.1.3:23232
		}
	}

	## TODO https://www.jetbrains.com/help/youtrack/server/reverse-proxy-configuration.html
	# JetBrains YouTrack
	@youtrack host youtrack.team.bytezone.dev
	handle @youtrack {
		request_body {
			max_size 10MB
		}

		reverse_proxy /* {
			to http://172.18.1.8:8080
		}
	}

	handle {
		abort
	}
}

5. Links to relevant resources:

Also reported on the github.com/caddy-dns/porkbun repo (#20)

2

Hi @CodeMan99,

This likely is only supplemental information for you. :slight_smile:

Here is a list of issued certificates crt.sh | team.bytezone.dev,
it only has the Common Name and SANs *.team.bytezone.dev, but one should also have team.bytezone.dev in the SANs as well (probably in the Common Name instead of the wildcard, too).

For the domain name example.com the SANs needed are example.com and *.example.com and that would cover
one.example.com
two.example.com
three.example.com

But not
four.subdomain.example.com
five.subdomain.example.com
six.subdomain.example.com
to cover those you also need SANs of subdomain.example.com and *subdomain.example.com on a certificate.

3

I honestly don’t understand what you mean Bruce. Maybe I forgot my smart pills.

Regardless, my Caddyfile looks almost identical to the official example and everything worked flawless on certificate creation.

4

Hi @CodeMan99,

I too am still learning Caddy! :slight_smile:

I was sharing more what I know about how Let’s Encrypt works.
You probably should have both team.bytezone.dev and *.team.bytezone.dev in the SANs when requesting a wildcard certificate.

No I probably forgot my “don’t flubber up talking (or typing) speak” pills. :frowning:

Kindly wait for more knowledgeable Caddy community volunteers to assist.

5

This is most curious. It seems like Caddy’s trying to add a record for team.bytezone.dev but the zone should most definitely be bytezone.dev.

My understanding is that Caddy should be doing an SOA lookup on the site address in order to determine the base domain via DNS. I’ve seen that process fail outright before when flaky local DNS resolvers mishandled the request/errored out, but off the top of my head I don’t recall it producing a straight-up-incorrect response.

In your tls directive for *.team.bytezone.dev, try adding a resolvers subdirective pointing Caddy at known good public DNS (like Google or Cloudflare or maybe even directly at your Porkbun nameserver), just to rule out local DNS shenanigans.

https://caddyserver.com/docs/caddyfile/directives/tls#resolvers

2 Likes
7

Thank you! Adding resolvers to the configuration worked like a charm! Excellent :slight_smile:

1 Like