Hello! I started working on a new Caddy setup today and ran into a strange problem regarding the automatic HTTPS feature.
1. The problem I’m having:
I’m trying to set up Caddy as a reverse proxy to a service running in a local container. For HTTPS, I’d like Caddy to use a root CA certificate that is already deployed between my test machines, instead of generating a self-signed one.
I configured my CA public and private key in the global config, under the PKI section. It seems like Caddy successfully starts and generates an intermediate from the root CA I provided, as well as a certificate for the website itself. curl’ing to the website also seems to work fine.
However, both Chrome and Firefox refuse to proceed to the site, and show a warning page complaining about the site certificate being invalid.
The errors appear when trying to access the site both from a Linux and a Windows machine. I have verified that the root CA is trusted in both machines.
2. Error messages and/or full log output:
- Chrome: net::ERR_CERT_INVALID, without much information otherwise.
- Firefox: SEC_ERROR_CERT_SIGNATURE_ALGORITHM_DISABLED
“The certificate is not trusted because it was signed using a signature algorithm that was disabled because that algorithm is not secure.”
Here is the certificate that Caddy generated for the site:
Certificate:
Data:
Version: 3 (0x2)
Serial Number:
3c:2f:88:c3:66:5c:bb:e8:6d:56:2b:f5:c5:15:f5:70
Signature Algorithm: ecdsa-with-SHA256
Issuer: CN=Testnet Root CA - ECC Intermediate
Validity
Not Before: Mar 22 20:01:49 2025 GMT
Not After : Mar 23 08:01:49 2025 GMT
Subject:
Subject Public Key Info:
Public Key Algorithm: id-ecPublicKey
Public-Key: (256 bit)
pub:
04:cc:86:7b:66:ae:97:fb:82:e7:6f:9a:e8:95:a0:
af:63:b8:6c:3d:30:be:83:83:da:2a:08:40:86:0e:
a0:e5:79:38:1d:6b:8f:5f:9b:c4:87:ff:db:79:b7:
f5:02:be:d4:63:d6:4d:4c:ce:85:48:f0:64:c4:86:
d0:0d:48:74:5e
ASN1 OID: prime256v1
NIST CURVE: P-256
X509v3 extensions:
X509v3 Key Usage: critical
Digital Signature
X509v3 Extended Key Usage:
TLS Web Server Authentication, TLS Web Client Authentication
X509v3 Subject Key Identifier:
59:01:AB:06:0F:AB:BA:99:51:F4:3E:F1:A4:59:F2:AD:8E:64:10:52
X509v3 Authority Key Identifier:
F2:44:97:2C:FE:AD:24:50:09:A2:B6:2F:B9:30:0B:B1:21:F4:38:A5
X509v3 Subject Alternative Name: critical
DNS:chat.opal
Signature Algorithm: ecdsa-with-SHA256
Signature Value:
30:46:02:21:00:c8:9d:39:c3:a9:52:d0:22:ac:20:d4:82:87:
d2:76:a5:dc:9b:a5:e5:97:c2:52:c4:49:74:b9:ab:76:af:75:
33:02:21:00:c1:62:d4:43:bf:30:9a:a7:35:74:f0:48:b2:80:
07:a7:72:b2:12:d7:ec:f0:fb:76:0c:71:f2:fd:24:35:36:95
Caddy log:
Mar 22 22:01:49 opalium systemd[1]: Starting Caddy web server...
Mar 22 22:01:49 opalium caddy[98315]: {"level":"info","ts":1742673709.240114,"msg":"using config from file","file":"/etc/caddy/Caddyfile"}
Mar 22 22:01:49 opalium caddy[98315]: {"level":"info","ts":1742673709.2406673,"msg":"adapted config to JSON","adapter":"caddyfile"}
Mar 22 22:01:49 opalium caddy[98315]: {"level":"info","ts":1742673709.240857,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc0006c2780"}
Mar 22 22:01:49 opalium caddy[98315]: {"level":"info","ts":1742673709.2555566,"logger":"http.auto_https","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443}
Mar 22 22:01:49 opalium caddy[98315]: {"level":"info","ts":1742673709.2555687,"logger":"http.auto_https","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
Mar 22 22:01:49 opalium caddy[98315]: {"level":"debug","ts":1742673709.2555816,"logger":"http.auto_https","msg":"adjusted config","tls":{"automation":{"policies":[{"subjects":["chat.opal"]},{}]}},"http":{"servers":{"remaining_auto_https_redirects":{"listen":[":80"],"routes":[{},{}],"logs":{"logger_names":{"chat.opal":["log0"]}}},"srv0":{"listen":[":443"],"routes":[{"handle":[{"handler":"subroute","routes":[{"handle":[{"handler":"reverse_proxy","upstreams":[{"dial":":3000"}]}]}]}],"terminal":true}],"tls_connection_policies":[{}],"automatic_https":{},"logs":{"logger_names":{"chat.opal":["log0"]}}}}}}
Mar 22 22:01:49 opalium caddy[98315]: {"level":"info","ts":1742673709.2556894,"logger":"tls.cache.maintenance","msg":"stopped background certificate maintenance","cache":"0xc0006c2780"}
Mar 22 22:01:49 opalium caddy[98315]: Valid configuration
Mar 22 22:01:49 opalium caddy[98334]: caddy.HomeDir=/var/lib/caddy
Mar 22 22:01:49 opalium caddy[98334]: caddy.AppDataDir=/var/lib/caddy
Mar 22 22:01:49 opalium caddy[98334]: caddy.AppConfigDir=/etc/caddy
Mar 22 22:01:49 opalium caddy[98334]: caddy.ConfigAutosavePath=/var/lib/caddy/autosave.json
Mar 22 22:01:49 opalium caddy[98334]: caddy.Version=v2.9.1
Mar 22 22:01:49 opalium caddy[98334]: runtime.GOOS=linux
Mar 22 22:01:49 opalium caddy[98334]: runtime.GOARCH=amd64
Mar 22 22:01:49 opalium caddy[98334]: runtime.Compiler=gc
Mar 22 22:01:49 opalium caddy[98334]: runtime.NumCPU=16
Mar 22 22:01:49 opalium caddy[98334]: runtime.GOMAXPROCS=16
Mar 22 22:01:49 opalium caddy[98334]: runtime.Version=go1.24.1
Mar 22 22:01:49 opalium caddy[98334]: os.Getwd=/
Mar 22 22:01:49 opalium caddy[98334]: PATH=/usr/local/sbin:/usr/local/bin:/usr/bin
Mar 22 22:01:49 opalium caddy[98334]: XDG_DATA_DIRS=/var/lib/flatpak/exports/share:/usr/local/share/:/usr/share/
Mar 22 22:01:49 opalium caddy[98334]: NOTIFY_SOCKET=/run/systemd/notify
Mar 22 22:01:49 opalium caddy[98334]: USER=caddy
Mar 22 22:01:49 opalium caddy[98334]: LOGNAME=caddy
Mar 22 22:01:49 opalium caddy[98334]: HOME=/var/lib/caddy
Mar 22 22:01:49 opalium caddy[98334]: INVOCATION_ID=ac840e5959d0452aabac267754099487
Mar 22 22:01:49 opalium caddy[98334]: JOURNAL_STREAM=9:506356
Mar 22 22:01:49 opalium caddy[98334]: SYSTEMD_EXEC_PID=98334
Mar 22 22:01:49 opalium caddy[98334]: XDG_DATA_HOME=/var/lib
Mar 22 22:01:49 opalium caddy[98334]: XDG_CONFIG_HOME=/etc
Mar 22 22:01:49 opalium caddy[98334]: {"level":"info","ts":1742673709.31644,"msg":"using config from file","file":"/etc/caddy/Caddyfile"}
Mar 22 22:01:49 opalium caddy[98334]: {"level":"info","ts":1742673709.3170245,"msg":"adapted config to JSON","adapter":"caddyfile"}
Mar 22 22:01:49 opalium caddy[98334]: {"level":"info","ts":1742673709.3173041,"logger":"admin","msg":"admin endpoint started","address":"unix//run/caddy/admin.socket","enforce_origin":false,"origins":["","//127.0.0.1","//::1"]}
Mar 22 22:01:49 opalium caddy[98334]: {"level":"info","ts":1742673709.3183305,"logger":"http.auto_https","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443}
Mar 22 22:01:49 opalium caddy[98334]: {"level":"info","ts":1742673709.318339,"logger":"http.auto_https","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
Mar 22 22:01:49 opalium caddy[98334]: {"level":"debug","ts":1742673709.318351,"logger":"http.auto_https","msg":"adjusted config","tls":{"automation":{"policies":[{"subjects":["chat.opal"]},{}]}},"http":{"servers":{"remaining_auto_https_redirects":{"listen":[":80"],"routes":[{},{}],"logs":{"logger_names":{"chat.opal":["log0"]}}},"srv0":{"listen":[":443"],"routes":[{"handle":[{"handler":"subroute","routes":[{"handle":[{"handler":"reverse_proxy","upstreams":[{"dial":":3000"}]}]}]}],"terminal":true}],"tls_connection_policies":[{}],"automatic_https":{},"logs":{"logger_names":{"chat.opal":["log0"]}}}}}}
Mar 22 22:01:49 opalium caddy[98334]: {"level":"info","ts":1742673709.3183756,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc00076a100"}
Mar 22 22:01:49 opalium caddy[98334]: {"level":"info","ts":1742673709.3184392,"logger":"pki.ca.testnet","msg":"root certificate is already trusted by system","path":"/etc/pki/ca.crt"}
Mar 22 22:01:49 opalium caddy[98334]: {"level":"debug","ts":1742673709.3185081,"logger":"http","msg":"starting server loop","address":"[::]:80","tls":false,"http3":false}
Mar 22 22:01:49 opalium caddy[98334]: {"level":"warn","ts":1742673709.3185213,"logger":"http","msg":"HTTP/2 skipped because it requires TLS","network":"tcp","addr":":80"}
Mar 22 22:01:49 opalium caddy[98334]: {"level":"warn","ts":1742673709.3185236,"logger":"http","msg":"HTTP/3 skipped because it requires TLS","network":"tcp","addr":":80"}
Mar 22 22:01:49 opalium caddy[98334]: {"level":"info","ts":1742673709.3185256,"logger":"http.log","msg":"server running","name":"remaining_auto_https_redirects","protocols":["h1","h2","h3"]}
Mar 22 22:01:49 opalium caddy[98334]: {"level":"debug","ts":1742673709.3185408,"logger":"http","msg":"starting server loop","address":"[::]:443","tls":true,"http3":false}
Mar 22 22:01:49 opalium caddy[98334]: {"level":"info","ts":1742673709.3185437,"logger":"http","msg":"enabling HTTP/3 listener","addr":":443"}
Mar 22 22:01:49 opalium caddy[98334]: {"level":"info","ts":1742673709.3185713,"msg":"failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 7168 kiB, got: 416 kiB). See https://github.com/quic-go/quic-go/wiki/UDP-Buffer-Sizes for details."}
Mar 22 22:01:49 opalium caddy[98334]: {"level":"info","ts":1742673709.3188076,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2","h3"]}
Mar 22 22:01:49 opalium caddy[98334]: {"level":"info","ts":1742673709.318822,"logger":"http","msg":"enabling automatic TLS certificate management","domains":["chat.opal"]}
Mar 22 22:01:49 opalium caddy[98334]: {"level":"info","ts":1742673709.3188965,"msg":"autosaved config (load with --resume flag)","file":"/var/lib/caddy/autosave.json"}
Mar 22 22:01:49 opalium caddy[98334]: {"level":"info","ts":1742673709.318922,"msg":"serving initial configuration"}
Mar 22 22:01:49 opalium systemd[1]: Started Caddy web server.
Mar 22 22:01:49 opalium caddy[98334]: {"level":"info","ts":1742673709.321358,"logger":"tls","msg":"cleaning storage unit","storage":"FileStorage:/var/lib/caddy"}
Mar 22 22:01:49 opalium caddy[98334]: {"level":"info","ts":1742673709.3241239,"logger":"tls.obtain","msg":"acquiring lock","identifier":"chat.opal"}
Mar 22 22:01:49 opalium caddy[98334]: {"level":"info","ts":1742673709.3268013,"logger":"tls","msg":"finished cleaning storage units"}
Mar 22 22:01:49 opalium caddy[98334]: {"level":"info","ts":1742673709.3294415,"logger":"tls.obtain","msg":"lock acquired","identifier":"chat.opal"}
Mar 22 22:01:49 opalium caddy[98334]: {"level":"info","ts":1742673709.3294735,"logger":"tls.obtain","msg":"obtaining certificate","identifier":"chat.opal"}
Mar 22 22:01:49 opalium caddy[98334]: {"level":"debug","ts":1742673709.3294861,"logger":"events","msg":"event","name":"cert_obtaining","id":"e3f9a4da-6af9-4737-916a-c18d33a1e106","origin":"tls","data":{"identifier":"chat.opal"}}
Mar 22 22:01:49 opalium caddy[98334]: {"level":"debug","ts":1742673709.3295605,"logger":"tls","msg":"created CSR","identifiers":["chat.opal"],"san_dns_names":["chat.opal"],"san_emails":[],"common_name":"","extra_extensions":0}
Mar 22 22:01:49 opalium caddy[98334]: {"level":"debug","ts":1742673709.3297248,"logger":"tls.obtain","msg":"trying issuer 1/1","issuer":"testnet"}
Mar 22 22:01:49 opalium caddy[98334]: {"level":"debug","ts":1742673709.3299193,"logger":"pki.ca.testnet","msg":"using intermediate signer","serial":"11662252775630462296731532533870731060","not_before":"2025-03-22 20:01:49 +0000 UTC","not_after":"2025-03-29 20:01:49 +0000 UTC"}
Mar 22 22:01:49 opalium caddy[98334]: {"level":"info","ts":1742673709.3386323,"logger":"tls.obtain","msg":"certificate obtained successfully","identifier":"chat.opal","issuer":"testnet"}
Mar 22 22:01:49 opalium caddy[98334]: {"level":"debug","ts":1742673709.3386652,"logger":"events","msg":"event","name":"cert_obtained","id":"c787152e-282a-42a3-942c-f091e8340570","origin":"tls","data":{"certificate_path":"certificates/testnet/chat.opal/chat.opal.crt","csr_pem":"LS0tLS1CRUdJTiBDRVJUSUZJQ0FURSBSRVFVRVNULS0tLS0KTUlIak1JR0pBZ0VBTUFBd1dUQVRCZ2NxaGtqT1BRSUJCZ2dxaGtqT1BRTUJCd05DQUFUTWhudG1ycGY3Z3VkdgptdWlWb0s5anVHdzlNTDZEZzlvcUNFQ0dEcURsZVRnZGE0OWZtOFNILzl0NXQvVUN2dFJqMWsxTXpvVkk4R1RFCmh0QU5TSFJlb0Njd0pRWUpLb1pJaHZjTkFRa09NUmd3RmpBVUJnTlZIUkVFRFRBTGdnbGphR0YwTG05d1lXd3cKQ2dZSUtvWkl6ajBFQXdJRFNRQXdSZ0loQU9QZFJtNmgvek5zZDZLY1hqdld3SG5CYTQ3ZStOREN0M01qZnJEZApLRmRnQWlFQXdwc05rQVpXZ3FrNlNONDJBUXprVXBjZURnUXpyU1poNzh3M2JxdVJIbjQ9Ci0tLS0tRU5EIENFUlRJRklDQVRFIFJFUVVFU1QtLS0tLQo=","identifier":"chat.opal","issuer":"testnet","metadata_path":"certificates/testnet/chat.opal/chat.opal.json","private_key_path":"certificates/testnet/chat.opal/chat.opal.key","renewal":false,"storage_path":"certificates/testnet/chat.opal"}}
Mar 22 22:01:49 opalium caddy[98334]: {"level":"info","ts":1742673709.3386781,"logger":"tls.obtain","msg":"releasing lock","identifier":"chat.opal"}
Mar 22 22:01:49 opalium caddy[98334]: {"level":"debug","ts":1742673709.3388553,"logger":"tls.cache","msg":"added certificate to cache","subjects":["chat.opal"],"expiration":1742716910,"managed":true,"issuer_key":"testnet","hash":"b2cadc04950191c96a4fc5b974e649e18cbe5cb0361f08f040f349b5ed7847ff","cache_size":1,"cache_capacity":10000}
Mar 22 22:01:49 opalium caddy[98334]: {"level":"debug","ts":1742673709.3388665,"logger":"events","msg":"event","name":"cached_managed_cert","id":"651f0f07-0325-43bb-933c-f28ed0e120a3","origin":"tls","data":{"sans":["chat.opal"]}}
Mar 22 22:04:55 opalium caddy[98334]: {"level":"debug","ts":1742673895.4976525,"logger":"events","msg":"event","name":"tls_get_certificate","id":"eedb5b67-577c-4d40-94f6-d659f5af6eff","origin":"tls","data":{"client_hello":{"CipherSuites":[19018,4865,4866,4867,49195,49199,49196,49200,52393,52392,49171,49172,156,157,47,53],"ServerName":"chat.opal","SupportedCurves":[23130,4588,29,23,24],"SupportedPoints":"AA==","SignatureSchemes":[1027,2052,1025,1283,2053,1281,2054,1537],"SupportedProtos":["h2","http/1.1"],"SupportedVersions":[64250,772,771],"RemoteAddr":{"IP":"10.10.10.10","Port":58944,"Zone":""},"LocalAddr":{"IP":"10.10.10.10","Port":443,"Zone":""}}}}
Mar 22 22:04:55 opalium caddy[98334]: {"level":"debug","ts":1742673895.4977386,"logger":"tls.handshake","msg":"choosing certificate","identifier":"chat.opal","num_choices":1}
Mar 22 22:04:55 opalium caddy[98334]: {"level":"debug","ts":1742673895.497744,"logger":"tls.handshake","msg":"default certificate selection results","identifier":"chat.opal","subjects":["chat.opal"],"managed":true,"issuer_key":"testnet","hash":"b2cadc04950191c96a4fc5b974e649e18cbe5cb0361f08f040f349b5ed7847ff"}
Mar 22 22:04:55 opalium caddy[98334]: {"level":"debug","ts":1742673895.4977503,"logger":"tls.handshake","msg":"matched certificate in cache","remote_ip":"10.10.10.10","remote_port":"58944","subjects":["chat.opal"],"managed":true,"expiration":1742716910,"hash":"b2cadc04950191c96a4fc5b974e649e18cbe5cb0361f08f040f349b5ed7847ff"}
Mar 22 22:04:55 opalium caddy[98334]: {"level":"debug","ts":1742673895.4990656,"logger":"http.stdlib","msg":"http: TLS handshake error from 10.10.10.10:58944: remote error: tls: unknown certificate"}
3. Caddy version:
v2.9.1
4. How I installed and ran Caddy:
a. System environment:
EndeavourOS (Arch Linux) 6.13.7 (Zen Kernel)
systemd 257 (257.4-1-arch)
I’m running Caddy as a systemd service, per the default configuration of the package.
b. Command:
sudo systemctl (re)start caddy
c. Service/unit/compose file:
Just the default one that came with the package:
# caddy.service
[Unit]
Description=Caddy web server
Documentation=https://caddyserver.com/docs/
After=network-online.target
Wants=network-online.target
StartLimitIntervalSec=14400
StartLimitBurst=10
[Service]
Type=notify
User=caddy
Group=caddy
Environment=XDG_DATA_HOME=/var/lib
Environment=XDG_CONFIG_HOME=/etc
ExecStartPre=/usr/bin/caddy validate --config /etc/caddy/Caddyfile
ExecStart=/usr/bin/caddy run --environ --config /etc/caddy/Caddyfile
ExecReload=/usr/bin/caddy reload --config /etc/caddy/Caddyfile --force
ExecStopPost=/usr/bin/rm -f /run/caddy/admin.socket
# Do not allow the process to be restarted in a tight loop. If the
# process fails to start, something critical needs to be fixed.
Restart=on-abnormal
# Use graceful shutdown with a reasonable timeout
TimeoutStopSec=5s
LimitNOFILE=1048576
LimitNPROC=512
# Hardening options
AmbientCapabilities=CAP_NET_BIND_SERVICE
CapabilityBoundingSet=CAP_NET_BIND_SERVICE
DevicePolicy=closed
LockPersonality=true
MemoryAccounting=true
MemoryDenyWriteExecute=true
NoNewPrivileges=true
PrivateDevices=true
PrivateTmp=true
ProcSubset=pid
ProtectClock=true
ProtectControlGroups=true
ProtectHome=true
ProtectHostname=true
ProtectKernelLogs=true
ProtectKernelModules=true
ProtectKernelTunables=true
ProtectProc=invisible
ProtectSystem=strict
RemoveIPC=true
ReadWritePaths=/var/lib/caddy /var/log/caddy /run/caddy
RestrictNamespaces=true
RestrictRealtime=true
RestrictSUIDSGID=true
[Install]
WantedBy=multi-user.target
d. My complete Caddy config:
# Global configuration
{
# Restrict the admin interface to a local unix file socket whose directory
# is restricted to caddy:caddy. By default the TCP socket allows arbitrary
# modification for any process and user that has access to the local
# interface. If admin over TCP is turned on one should make sure
# implications are well understood.
admin "unix//run/caddy/admin.socket"
pki {
ca testnet {
name "Testnet Root CA"
root {
format pem_file
cert /etc/pki/ca.crt
key /etc/pki/private/ca.key
}
}
}
debug
}
chat.opal {
tls {
issuer internal {
ca testnet
}
}
log {
output file /var/log/caddy/chat.opal/access.log {
mode 640
}
}
reverse_proxy :3000
}
5. Links to relevant resources:
I’ve followed this and this pages in the docs to set up the config.
I’m honestly kinda stumped by this one, so your help is very much appreciated.
Cheers!
-Opalium