1. The problem I’m having:
This is my second time posting with this problem. Every ~3 months or so caddy appears to break my site due to an issue with the SSL certificate, after working fine for months.
This does not work across intranet, internet, Chrome, Edge, Firefox, Windows, Ubuntu etc.
2. Error messages and/or full log output:
ERR_CERT_DATE_INVALID
Was original error, but after following steps outlined below, I now receive:
ERR_CONNECTION_TIMED_OUT
Log:
Sep 26 13:23:07 calcifer caddy[1131]: {"level":"error","ts":1727320987.6222885,"logger":"tls.renew","msg":"could not get certificate from issuer","identifier":"mitchflix.net","issuer":"acme-v02.api.letsen
crypt.org-directory","error":"[mitchflix.net] solving challenges: presenting for challenge: adding temporary record for zone \"mitchflix.net.\": got error status: HTTP 403: [{Code:9109 Message:Invalid acc
ess token ErrorChain:[]}] (order=https://acme-staging-v02.api.letsencrypt.org/acme/order/112117554/19385210523) (ca=https://acme-staging-v02.api.letsencrypt.org/directory)"}
Sep 26 13:23:07 calcifer caddy[1131]: {"level":"error","ts":1727320987.6223314,"logger":"tls.renew","msg":"will retry","error":"[mitchflix.net] Renew: [mitchflix.net] solving challenges: presenting for ch
allenge: adding temporary record for zone \"mitchflix.net.\": got error status: HTTP 403: [{Code:9109 Message:Invalid access token ErrorChain:[]}] (order=https://acme-staging-v02.api.letsencrypt.org/acme/
order/112117554/19385210523) (ca=https://acme-staging-v02.api.letsencrypt.org/directory)","attempt":3,"retrying_in":120,"elapsed":185.195160652,"max_duration":2592000}
Sep 26 13:25:07 calcifer caddy[1131]: {"level":"info","ts":1727321107.6227562,"logger":"tls.renew","msg":"renewing certificate","identifier":"mitchflix.net","remaining":-70167.622750908}
Sep 26 13:25:07 calcifer caddy[1131]: {"level":"info","ts":1727321107.6231382,"logger":"tls.issuance.acme","msg":"using ACME account","account_id":"https://acme-staging-v02.api.letsencrypt.org/acme/acct/1
12117554","account_contact":[]}
Sep 26 13:25:08 calcifer caddy[1131]: {"level":"info","ts":1727321108.3648324,"logger":"tls.issuance.acme.acme_client","msg":"trying to solve challenge","identifier":"mitchflix.net","challenge_type":"dns-
01","ca":"https://acme-staging-v02.api.letsencrypt.org/directory"}
Sep 26 13:25:08 calcifer caddy[1131]: {"level":"error","ts":1727321108.6072648,"logger":"tls.issuance.acme.acme_client","msg":"cleaning up solver","identifier":"mitchflix.net","challenge_type":"dns-01","e
rror":"no memory of presenting a DNS record for \"_acme-challenge.mitchflix.net\" (usually OK if presenting also failed)"}
Sep 26 13:25:08 calcifer caddy[1131]: {"level":"error","ts":1727321108.8364334,"logger":"tls.renew","msg":"could not get certificate from issuer","identifier":"mitchflix.net","issuer":"acme-v02.api.letsen
crypt.org-directory","error":"[mitchflix.net] solving challenges: presenting for challenge: adding temporary record for zone \"mitchflix.net.\": got error status: HTTP 403: [{Code:9109 Message:Invalid acc
ess token ErrorChain:[]}] (order=https://acme-staging-v02.api.letsencrypt.org/acme/order/112117554/19385235233) (ca=https://acme-staging-v02.api.letsencrypt.org/directory)"}
Sep 26 13:25:08 calcifer caddy[1131]: {"level":"error","ts":1727321108.836478,"logger":"tls.renew","msg":"will retry","error":"[mitchflix.net] Renew: [mitchflix.net] solving challenges: presenting for cha
llenge: adding temporary record for zone \"mitchflix.net.\": got error status: HTTP 403: [{Code:9109 Message:Invalid access token ErrorChain:[]}] (order=https://acme-staging-v02.api.letsencrypt.org/acme/o
rder/112117554/19385235233) (ca=https://acme-staging-v02.api.letsencrypt.org/directory)","attempt":4,"retrying_in":300,"elapsed":306.409307337,"max_duration":2592000}
Sep 26 13:30:02 calcifer caddy[1131]: {"level":"info","ts":1727321402.124448,"logger":"tls","msg":"certificate needs renewal based on ARI window","subjects":["mitchflix.net"],"expiration":1727250940,"ari_
cert_id":"kydGmAOpUWiOmNbEQkjbI79YlNI.A7DOLo__OaJa8d7gqzZ_zhJI","next_ari_update":1727333931.3538516,"renew_check_interval":600,"window_start":1724573739,"window_end":1724746539,"selected_time":1724594516
,"renewal_cutoff":1724593916}
Sep 26 13:30:02 calcifer caddy[1131]: {"level":"info","ts":1727321402.1246493,"logger":"tls.cache.maintenance","msg":"certificate expires soon; queuing for renewal","identifiers":["mitchflix.net"],"remain
ing":-70462.124648851}
Sep 26 13:30:08 calcifer caddy[1131]: {"level":"info","ts":1727321408.8371227,"logger":"tls.renew","msg":"renewing certificate","identifier":"mitchflix.net","remaining":-70468.837117669}
Sep 26 13:30:08 calcifer caddy[1131]: {"level":"info","ts":1727321408.8375492,"logger":"tls.issuance.acme","msg":"using ACME account","account_id":"https://acme-staging-v02.api.letsencrypt.org/acme/acct/1
12117554","account_contact":[]}
Sep 26 13:30:10 calcifer caddy[1131]: {"level":"info","ts":1727321410.0157108,"logger":"tls.issuance.acme.acme_client","msg":"trying to solve challenge","identifier":"mitchflix.net","challenge_type":"dns-
01","ca":"https://acme-staging-v02.api.letsencrypt.org/directory"}
Sep 26 13:30:10 calcifer caddy[1131]: {"level":"error","ts":1727321410.2892694,"logger":"tls.issuance.acme.acme_client","msg":"cleaning up solver","identifier":"mitchflix.net","challenge_type":"dns-01","e
rror":"no memory of presenting a DNS record for \"_acme-challenge.mitchflix.net\" (usually OK if presenting also failed)"}
Sep 26 13:30:10 calcifer caddy[1131]: {"level":"error","ts":1727321410.5199587,"logger":"tls.renew","msg":"could not get certificate from issuer","identifier":"mitchflix.net","issuer":"acme-v02.api.letsen
crypt.org-directory","error":"[mitchflix.net] solving challenges: presenting for challenge: adding temporary record for zone \"mitchflix.net.\": got error status: HTTP 403: [{Code:9109 Message:Invalid acc
ess token ErrorChain:[]}] (order=https://acme-staging-v02.api.letsencrypt.org/acme/order/112117554/19385303783) (ca=https://acme-staging-v02.api.letsencrypt.org/directory)"}
Sep 26 13:30:10 calcifer caddy[1131]: {"level":"error","ts":1727321410.520009,"logger":"tls.renew","msg":"will retry","error":"[mitchflix.net] Renew: [mitchflix.net] solving challenges: presenting for cha
llenge: adding temporary record for zone \"mitchflix.net.\": got error status: HTTP 403: [{Code:9109 Message:Invalid access token ErrorChain:[]}] (order=https://acme-staging-v02.api.letsencrypt.org/acme/o
rder/112117554/19385303783) (ca=https://acme-staging-v02.api.letsencrypt.org/directory)","attempt":5,"retrying_in":600,"elapsed":608.092838237,"max_duration":2592000}
Sep 26 13:33:58 calcifer caddy[1131]: {"level":"info","ts":1727321638.0443811,"logger":"admin.api","msg":"received request","method":"GET","host":"localhost:2019","uri":"/pki/ca/local","remote_ip":"127.0.
0.1","remote_port":"44632","headers":{"Accept-Encoding":["gzip"],"Origin":["http://localhost:2019"],"User-Agent":["Go-http-client/1.1"]}}
Sep 26 13:35:37 calcifer caddy[1131]: {"level":"info","ts":1727321737.0287225,"logger":"admin.api","msg":"received request","method":"GET","host":"localhost:2019","uri":"/pki/ca/local","remote_ip":"127.0.
0.1","remote_port":"36424","headers":{"Accept-Encoding":["gzip"],"Origin":["http://localhost:2019"],"User-Agent":["Go-http-client/1.1"]}}
Sep 26 13:35:45 calcifer caddy[1131]: {"level":"info","ts":1727321745.4768064,"l
ogger":"admin.api","msg":"received request","method":"POST","host":"localhost:2019","uri":"/stop","remote_ip":"127.0.0.1","remote_port":"36430","headers":{"Accept-Encoding":["gzip"],"Content-Length":["0"],"Origin":["http://localhost:2019"],"User-Agent":["Go-http-client/1.1"]}}
Sep 26 13:35:45 calcifer caddy[1131]: {"level":"warn","ts":1727321745.476861,"logger":"admin.api","msg":"exiting; byeee!! 👋"}
Sep 26 13:35:45 calcifer caddy[1131]: {"level":"info","ts":1727321745.4768698,"logger":"http","msg":"servers shutting down with eternal grace period"}
Sep 26 13:35:50 calcifer caddy[1131]: {"level":"info","ts":1727321750.2719579,"logger":"tls.renew","msg":"releasing lock","identifier":"mitchflix.net"}
Sep 26 13:35:50 calcifer caddy[1131]: {"level":"error","ts":1727321750.2720494,"logger":"admin.api","msg":"unable to clean up lock in storage backend","storage":"FileStorage:/var/lib/caddy/.local/share/caddy","lock_key":"issue_cert_mitchflix.net","error":"remove /var/lib/caddy/.local/share/caddy/locks/issue_cert_mitchflix.net.lock: no such file or directory"}
Sep 26 13:35:50 calcifer caddy[1131]: {"level":"error","ts":1727321750.2720857,"logger":"tls","msg":"job failed","error":"mitchflix.net: renewing certificate: context canceled"}
Sep 26 13:35:50 calcifer caddy[1131]: {"level":"info","ts":1727321750.2721326,"logger":"admin","msg":"stopped previous server","address":"localhost:2019"}
Sep 26 13:35:50 calcifer caddy[1131]: {"level":"info","ts":1727321750.2721395,"logger":"admin.api","msg":"shutdown complete","exit_code":0}
Sep 26 13:35:50 calcifer systemd[1]: caddy.service: Deactivated successfully.
3. Caddy version:
v2.8.4 h1:q3pe0wpBj1OcHFZ3n/1nl4V4bxBrYoSoab7rL9BMYNk=
4. How I installed and ran Caddy:
Caddy is run via the “caddy” command. It automatically starts on server start. Caddy is hosted locally on an Ubuntu machine, and routed to my Cloudflare domain: ‘mitchflix.net’
a. System environment:
Ubuntu 22.04 (I am aware new update came out, I want to fix this before updating to avoid complications). Hosted via caddy command line, no docker or systemd.
b. Command:
caddy stop
caddy start
c. Service/unit/compose file:
d. My complete Caddy config:
When entering
caddy fmt
I receive:
Error: reading input file: open Caddyfile: no such file or directory
My Caddyfile is hosted in /etc/caddy/Caddyfile and reads:
mitchflix.net {
reverse_proxy 192.168.1.31:8096
tls {
issuer acme {
dns cloudflare {env.CF_API_TOKEN}
resolvers 1.1.1.1
propagation_delay 60s
propagation_timeout -1
}
}
}
5. Links to relevant resources:
I attempted to first google/fix the issue. I followed Francis’ instruction at: NET::ERR_CERT_AUTHORITY_INVALID · Issue #6133 · caddyserver/caddy · GitHub to enter:
sudo caddy trust --address localhost:3019
This port was ‘2019’ on my machine so I assumed that was a typo, and entered 2019 instead. This gave me:
2024/09/26 03:33:58.045 WARN installing root certificate (you might be prompted for password) {"path": "localhost:2019/pki/ca/local"}
2024/09/26 03:33:58.045 INFO warning: "certutil" is not available, install "certutil" with "apt install libnss3-tools" or "yum install nss-tools" and try again
2024/09/26 03:33:58.045 INFO define JAVA_HOME environment variable to use the Java trust
2024/09/26 03:33:58.565 INFO certificate installed properly in linux trusts
I installed the suggested tools with the command provided, and now see:
ERR_CONNECTION_TIMED_OUT
