Reverse Proxy with DynamicDNS and Cloudflare not working

Help
1

1. The problem I’m having:

I’ve created a caddy file using two modules: dynamic_dns and the cloudflare module. I’ve set everything up, after inspiring from other forum posts who got it working. However, when I try to access the website (from within my house) i get the following error in the browser: Error code: SSL_ERROR_INTERNAL_ERROR_ALERT.

I also am not sure if in Cloudflare I should create DNS records for each subdomain individually or if I can just use * as the name.

2. Error messages and/or full log output:

It looks like caddy cannot obtain certificates from letsencrypt. The Virtual Machine where Caddy Runs is named docker01. Caddy does not run in a docker environment.

May 08 22:30:40 docker01 caddy[420603]: {"level":"debug","ts":1715207440.940805,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-v02.api.letsencrypt.org/acme/new-order","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.8.0-beta.2 CertMagic acmez (linux; amd64)"]},"response_headers":{"Boulder-Requester":["1716261317"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["107"],"Content-Type":["application/problem+json"],"Date":["Wed, 08 May 2024 22:30:40 GMT"],"Link":["<https://acme-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["KBbvukTesQyt53KSul4U8-b_bPaRm4K_OQ-zWm_YDRGjPrFBoAk"],"Server":["nginx"]},"status_code":400}
May 08 22:30:40 docker01 caddy[420603]: {"level":"error","ts":1715207440.941141,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"snellius.gorgonea.nl","issuer":"acme-v02.api.letsencrypt.org-directory","error":"HTTP 400 urn:ietf:params:acme:error:malformed - JWS verification error"}
May 08 22:30:40 docker01 caddy[420603]: {"level":"debug","ts":1715207440.9412513,"logger":"events","msg":"event","name":"cert_failed","id":"4dae8ef3-8a90-411a-844b-e6802328775b","origin":"tls","data":{"error":{},"identifier":"snellius.gorgonea.nl","issuers":["acme-v02.api.letsencrypt.org-directory"],"renewal":false}}
May 08 22:30:40 docker01 caddy[420603]: {"level":"error","ts":1715207440.941296,"logger":"tls.obtain","msg":"will retry","error":"[snellius.gorgonea.nl] Obtain: [snellius.gorgonea.nl] creating new order: attempt 1: https://acme-v02.api.letsencrypt.org/acme/new-order: HTTP 400 urn:ietf:params:acme:error:malformed - JWS verification error (ca=https://acme-v02.api.letsencrypt.org/directory)","attempt":1,"retrying_in":60,"elapsed":0.569225729,"max_duration":2592000}
May 08 22:30:46 docker01 caddy[420603]: {"level":"info","ts":1715207446.6757905,"logger":"dynamic_dns","msg":"finished updating DNS","current_ips":["31.20.131.61"]}
May 08 22:30:51 docker01 caddy[420603]: {"level":"info","ts":1715207451.6427345,"logger":"admin.api","msg":"received request","method":"POST","host":"localhost:2019","uri":"/load","remote_ip":"127.0.0.1","remote_port":"41036","headers":{"Accept-Encoding":["gzip"],"Content-Length":["945"],"Content-Type":["application/json"],"Origin":["http://localhost:2019"],"User-Agent":["Go-http-client/1.1"]}}
May 08 22:30:51 docker01 caddy[420603]: {"level":"info","ts":1715207451.6428804,"msg":"config is unchanged"}
May 08 22:30:51 docker01 caddy[420603]: {"level":"info","ts":1715207451.6429079,"logger":"admin.api","msg":"load complete"}



May 08 22:31:40 docker01 caddy[420603]: {"level":"info","ts":1715207500.9424124,"logger":"tls.obtain","msg":"obtaining certificate","identifier":"snellius.gorgonea.nl"}
May 08 22:31:40 docker01 caddy[420603]: {"level":"debug","ts":1715207500.9424968,"logger":"events","msg":"event","name":"cert_obtaining","id":"ea787d09-64f0-43fe-989b-f0c5eb1e6bf0","origin":"tls","data":{"identifier":"snellius.gorgonea.nl"}}
May 08 22:31:40 docker01 caddy[420603]: {"level":"debug","ts":1715207500.9426737,"logger":"tls.obtain","msg":"trying issuer 1/1","issuer":"acme-v02.api.letsencrypt.org-directory"}
May 08 22:31:40 docker01 caddy[420603]: {"level":"info","ts":1715207500.94283,"logger":"tls.issuance.acme","msg":"using ACME account","account_id":"https://acme-staging-v02.api.letsencrypt.org/acme/acct/147515174","account_contact":["mailto:<REDACTED_EMAIL>"]}
May 08 22:31:40 docker01 caddy[420603]: {"level":"debug","ts":1715207500.9428473,"logger":"tls.issuance.acme.acme_client","msg":"creating order","account":"https://acme-staging-v02.api.letsencrypt.org/acme/acct/147515174","identifiers":["snellius.gorgonea.nl"]}
May 08 22:31:41 docker01 caddy[420603]: {"level":"debug","ts":1715207501.4330933,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"HEAD","url":"https://acme-staging-v02.api.letsencrypt.org/acme/new-nonce","headers":{"User-Agent":["Caddy/2.8.0-beta.2 CertMagic acmez (linux; amd64)"]},"response_headers":{"Cache-Control":["public, max-age=0, no-cache"],"Date":["Wed, 08 May 2024 22:31:41 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["5RI07lRcFeMe_pVyXa4oB0-yIkIP-v-zRdx9oP9eZBvB0RH02ps"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":200}
May 08 22:31:41 docker01 caddy[420603]: {"level":"debug","ts":1715207501.6303957,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-staging-v02.api.letsencrypt.org/acme/new-order","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.8.0-beta.2 CertMagic acmez (linux; amd64)"]},"response_headers":{"Boulder-Requester":["147515174"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["357"],"Content-Type":["application/json"],"Date":["Wed, 08 May 2024 22:31:41 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Location":["https://acme-staging-v02.api.letsencrypt.org/acme/order/147515174/16397544664"],"Replay-Nonce":["5RI07lRcWu_VBcRloyN9-8NwW2GGqghNzwFQIudzp9P3wWtoFiM"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":201}
May 08 22:31:41 docker01 caddy[420603]: {"level":"debug","ts":1715207501.7933238,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/12288793884","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.8.0-beta.2 CertMagic acmez (linux; amd64)"]},"response_headers":{"Boulder-Requester":["147515174"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["522"],"Content-Type":["application/json"],"Date":["Wed, 08 May 2024 22:31:41 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["5RI07lRclakc1JAtglyKg4u4RMGnSy4XjtZiotHmxPgB9Ld0d3U"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":200}
May 08 22:31:41 docker01 caddy[420603]: {"level":"debug","ts":1715207501.793617,"logger":"tls.issuance.acme.acme_client","msg":"skipping challenge initiation because authorization is not pending","identifier":"snellius.gorgonea.nl","authz_status":"valid"}
May 08 22:31:41 docker01 caddy[420603]: {"level":"info","ts":1715207501.7936597,"logger":"tls.issuance.acme.acme_client","msg":"authorization finalized","identifier":"snellius.gorgonea.nl","authz_status":"valid"}
May 08 22:31:41 docker01 caddy[420603]: {"level":"info","ts":1715207501.7936847,"logger":"tls.issuance.acme.acme_client","msg":"validations succeeded; finalizing order","order":"https://acme-staging-v02.api.letsencrypt.org/acme/order/147515174/16397544664"}
May 08 22:31:41 docker01 caddy[420603]: {"level":"debug","ts":1715207501.980057,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-staging-v02.api.letsencrypt.org/acme/finalize/147515174/16397544664","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.8.0-beta.2 CertMagic acmez (linux; amd64)"]},"response_headers":{"Boulder-Requester":["147515174"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["362"],"Content-Type":["application/json"],"Date":["Wed, 08 May 2024 22:31:41 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Location":["https://acme-staging-v02.api.letsencrypt.org/acme/order/147515174/16397544664"],"Replay-Nonce":["hqsVkgou_4KbgGRcz5-2Cz7mT19pXnnPnnzdOj2RiMfb4Y8tRfg"],"Retry-After":["3"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":200}
May 08 22:31:45 docker01 caddy[420603]: {"level":"debug","ts":1715207505.148249,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-staging-v02.api.letsencrypt.org/acme/order/147515174/16397544664","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.8.0-beta.2 CertMagic acmez (linux; amd64)"]},"response_headers":{"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["469"],"Content-Type":["application/json"],"Date":["Wed, 08 May 2024 22:31:45 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["5RI07lRc1Uoqq1NuSnDD4KrVf2W7UdT1bnvqCCar7Mla1BYbXYA"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":200}
May 08 22:31:45 docker01 caddy[420603]: {"level":"debug","ts":1715207505.3170068,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-staging-v02.api.letsencrypt.org/acme/cert/2b0bcf47f3bca5e15b1aaf82e4b2070043ad","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.8.0-beta.2 CertMagic acmez (linux; amd64)"]},"response_headers":{"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["2994"],"Content-Type":["application/pem-certificate-chain"],"Date":["Wed, 08 May 2024 22:31:45 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\"","<https://acme-staging-v02.api.letsencrypt.org/acme/cert/2b0bcf47f3bca5e15b1aaf82e4b2070043ad/1>;rel=\"alternate\""],"Replay-Nonce":["5RI07lRc-Vumt-v2dojHYDxB49Y9R5TmbP6Nd2xclN47AN942gc"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":200}
May 08 22:31:45 docker01 caddy[420603]: {"level":"debug","ts":1715207505.317128,"logger":"tls.issuance.acme.acme_client","msg":"getting renewal info","names":["snellius.gorgonea.nl"]}
May 08 22:31:45 docker01 caddy[420603]: {"level":"debug","ts":1715207505.6393247,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"GET","url":"https://acme-staging-v02.api.letsencrypt.org/draft-ietf-acme-ari-02/renewalInfo//_EbRAUNfu3umPTBorhG64LxtydM.KwvPR_O8peFbGq-C5LIHAEOt","headers":{"User-Agent":["Caddy/2.8.0-beta.2 CertMagic acmez (linux; amd64)"]},"response_headers":{"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["101"],"Content-Type":["application/json"],"Date":["Wed, 08 May 2024 22:31:45 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Retry-After":["21600"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":200}
May 08 22:31:45 docker01 caddy[420603]: {"level":"info","ts":1715207505.6395576,"logger":"tls.issuance.acme.acme_client","msg":"got renewal info","names":["snellius.gorgonea.nl"],"window_start":1720302701,"window_end":1720475501,"selected_time":1720420747,"recheck_after":1715229105.6395473,"explanation_url":""}
May 08 22:31:45 docker01 caddy[420603]: {"level":"debug","ts":1715207505.8087447,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-staging-v02.api.letsencrypt.org/acme/cert/2b0bcf47f3bca5e15b1aaf82e4b2070043ad/1","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.8.0-beta.2 CertMagic acmez (linux; amd64)"]},"response_headers":{"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["2433"],"Content-Type":["application/pem-certificate-chain"],"Date":["Wed, 08 May 2024 22:31:45 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\"","<https://acme-staging-v02.api.letsencrypt.org/acme/cert/2b0bcf47f3bca5e15b1aaf82e4b2070043ad/0>;rel=\"alternate\""],"Replay-Nonce":["5RI07lRcerlhIEW0t6q4j2vRd65_wOYMz2gyr2eGG6qR5S4eXAY"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":200}
May 08 22:31:45 docker01 caddy[420603]: {"level":"debug","ts":1715207505.8091002,"logger":"tls.issuance.acme.acme_client","msg":"getting renewal info","names":["snellius.gorgonea.nl"]}
May 08 22:31:46 docker01 caddy[420603]: {"level":"debug","ts":1715207506.130292,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"GET","url":"https://acme-staging-v02.api.letsencrypt.org/draft-ietf-acme-ari-02/renewalInfo//_EbRAUNfu3umPTBorhG64LxtydM.KwvPR_O8peFbGq-C5LIHAEOt","headers":{"User-Agent":["Caddy/2.8.0-beta.2 CertMagic acmez (linux; amd64)"]},"response_headers":{"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["101"],"Content-Type":["application/json"],"Date":["Wed, 08 May 2024 22:31:46 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Retry-After":["21600"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":200}
May 08 22:31:46 docker01 caddy[420603]: {"level":"info","ts":1715207506.1304204,"logger":"tls.issuance.acme.acme_client","msg":"got renewal info","names":["snellius.gorgonea.nl"],"window_start":1720302701,"window_end":1720475501,"selected_time":1720319364,"recheck_after":1715229106.1304157,"explanation_url":""}
May 08 22:31:46 docker01 caddy[420603]: {"level":"info","ts":1715207506.130465,"logger":"tls.issuance.acme.acme_client","msg":"successfully downloaded available certificate chains","count":2,"first_url":"https://acme-staging-v02.api.letsencrypt.org/acme/cert/2b0bcf47f3bca5e15b1aaf82e4b2070043ad"}
May 08 22:31:46 docker01 caddy[420603]: {"level":"debug","ts":1715207506.130482,"logger":"tls.issuance.acme","msg":"selected certificate chain","url":"https://acme-staging-v02.api.letsencrypt.org/acme/cert/2b0bcf47f3bca5e15b1aaf82e4b2070043ad"}
May 08 22:31:46 docker01 caddy[420603]: {"level":"info","ts":1715207506.1308331,"logger":"tls.issuance.acme","msg":"waiting on internal rate limiter","identifiers":["snellius.gorgonea.nl"],"ca":"https://acme-v02.api.letsencrypt.org/directory","account":"<REDACTED_EMAIL>"}
May 08 22:31:46 docker01 caddy[420603]: {"level":"info","ts":1715207506.1308713,"logger":"tls.issuance.acme","msg":"done waiting on internal rate limiter","identifiers":["snellius.gorgonea.nl"],"ca":"https://acme-v02.api.letsencrypt.org/directory","account":"<REDACTED_EMAIL>"}
May 08 22:31:46 docker01 caddy[420603]: {"level":"info","ts":1715207506.1309032,"logger":"tls.issuance.acme","msg":"using ACME account","account_id":"https://acme-v02.api.letsencrypt.org/acme/acct/1716261317","account_contact":["mailto:<REDACTED_EMAIL>"]}
May 08 22:31:46 docker01 caddy[420603]: {"level":"debug","ts":1715207506.130923,"logger":"tls.issuance.acme.acme_client","msg":"creating order","account":"https://acme-v02.api.letsencrypt.org/acme/acct/1716261317","identifiers":["snellius.gorgonea.nl"]}
May 08 22:31:46 docker01 caddy[420603]: {"level":"debug","ts":1715207506.272741,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"HEAD","url":"https://acme-v02.api.letsencrypt.org/acme/new-nonce","headers":{"User-Agent":["Caddy/2.8.0-beta.2 CertMagic acmez (linux; amd64)"]},"response_headers":{"Cache-Control":["public, max-age=0, no-cache"],"Date":["Wed, 08 May 2024 22:31:46 GMT"],"Link":["<https://acme-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["KBbvukTeJ2_Lg4yE9oXtIIw3MYP1FvnuYE7_Ur2gw47AnGnIofE"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":200}
May 08 22:31:46 docker01 caddy[420603]: {"level":"debug","ts":1715207506.4081025,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-v02.api.letsencrypt.org/acme/new-order","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.8.0-beta.2 CertMagic acmez (linux; amd64)"]},"response_headers":{"Boulder-Requester":["1716261317"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["107"],"Content-Type":["application/problem+json"],"Date":["Wed, 08 May 2024 22:31:46 GMT"],"Link":["<https://acme-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["RHbEAk-Jo2s7UNXw0sALRVytazKxqONV73UMKHsLtVi5BiZCEws"],"Server":["nginx"]},"status_code":400}
May 08 22:31:46 docker01 caddy[420603]: {"level":"error","ts":1715207506.4084003,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"snellius.gorgonea.nl","issuer":"acme-v02.api.letsencrypt.org-directory","error":"HTTP 400 urn:ietf:params:acme:error:malformed - JWS verification error"}
May 08 22:31:46 docker01 caddy[420603]: {"level":"debug","ts":1715207506.4085228,"logger":"events","msg":"event","name":"cert_failed","id":"e21c8473-88e3-4058-b251-af64b266ef53","origin":"tls","data":{"error":{"Err":{}},"identifier":"snellius.gorgonea.nl","issuers":["acme-v02.api.letsencrypt.org-directory"],"renewal":false}}
May 08 22:31:46 docker01 caddy[420603]: {"level":"info","ts":1715207506.408604,"logger":"tls.obtain","msg":"releasing lock","identifier":"snellius.gorgonea.nl"}
May 08 22:31:46 docker01 caddy[420603]: {"level":"error","ts":1715207506.4088492,"logger":"tls","msg":"job failed","error":"snellius.gorgonea.nl: obtaining certificate: [snellius.gorgonea.nl] Obtain: [snellius.gorgonea.nl] creating new order: attempt 1: https://acme-v02.api.letsencrypt.org/acme/new-order: HTTP 400 urn:ietf:params:acme:error:malformed - JWS verification error (ca=https://acme-v02.api.letsencrypt.org/directory)"}

3. Caddy version:

Installed the most recent version : 2.8.0-beta.2
This is because i needed to use the master keyword when building caddy, dynamic_dns has an open issue about this (Module not building with xcaddy · Issue #67 · mholt/caddy-dynamicdns · GitHub)

4. How I installed and ran Caddy:

a. System environment:

            .-/+oossssoo+/-.               agorgan@docker01
        `:+ssssssssssssssssss+:`           ----------------
      -+ssssssssssssssssssyyssss+-         OS: Ubuntu 22.04.4 LTS x86_64
    .ossssssssssssssssssdMMMNysssso.       Host: KVM/QEMU (Standard PC (i440FX + PIIX, 1996) pc-i440fx-8.1)
   /ssssssssssshdmmNNmmyNMMMMhssssss/      Kernel: 5.15.0-102-generic
  +ssssssssshmydMMMMMMMNddddyssssssss+     Uptime: 23 days, 10 hours, 47 mins
 /sssssssshNMMMyhhyyyyhmNMMMNhssssssss/    Packages: 881 (dpkg), 5 (snap)
.ssssssssdMMMNhsssssssssshNMMMdssssssss.   Shell: bash 5.1.16
+sssshhhyNMMNyssssssssssssyNMMMysssssss+   Resolution: 1280x800
ossyNMMMNyMMhsssssssssssssshmmmhssssssso   Terminal: /dev/pts/6
ossyNMMMNyMMhsssssssssssssshmmmhssssssso   CPU: QEMU Virtual version 2.5+ (4) @ 3.095GHz
+sssshhhyNMMNyssssssssssssyNMMMysssssss+   GPU: 00:02.0 Vendor 1234 Device 1111
.ssssssssdMMMNhsssssssssshNMMMdssssssss.   Memory: 855MiB / 7937MiB
 /sssssssshNMMMyhhyyyyhdNMMMNhssssssss/
  +sssssssssdmydMMMMMMMMddddyssssssss+
   /ssssssssssshdmNNNNmyNMMMMhssssss/
    .ossssssssssssssssssdMMMNysssso.
      -+sssssssssssssssssyyyssss+-
        `:+ssssssssssssssssss+:`
            .-/+oossssoo+/-.

b. Command:

### Editing /etc/systemd/system/caddy.service.d/override.conf
### Anything between here and the comment below will become the new contents of the file
[Service]
Environment="CLOUDFLARE_API_TOKEN=realkeyhere"

### Lines below this comment will be discarded

### /etc/systemd/system/caddy.service
# # caddy.service
# #
# # For using Caddy with a config file.
# #
# # Make sure the ExecStart and ExecReload commands are correct
# # for your installation.
# #
# # See https://caddyserver.com/docs/install for instructions.
# #
# # WARNING: This service does not use the --resume flag, so if you
# # use the API to make changes, they will be overwritten by the
# # Caddyfile next time the service is restarted. If you intend to
# # use Caddy's API to configure it, add the --resume flag to the
# # `caddy run` command or use the caddy-api.service file instead.
# 
# [Unit]
# Description=Caddy
# Documentation=https://caddyserver.com/docs/
# After=network.target network-online.target
# Requires=network-online.target
# 
# [Service]
# Type=notify
# User=caddy
# Group=caddy
# ExecStart=/usr/bin/caddy run --environ --config /etc/caddy/Caddyfile
# ExecReload=/usr/bin/caddy reload --config /etc/caddy/Caddyfile --force
# TimeoutStopSec=5s
# LimitNOFILE=1048576
# LimitNPROC=512
# PrivateTmp=true
# ProtectSystem=full
# AmbientCapabilities=CAP_NET_ADMIN CAP_NET_BIND_SERVICE
# 
# [Install]
# WantedBy=multi-user.target

c. Service/unit/compose file:

#For Starting Caddy
systemctl start caddy

#To Reload the config
caddy reload

d. My complete Caddy config:

{
	email <REDACTED>

	acme_dns cloudflare {env.CLOUDFLARE_API_TOKEN}

	debug

	dynamic_dns {
		provider cloudflare {env.CLOUDFLARE_API_TOKEN}
		domains {
			gorgonea.nl @ www
		}
		dynamic_domains
		versions ipv4
	}
}

home.gorgonea.nl {
	tls {
		dns cloudflare {env.CLOUDFLARE_API_TOKEN}
	}

	reverse_proxy 192.168.1.50:8123
}

snellius.gorgonea.nl {
	tls {
		dns cloudflare {env.CLOUDFLARE_API_TOKEN}
	}

	reverse_proxy 192.168.1.50:8006
}

5. Links to relevant resources:

2

What do you have in your Caddy data directory, under acme//users?

3

That’s a weird one. We’ve seen this before, but we’ve never really been able to track down why it happens.

As Matt said, if you can dump what’s in /var/lib/caddy/.local/share/caddy/acme/users, might help us understand what’s going on (don’t share your private key from there though)

Afterwards, maybe you can clear our Caddy’s storage then restart, to try to start from a fresh state, might fix it. Wipe out /var/lib/caddy/.local/share/caddy then restart the service.

1 Like
4

Thanks for the reply, I cannot seem to find the /users directory.

user@docker01:~# ls -lah /var/lib/caddy/.local/share/caddy/acme/
total 20K
drwx------ 5 caddy caddy 4.0K May  8 21:04 .
drwx------ 6 caddy caddy 4.0K May  8 22:30 ..
drwx------ 4 caddy caddy 4.0K May  8 20:12 acme-staging-v02.api.letsencrypt.org-directory
drwx------ 4 caddy caddy 4.0K May  8 20:11 acme-v02.api.letsencrypt.org-directory
drwx------ 3 caddy caddy 4.0K May  8 21:04 acme.zerossl.com-v2-dv90
root@docker01:~# cat /var/lib/caddy/.local/share/caddy/acme/users
cat: /var/lib/caddy/.local/share/caddy/acme/users: No such file or directory
user@docker01:~#

Update: I’ve found it in one of the acme directories (only looked at the one that also appears in the error message):

user@docker01:~# ls -lah /var/lib/caddy/.local/share/caddy/acme/acme-v02.api.letsencrypt.org-directory/
total 16K
drwx------ 4 caddy caddy 4.0K May  8 20:11 .
drwx------ 5 caddy caddy 4.0K May  8 21:04 ..
drwx------ 2 caddy caddy 4.0K May  8 20:16 challenge_tokens
drwx------ 4 caddy caddy 4.0K May  8 21:04 users
user@docker01:~# ls -lah /var/lib/caddy/.local/share/caddy/acme/acme-v02.api.letsencrypt.org-directory/users/
total 16K
drwx------ 4 caddy caddy 4.0K May  8 21:04 .
drwx------ 4 caddy caddy 4.0K May  8 20:11 ..
drwx------ 2 caddy caddy 4.0K May  8 21:04 <MY_EMAIL_ADDRESS>@gmail.com
drwx------ 2 caddy caddy 4.0K May  8 20:11 default
user@docker01:~# ls -lah /var/lib/caddy/.local/share/caddy/acme/acme-v02.api.letsencrypt.org-directory/users/andrei.gorgan01@gmail.com/
total 16K
drwx------ 2 caddy caddy 4.0K May  8 21:04 .
drwx------ 4 caddy caddy 4.0K May  8 21:04 ..
-rw------- 1 caddy caddy  197 May  8 21:04  <MY_EMAIL_ADDRESS>.json
-rw------- 1 caddy caddy  227 May  8 21:04 <MY_EMAIL_ADDRESS>.key
user@docker01:~# ls -lah /var/lib/caddy/.local/share/caddy/acme/acme-v02.api.letsencrypt.org-directory/users/default
total 16K
drwx------ 2 caddy caddy 4.0K May  8 20:11 .
drwx------ 4 caddy caddy 4.0K May  8 21:04 ..
-rw------- 1 caddy caddy  142 May  8 20:11 default.json
-rw------- 1 caddy caddy  227 May  8 20:11 default.key

Looking inside, both JSONs of my <USER_EMAIL> one seems fine:

# One for my user:
{
        "status": "valid",
        "contact": [
                "mailto:<USER_EMAIL_ADDRESS>@gmail.com"
        ],
        "termsOfServiceAgreed": true,
        "orders": "",
        "location": "https://acme-v02.api.letsencrypt.org/acme/acct/1716261317"
}

# Default One:
{
        "status": "valid",
        "termsOfServiceAgreed": true,
        "orders": "",
        "location": "https://acme-v02.api.letsencrypt.org/acme/acct/1716200097"
}
5

Sorry for spamming with another post. Seems like clearing out the .local/share/caddy directory and restarting caddy has fixed the error . I now don’t get any error for TLS, however I still cannot reach my services .

May 09 07:34:34 docker01 caddy[423368]: {"level":"error","ts":1715240074.8318574,"logger":"http.log.error","msg":"dial tcp 192.168.1.50:8123: connect: connection refused","request":{"remote_ip":"31.20.131.61","remote_port":"45397","client_ip":"31.20.131.61","proto":"HTTP/2.0","method":"GET","host":"home.gorgonea.nl","uri":"/service_worker.js","headers":{"Sec-Fetch-Mode":["same-origin"],"Pragma":["no-cache"],"Cache-Control":["no-cache"],"Te":["trailers"],"Accept":["*/*"],"Accept-Language":["en-GB,en;q=0.5"],"Accept-Encoding":["gzip, deflate, br"],"Cookie":["REDACTED"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:125.0) Gecko/20100101 Firefox/125.0"],"Service-Worker":["script"],"Sec-Fetch-Dest":["serviceworker"],"Sec-Fetch-Site":["same-origin"]},"tls":{"resumed":false,"version":772,"cipher_suite":4867,"proto":"h2","server_name":"home.gorgonea.nl"}},"duration":0.001360666,"status":502,"err_id":"kppz80i5k","err_trace":"reverseproxy.statusError (reverseproxy.go:1269)"}

I’m not sure what I am doing wrong

6

Thanks, yeah the users stuff looks normal :man_shrugging: Matt might have thoughts. Obviously I had you wipe it out so not really possible to dig deeper anymore :joy: but I dunno.

This means Caddy wasn’t able to connect to your app. Are you sure that’s the correct IP and port? Does that machine have somekind of firewall blocking incoming connections? Either way, it’s a networking problem between Caddy and your app, not a problem with Caddy itself at this point.

1 Like
7

Alright! Thanks, i’ll look into that and figure it out

8

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.