Help Noob With SSL + DDNS

1. The problem I’m having:

I am trying to setup a redirect from my domain (smithbury.com) to a Homarr dashboard with plans to then add more redirects from subdomains. E.g. plex.smithbury.

When accessing smithbury.com from a domain I receive an SSL_ERROR_INTERNAL_ERROR_ALERT error.

Using curl -vL, I get the following output.


curl -vL https://smithbury.com

* Trying 203.59.204.167:443...

* Connected to smithbury.com (203.59.204.167) port 443 (#0)

* ALPN, offering h2

* ALPN, offering http/1.1

* CAfile: /etc/ssl/certs/ca-certificates.crt

* CApath: /usr/lib/ssl/certs

* TLSv1.0 (OUT), TLS header, Certificate Status (22):

* TLSv1.3 (OUT), TLS handshake, Client hello (1):

* TLSv1.2 (IN), TLS header, Unknown (21):

* TLSv1.3 (IN), TLS alert, internal error (592):

* error:0A000438:SSL routines::tlsv1 alert internal error

* Closing connection 0

curl: (35) error:0A000438:SSL routines::tlsv1 alert internal error

2. Error messages and/or full log output:


May 20 12:23:13 mercury caddy[13161]: {"level":"error","ts":1716207793.711178,"logger":"tls.issuance.zerossl.acme_client","msg":"cleaning up solver","identifier":"www.smithbury.com","challenge_type":"dns-01","error":"no memory of presenting a DNS record for \"_acme-challenge.www.smithbury.com\" (usually OK if presenting also failed)"}

May 20 12:23:14 mercury caddy[13161]: {"level":"debug","ts":1716207794.0426433,"logger":"tls.issuance.zerossl.acme_client","msg":"http request","method":"POST","url":"https://acme.zerossl.com/v2/DV90/authz/mL_vwfeLhjXMPZWT9lfKEQ","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.7.6 CertMagic acmez (linux; amd64)"]},"response_headers":{"Access-Control-Allow-Origin":["*"],"Cache-Control":["max-age=0, no-cache, no-store"],"Content-Length":["129"],"Content-Type":["application/json"],"Date":["Mon, 20 May 2024 12:23:13 GMT"],"Link":["<https://acme.zerossl.com/v2/DV90>;rel=\"index\""],"Replay-Nonce":["ZstMlgkNYz9IkwIL9ee5dIS5MaWp5Djg7wcYhQYxeYY"],"Server":["nginx"],"Strict-Transport-Security":["max-age=15724800; includeSubDomains"]},"status_code":200}

May 20 12:23:14 mercury caddy[13161]: {"level":"error","ts":1716207794.0430124,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"www.smithbury.com","issuer":"acme.zerossl.com-v2-DV90","error":"[www.smithbury.com] solving challenges: presenting for challenge: adding temporary record for zone \"smithbury.com.\": got error status: HTTP 400: [{Code:6003 Message:Invalid request headers ErrorChain:[{Code:6111 Message:Invalid format for Authorization header}]}] (order=https://acme.zerossl.com/v2/DV90/order/igBH1w1EdpjYgyjyYabGWw) (ca=https://acme.zerossl.com/v2/DV90)"}

May 20 12:23:14 mercury caddy[13161]: {"level":"debug","ts":1716207794.0430863,"logger":"events","msg":"event","name":"cert_failed","id":"82312e4a-a6eb-4a98-984e-949fe01c9c36","origin":"tls","data":{"error":{},"identifier":"www.smithbury.com","issuers":["acme-v02.api.letsencrypt.org-directory","acme.zerossl.com-v2-DV90"],"renewal":false}}

May 20 12:23:14 mercury caddy[13161]: {"level":"error","ts":1716207794.043133,"logger":"tls.obtain","msg":"will retry","error":"[www.smithbury.com] Obtain: [www.smithbury.com] solving challenges: presenting for challenge: adding temporary record for zone \"smithbury.com.\": got error status: HTTP 400: [{Code:6003 Message:Invalid request headers ErrorChain:[{Code:6111 Message:Invalid format for Authorization header}]}] (order=https://acme.zerossl.com/v2/DV90/order/igBH1w1EdpjYgyjyYabGWw) (ca=https://acme.zerossl.com/v2/DV90)","attempt":2,"retrying_in":120,"elapsed":68.216015855,"max_duration":2592000}

May 20 12:25:14 mercury caddy[13161]: {"level":"info","ts":1716207914.043403,"logger":"tls.obtain","msg":"obtaining certificate","identifier":"www.smithbury.com"}

May 20 12:25:14 mercury caddy[13161]: {"level":"debug","ts":1716207914.0437398,"logger":"events","msg":"event","name":"cert_obtaining","id":"a64708bd-f1cd-4e1c-8cfe-1ab9a14ab5d8","origin":"tls","data":{"identifier":"www.smithbury.com"}}

May 20 12:25:14 mercury caddy[13161]: {"level":"debug","ts":1716207914.0438967,"logger":"tls.obtain","msg":"trying issuer 1/2","issuer":"acme-v02.api.letsencrypt.org-directory"}

May 20 12:25:14 mercury caddy[13161]: {"level":"debug","ts":1716207914.2904918,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"HEAD","url":"https://acme-staging-v02.api.letsencrypt.org/acme/new-nonce","headers":{"User-Agent":["Caddy/2.7.6 CertMagic acmez (linux; amd64)"]},"response_headers":{"Cache-Control":["public, max-age=0, no-cache"],"Date":["Mon, 20 May 2024 12:25:14 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["Ppg7Meef21ntALKfQvqLzgxbbANOKqMUqGpE2BVv_VHJ-yVAPWk"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":200}

May 20 12:25:14 mercury caddy[13161]: {"level":"debug","ts":1716207914.5803177,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-staging-v02.api.letsencrypt.org/acme/new-order","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.7.6 CertMagic acmez (linux; amd64)"]},"response_headers":{"Boulder-Requester":["148725774"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["356"],"Content-Type":["application/json"],"Date":["Mon, 20 May 2024 12:25:14 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Location":["https://acme-staging-v02.api.letsencrypt.org/acme/order/148725774/16650297074"],"Replay-Nonce":["jZ8PWoQvGKEj2T1ZI2p0vBEvlEwaIq41sVnv0ScDoA51nOnPvoI"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":201}

May 20 12:25:14 mercury caddy[13161]: {"level":"debug","ts":1716207914.8344986,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/12438691924","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.7.6 CertMagic acmez (linux; amd64)"]},"response_headers":{"Boulder-Requester":["148725774"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["822"],"Content-Type":["application/json"],"Date":["Mon, 20 May 2024 12:25:14 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["Ppg7Meef64bCy1oa0MKB_g1zPOyfA5tw9fcIxc6POuv1sQjHXwE"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":200}

May 20 12:25:14 mercury caddy[13161]: {"level":"debug","ts":1716207914.834714,"logger":"tls.issuance.acme.acme_client","msg":"no solver configured","challenge_type":"tls-alpn-01"}

May 20 12:25:14 mercury caddy[13161]: {"level":"debug","ts":1716207914.8347607,"logger":"tls.issuance.acme.acme_client","msg":"no solver configured","challenge_type":"http-01"}

May 20 12:25:14 mercury caddy[13161]: {"level":"info","ts":1716207914.8347926,"logger":"tls.issuance.acme.acme_client","msg":"trying to solve challenge","identifier":"www.smithbury.com","challenge_type":"dns-01","ca":"https://acme-staging-v02.api.letsencrypt.org/directory"}

May 20 12:25:15 mercury caddy[13161]: {"level":"error","ts":1716207915.8539267,"logger":"tls.issuance.acme.acme_client","msg":"cleaning up solver","identifier":"www.smithbury.com","challenge_type":"dns-01","error":"no memory of presenting a DNS record for \"_acme-challenge.www.smithbury.com\" (usually OK if presenting also failed)"}

May 20 12:25:16 mercury caddy[13161]: {"level":"debug","ts":1716207916.113597,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/12438691924","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.7.6 CertMagic acmez (linux; amd64)"]},"response_headers":{"Boulder-Requester":["148725774"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["826"],"Content-Type":["application/json"],"Date":["Mon, 20 May 2024 12:25:15 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["Ppg7MeefnJrJDu0U_04tMicZp0-t1K8QKybB5RARCeYxV59OX3M"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":200}

May 20 12:25:16 mercury caddy[13161]: {"level":"error","ts":1716207916.11385,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"www.smithbury.com","issuer":"acme-v02.api.letsencrypt.org-directory","error":"[www.smithbury.com] solving challenges: presenting for challenge: adding temporary record for zone \"smithbury.com.\": got error status: HTTP 400: [{Code:6003 Message:Invalid request headers ErrorChain:[{Code:6111 Message:Invalid format for Authorization header}]}] (order=https://acme-staging-v02.api.letsencrypt.org/acme/order/148725774/16650297074) (ca=https://acme-staging-v02.api.letsencrypt.org/directory)"}

May 20 12:25:16 mercury caddy[13161]: {"level":"debug","ts":1716207916.1139154,"logger":"tls.obtain","msg":"trying issuer 2/2","issuer":"acme.zerossl.com-v2-DV90"}

May 20 12:25:17 mercury caddy[13161]: {"level":"debug","ts":1716207917.2459757,"logger":"tls.issuance.zerossl.acme_client","msg":"http request","method":"HEAD","url":"https://acme.zerossl.com/v2/DV90/newNonce","headers":{"User-Agent":["Caddy/2.7.6 CertMagic acmez (linux; amd64)"]},"response_headers":{"Access-Control-Allow-Origin":["*"],"Cache-Control":["max-age=0, no-cache, no-store"],"Content-Type":["application/octet-stream"],"Date":["Mon, 20 May 2024 12:25:17 GMT"],"Link":["<https://acme.zerossl.com/v2/DV90>;rel=\"index\""],"Replay-Nonce":["891LGwdmDPz8nABVbWTDAz9LdrfsBNaDr6WSfsySkrM"],"Server":["nginx"],"Strict-Transport-Security":["max-age=15724800; includeSubDomains"]},"status_code":200}

May 20 12:25:17 mercury caddy[13161]: {"level":"debug","ts":1716207917.6100025,"logger":"tls.issuance.zerossl.acme_client","msg":"http request","method":"POST","url":"https://acme.zerossl.com/v2/DV90/newOrder","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.7.6 CertMagic acmez (linux; amd64)"]},"response_headers":{"Access-Control-Allow-Origin":["*"],"Cache-Control":["max-age=0, no-cache, no-store"],"Content-Length":["279"],"Content-Type":["application/json"],"Date":["Mon, 20 May 2024 12:25:17 GMT"],"Location":["https://acme.zerossl.com/v2/DV90/order/_0gG-op1JEvx4RXTijDSGg"],"Replay-Nonce":["cQckw4e3KuouTHVUBq1aihx0ZYu8Y_9OTySquji_Cgw"],"Server":["nginx"],"Strict-Transport-Security":["max-age=15724800; includeSubDomains"]},"status_code":201}

May 20 12:25:17 mercury caddy[13161]: {"level":"debug","ts":1716207917.9054577,"logger":"tls.issuance.zerossl.acme_client","msg":"http request","method":"POST","url":"https://acme.zerossl.com/v2/DV90/authz/D8hWb7hZZtJwD7tEm8XRCg","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.7.6 CertMagic acmez (linux; amd64)"]},"response_headers":{"Access-Control-Allow-Origin":["*"],"Cache-Control":["max-age=0, no-cache, no-store"],"Content-Length":["447"],"Content-Type":["application/json"],"Date":["Mon, 20 May 2024 12:25:17 GMT"],"Link":["<https://acme.zerossl.com/v2/DV90>;rel=\"index\""],"Replay-Nonce":["WfZgh_4WAgK8n7HD-f37znJwmFAFPcQ3d2eTw06yR5U"],"Retry-After":["5"],"Server":["nginx"],"Strict-Transport-Security":["max-age=15724800; includeSubDomains"]},"status_code":200}

May 20 12:25:17 mercury caddy[13161]: {"level":"debug","ts":1716207917.9056566,"logger":"tls.issuance.zerossl.acme_client","msg":"no solver configured","challenge_type":"http-01"}

May 20 12:25:17 mercury caddy[13161]: {"level":"info","ts":1716207917.9057152,"logger":"tls.issuance.zerossl.acme_client","msg":"trying to solve challenge","identifier":"www.smithbury.com","challenge_type":"dns-01","ca":"https://acme.zerossl.com/v2/DV90"}

May 20 12:25:18 mercury caddy[13161]: {"level":"error","ts":1716207918.967764,"logger":"tls.issuance.zerossl.acme_client","msg":"cleaning up solver","identifier":"www.smithbury.com","challenge_type":"dns-01","error":"no memory of presenting a DNS record for \"_acme-challenge.www.smithbury.com\" (usually OK if presenting also failed)"}

May 20 12:25:19 mercury caddy[13161]: {"level":"debug","ts":1716207919.3336048,"logger":"tls.issuance.zerossl.acme_client","msg":"http request","method":"POST","url":"https://acme.zerossl.com/v2/DV90/authz/D8hWb7hZZtJwD7tEm8XRCg","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.7.6 CertMagic acmez (linux; amd64)"]},"response_headers":{"Access-Control-Allow-Origin":["*"],"Cache-Control":["max-age=0, no-cache, no-store"],"Content-Length":["129"],"Content-Type":["application/json"],"Date":["Mon, 20 May 2024 12:25:19 GMT"],"Link":["<https://acme.zerossl.com/v2/DV90>;rel=\"index\""],"Replay-Nonce":["2CSp3xKQFSoxrSZ9jSXqO1T89t9h3uWAB38sJXt6qjE"],"Server":["nginx"],"Strict-Transport-Security":["max-age=15724800; includeSubDomains"]},"status_code":200}

May 20 12:25:19 mercury caddy[13161]: {"level":"error","ts":1716207919.333841,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"www.smithbury.com","issuer":"acme.zerossl.com-v2-DV90","error":"[www.smithbury.com] solving challenges: presenting for challenge: adding temporary record for zone \"smithbury.com.\": got error status: HTTP 400: [{Code:6003 Message:Invalid request headers ErrorChain:[{Code:6111 Message:Invalid format for Authorization header}]}] (order=https://acme.zerossl.com/v2/DV90/order/_0gG-op1JEvx4RXTijDSGg) (ca=https://acme.zerossl.com/v2/DV90)"}

May 20 12:25:19 mercury caddy[13161]: {"level":"debug","ts":1716207919.333914,"logger":"events","msg":"event","name":"cert_failed","id":"0caecbd0-a328-4e03-a011-701fa08eca06","origin":"tls","data":{"error":{},"identifier":"www.smithbury.com","issuers":["acme-v02.api.letsencrypt.org-directory","acme.zerossl.com-v2-DV90"],"renewal":false}}

May 20 12:25:19 mercury caddy[13161]: {"level":"error","ts":1716207919.3339562,"logger":"tls.obtain","msg":"will retry","error":"[www.smithbury.com] Obtain: [www.smithbury.com] solving challenges: presenting for challenge: adding temporary record for zone \"smithbury.com.\": got error status: HTTP 400: [{Code:6003 Message:Invalid request headers ErrorChain:[{Code:6111 Message:Invalid format for Authorization header}]}] (order=https://acme.zerossl.com/v2/DV90/order/_0gG-op1JEvx4RXTijDSGg) (ca=https://acme.zerossl.com/v2/DV90)","attempt":3,"retrying_in":120,"elapsed":193.506839302,"max_duration":2592000}

May 20 12:27:19 mercury caddy[13161]: {"level":"info","ts":1716208039.3351498,"logger":"tls.obtain","msg":"obtaining certificate","identifier":"www.smithbury.com"}

May 20 12:27:19 mercury caddy[13161]: {"level":"debug","ts":1716208039.33553,"logger":"events","msg":"event","name":"cert_obtaining","id":"84094fc3-feef-4b22-bb2e-4a7a76eb813a","origin":"tls","data":{"identifier":"www.smithbury.com"}}

May 20 12:27:19 mercury caddy[13161]: {"level":"debug","ts":1716208039.3356485,"logger":"tls.obtain","msg":"trying issuer 1/2","issuer":"acme-v02.api.letsencrypt.org-directory"}

May 20 12:27:19 mercury caddy[13161]: {"level":"debug","ts":1716208039.5825477,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"HEAD","url":"https://acme-staging-v02.api.letsencrypt.org/acme/new-nonce","headers":{"User-Agent":["Caddy/2.7.6 CertMagic acmez (linux; amd64)"]},"response_headers":{"Cache-Control":["public, max-age=0, no-cache"],"Date":["Mon, 20 May 2024 12:27:19 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["jZ8PWoQvRsNCKpx4wZNqMcaTGX1A8pAJQHZQ8lQ_LvMrQWFSxlM"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":200}

May 20 12:27:19 mercury caddy[13161]: {"level":"debug","ts":1716208039.8870912,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-staging-v02.api.letsencrypt.org/acme/new-order","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.7.6 CertMagic acmez (linux; amd64)"]},"response_headers":{"Boulder-Requester":["148725774"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["356"],"Content-Type":["application/json"],"Date":["Mon, 20 May 2024 12:27:19 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Location":["https://acme-staging-v02.api.letsencrypt.org/acme/order/148725774/16650323254"],"Replay-Nonce":["jZ8PWoQvQjH93ZYGyvTk6HbsE1e7KxrdJ0N_ds1yk2PaCpefl6c"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":201}

May 20 12:27:20 mercury caddy[13161]: {"level":"debug","ts":1716208040.1426091,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/12438707424","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.7.6 CertMagic acmez (linux; amd64)"]},"response_headers":{"Boulder-Requester":["148725774"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["822"],"Content-Type":["application/json"],"Date":["Mon, 20 May 2024 12:27:19 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["jZ8PWoQvlH7QriLdovJTHIpBtKUhvMqcwoIWsuLga6tZDRZUYCE"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":200}

May 20 12:27:20 mercury caddy[13161]: {"level":"debug","ts":1716208040.1428862,"logger":"tls.issuance.acme.acme_client","msg":"no solver configured","challenge_type":"tls-alpn-01"}

May 20 12:27:20 mercury caddy[13161]: {"level":"debug","ts":1716208040.1429424,"logger":"tls.issuance.acme.acme_client","msg":"no solver configured","challenge_type":"http-01"}

May 20 12:27:20 mercury caddy[13161]: {"level":"info","ts":1716208040.1429906,"logger":"tls.issuance.acme.acme_client","msg":"trying to solve challenge","identifier":"www.smithbury.com","challenge_type":"dns-01","ca":"https://acme-staging-v02.api.letsencrypt.org/directory"}

May 20 12:27:20 mercury caddy[13161]: {"level":"error","ts":1716208040.4170015,"logger":"tls.issuance.acme.acme_client","msg":"cleaning up solver","identifier":"www.smithbury.com","challenge_type":"dns-01","error":"no memory of presenting a DNS record for \"_acme-challenge.www.smithbury.com\" (usually OK if presenting also failed)"}

May 20 12:27:20 mercury caddy[13161]: {"level":"debug","ts":1716208040.677108,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/12438707424","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.7.6 CertMagic acmez (linux; amd64)"]},"response_headers":{"Boulder-Requester":["148725774"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["826"],"Content-Type":["application/json"],"Date":["Mon, 20 May 2024 12:27:20 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["jZ8PWoQvd2XkTDzi3Qb0ceOcVO_ANRbKLekZVi0GT6luArSC8MM"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":200}

May 20 12:27:20 mercury caddy[13161]: {"level":"error","ts":1716208040.6773512,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"www.smithbury.com","issuer":"acme-v02.api.letsencrypt.org-directory","error":"[www.smithbury.com] solving challenges: presenting for challenge: adding temporary record for zone \"smithbury.com.\": got error status: HTTP 400: [{Code:6003 Message:Invalid request headers ErrorChain:[{Code:6111 Message:Invalid format for Authorization header}]}] (order=https://acme-staging-v02.api.letsencrypt.org/acme/order/148725774/16650323254) (ca=https://acme-staging-v02.api.letsencrypt.org/directory)"}

May 20 12:27:20 mercury caddy[13161]: {"level":"debug","ts":1716208040.6774182,"logger":"tls.obtain","msg":"trying issuer 2/2","issuer":"acme.zerossl.com-v2-DV90"}

May 20 12:27:21 mercury caddy[13161]: {"level":"debug","ts":1716208041.8673778,"logger":"tls.issuance.zerossl.acme_client","msg":"http request","method":"HEAD","url":"https://acme.zerossl.com/v2/DV90/newNonce","headers":{"User-Agent":["Caddy/2.7.6 CertMagic acmez (linux; amd64)"]},"response_headers":{"Access-Control-Allow-Origin":["*"],"Cache-Control":["max-age=0, no-cache, no-store"],"Content-Type":["application/octet-stream"],"Date":["Mon, 20 May 2024 12:27:21 GMT"],"Link":["<https://acme.zerossl.com/v2/DV90>;rel=\"index\""],"Replay-Nonce":["IGK7TcEFtRXjC7hwcEXw332uYXHczRTvxTLrQrxXxG8"],"Server":["nginx"],"Strict-Transport-Security":["max-age=15724800; includeSubDomains"]},"status_code":200}

May 20 12:27:22 mercury caddy[13161]: {"level":"debug","ts":1716208042.1993606,"logger":"tls.issuance.zerossl.acme_client","msg":"http request","method":"POST","url":"https://acme.zerossl.com/v2/DV90/newOrder","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.7.6 CertMagic acmez (linux; amd64)"]},"response_headers":{"Access-Control-Allow-Origin":["*"],"Cache-Control":["max-age=0, no-cache, no-store"],"Content-Length":["279"],"Content-Type":["application/json"],"Date":["Mon, 20 May 2024 12:27:22 GMT"],"Location":["https://acme.zerossl.com/v2/DV90/order/NUfgf_MzGNlkSZPwNOb6EQ"],"Replay-Nonce":["8qzlOUVLr-8pK_f2_v8m7f4x1G-YIscZVEYLoWap5jI"],"Server":["nginx"],"Strict-Transport-Security":["max-age=15724800; includeSubDomains"]},"status_code":201}

May 20 12:27:22 mercury caddy[13161]: {"level":"debug","ts":1716208042.5140436,"logger":"tls.issuance.zerossl.acme_client","msg":"http request","method":"POST","url":"https://acme.zerossl.com/v2/DV90/authz/-EIOiQsF9f7HvNyKG1-HcQ","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.7.6 CertMagic acmez (linux; amd64)"]},"response_headers":{"Access-Control-Allow-Origin":["*"],"Cache-Control":["max-age=0, no-cache, no-store"],"Content-Length":["447"],"Content-Type":["application/json"],"Date":["Mon, 20 May 2024 12:27:22 GMT"],"Link":["<https://acme.zerossl.com/v2/DV90>;rel=\"index\""],"Replay-Nonce":["-4FDjoZsXnG7l84Rw5e-IsibSqQVRB2Tl78tp00N1Uw"],"Retry-After":["5"],"Server":["nginx"],"Strict-Transport-Security":["max-age=15724800; includeSubDomains"]},"status_code":200}

May 20 12:27:22 mercury caddy[13161]: {"level":"debug","ts":1716208042.5142744,"logger":"tls.issuance.zerossl.acme_client","msg":"no solver configured","challenge_type":"http-01"}

May 20 12:27:22 mercury caddy[13161]: {"level":"info","ts":1716208042.5143492,"logger":"tls.issuance.zerossl.acme_client","msg":"trying to solve challenge","identifier":"www.smithbury.com","challenge_type":"dns-01","ca":"https://acme.zerossl.com/v2/DV90"}

May 20 12:27:22 mercury caddy[13161]: {"level":"error","ts":1716208042.9367487,"logger":"tls.issuance.zerossl.acme_client","msg":"cleaning up solver","identifier":"www.smithbury.com","challenge_type":"dns-01","error":"no memory of presenting a DNS record for \"_acme-challenge.www.smithbury.com\" (usually OK if presenting also failed)"}

May 20 12:27:23 mercury caddy[13161]: {"level":"debug","ts":1716208043.2926402,"logger":"tls.issuance.zerossl.acme_client","msg":"http request","method":"POST","url":"https://acme.zerossl.com/v2/DV90/authz/-EIOiQsF9f7HvNyKG1-HcQ","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.7.6 CertMagic acmez (linux; amd64)"]},"response_headers":{"Access-Control-Allow-Origin":["*"],"Cache-Control":["max-age=0, no-cache, no-store"],"Content-Length":["129"],"Content-Type":["application/json"],"Date":["Mon, 20 May 2024 12:27:23 GMT"],"Link":["<https://acme.zerossl.com/v2/DV90>;rel=\"index\""],"Replay-Nonce":["c9pD1cDsLefowZC-KRDcfJ3PrHSCq7oJWoMQA__oZDQ"],"Server":["nginx"],"Strict-Transport-Security":["max-age=15724800; includeSubDomains"]},"status_code":200}

May 20 12:27:23 mercury caddy[13161]: {"level":"error","ts":1716208043.2928693,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"www.smithbury.com","issuer":"acme.zerossl.com-v2-DV90","error":"[www.smithbury.com] solving challenges: presenting for challenge: adding temporary record for zone \"smithbury.com.\": got error status: HTTP 400: [{Code:6003 Message:Invalid request headers ErrorChain:[{Code:6111 Message:Invalid format for Authorization header}]}] (order=https://acme.zerossl.com/v2/DV90/order/NUfgf_MzGNlkSZPwNOb6EQ) (ca=https://acme.zerossl.com/v2/DV90)"}

May 20 12:27:23 mercury caddy[13161]: {"level":"debug","ts":1716208043.292941,"logger":"events","msg":"event","name":"cert_failed","id":"5e023972-cdaa-4f33-be73-6346f4e0d847","origin":"tls","data":{"error":{},"identifier":"www.smithbury.com","issuers":["acme-v02.api.letsencrypt.org-directory","acme.zerossl.com-v2-DV90"],"renewal":false}}

May 20 12:27:23 mercury caddy[13161]: {"level":"error","ts":1716208043.292987,"logger":"tls.obtain","msg":"will retry","error":"[www.smithbury.com] Obtain: [www.smithbury.com] solving challenges: presenting for challenge: adding temporary record for zone \"smithbury.com.\": got error status: HTTP 400: [{Code:6003 Message:Invalid request headers ErrorChain:[{Code:6111 Message:Invalid format for Authorization header}]}] (order=https://acme.zerossl.com/v2/DV90/order/NUfgf_MzGNlkSZPwNOb6EQ) (ca=https://acme.zerossl.com/v2/DV90)","attempt":4,"retrying_in":300,"elapsed":317.465869952,"max_duration":2592000}

3. Caddy version:

v2.7.6 h1:w0NymbG2m9PcvKWsrXO6EEkY9Ru4FJK8uQbYcev1p3A=

4. How I installed and ran Caddy:

a. System environment:

  • Ubuntu Server x64

  • Custom build with Cloudflare dns plugin.

  • Systemd

b. Command:

  • Installed basic package first as per steps here.

  • Then download custom build using command

curl -o caddy "https://caddyserver.com/api/download?os=linux&arch=amd64&p=github.com%2Fcaddy-dns%2Fcloudflare&idempotency=98480474495547"

  • Then followed steps outlined here to insert the custom caddy build.

  • Note initially was using caddy reload a lot before realising that running as a service meant I had to be be using the reload service command.

c. Service/unit/compose file:


[Unit]

Description=Caddy

Documentation=https://caddyserver.com/docs/

After=network.target network-online.target

Requires=network-online.target

[Service]

Type=notify

User=caddy

Group=caddy

ExecStart=/usr/bin/caddy run --environ --config /etc/caddy/Caddyfile

ExecReload=/usr/bin/caddy reload --config /etc/caddy/Caddyfile --force

TimeoutStopSec=5s

LimitNOFILE=1048576

LimitNPROC=512

PrivateTmp=true

ProtectSystem=full

AmbientCapabilities=CAP_NET_ADMIN CAP_NET_BIND_SERVICE

[Install]

WantedBy=multi-user.target

d. My complete Caddy config:


{

debug

acme_dns cloudflare [CLOUDFLARE_GLOBAL_API_TOKEN]

}

https://www.smithbury.com {

# Homarr host and port

reverse_proxy https://192.168.1.118:7575

}

5. Links to relevant resources:

Too many resources to cover. 4 VM rebuilds over 3 days trying to troubleshoot the issue with about 16 hours of troubleshooting or reviewing resources. Not helpful info here sorry.

Hello @jeff,

There is no DNS Record to www.smithbury.com , just for smithbury.com
thus not able to resolve for www.smithbury.com.
Permanent link to this check report
https://dnsspy.io/scan/smithbury.com

Edit:
Here is the first one from above.

Thank you. I thought I had created the DNS record as below? Note I only just created the www one when I saw that was still in my Caddyfile but the others have been up for 24+ hours and using that site reports no DNS record.

I updated my Caddyfile:

{
	debug
	acme_dns cloudflare [CLOUDFLARE_GLOBAL_API_TOKEN]
}

https://smithbury.com {
	# Homarr host and port
	reverse_proxy https://192.168.1.118:7575
}

Service Logs

May 20 19:53:11 mercury systemd[1]: Reloaded Caddy.

May 20 19:53:11 mercury caddy[615]: {"level":"info","ts":1716234791.3937802,"logger":"admin","msg":"stopped previous server","address":"localhost:2019"}

May 20 19:53:11 mercury caddy[615]: {"level":"info","ts":1716234791.394832,"logger":"tls.obtain","msg":"lock acquired","identifier":"smithbury.com"}

May 20 19:53:11 mercury caddy[615]: {"level":"info","ts":1716234791.3949523,"logger":"tls.obtain","msg":"obtaining certificate","identifier":"smithbury.com"}

May 20 19:53:11 mercury caddy[615]: {"level":"debug","ts":1716234791.3950102,"logger":"events","msg":"event","name":"cert_obtaining","id":"ec3c5538-0a7a-4f6a-b080-36bdd42d7782","origin":"tls","data":{"identifier":"smithbury.com"}}

May 20 19:53:11 mercury caddy[615]: {"level":"debug","ts":1716234791.3951595,"logger":"tls.obtain","msg":"trying issuer 1/2","issuer":"acme-v02.api.letsencrypt.org-directory"}

May 20 19:53:11 mercury caddy[615]: {"level":"info","ts":1716234791.3952796,"logger":"tls.issuance.acme","msg":"waiting on internal rate limiter","identifiers":["smithbury.com"],"ca":"https://acme-v02.api.letsencrypt.org/directory","account":"caddy@zerossl.com"}

May 20 19:53:11 mercury caddy[615]: {"level":"info","ts":1716234791.3953211,"logger":"tls.issuance.acme","msg":"done waiting on internal rate limiter","identifiers":["smithbury.com"],"ca":"https://acme-v02.api.letsencrypt.org/directory","account":"caddy@zerossl.com"}

May 20 19:53:12 mercury caddy[615]: {"level":"debug","ts":1716234792.15799,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"HEAD","url":"https://acme-v02.api.letsencrypt.org/acme/new-nonce","headers":{"User-Agent":["Caddy/2.7.6 CertMagic acmez (linux; amd64)"]},"response_headers":{"Cache-Control":["public, max-age=0, no-cache"],"Date":["Mon, 20 May 2024 19:53:12 GMT"],"Link":["<https://acme-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["YEjsK0pisS1ln2HXvAkamNFkVKdx281IaZqtKSAlDO1cdkOKH-4"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":200}

May 20 19:53:12 mercury caddy[615]: {"level":"debug","ts":1716234792.6008067,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-v02.api.letsencrypt.org/acme/new-order","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.7.6 CertMagic acmez (linux; amd64)"]},"response_headers":{"Boulder-Requester":["1734525182"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["339"],"Content-Type":["application/json"],"Date":["Mon, 20 May 2024 19:53:12 GMT"],"Link":["<https://acme-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Location":["https://acme-v02.api.letsencrypt.org/acme/order/1734525182/271102643222"],"Replay-Nonce":["YEjsK0pi8QDL4W5r9KQ5YiuvCihGqfOf6Cg__RfM8z5CsQfiszg"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":201}

May 20 19:53:12 mercury caddy[615]: {"level":"debug","ts":1716234792.84901,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-v02.api.letsencrypt.org/acme/authz-v3/353256225712","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.7.6 CertMagic acmez (linux; amd64)"]},"response_headers":{"Boulder-Requester":["1734525182"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["797"],"Content-Type":["application/json"],"Date":["Mon, 20 May 2024 19:53:12 GMT"],"Link":["<https://acme-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["YEjsK0piIxVYG161qea7B3YVqwvVtohl2tONUmARx2uy8FVfWlg"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":200}

May 20 19:53:12 mercury caddy[615]: {"level":"debug","ts":1716234792.8492086,"logger":"tls.issuance.acme.acme_client","msg":"no solver configured","challenge_type":"http-01"}

May 20 19:53:12 mercury caddy[615]: {"level":"debug","ts":1716234792.8492575,"logger":"tls.issuance.acme.acme_client","msg":"no solver configured","challenge_type":"tls-alpn-01"}

May 20 19:53:12 mercury caddy[615]: {"level":"info","ts":1716234792.8492951,"logger":"tls.issuance.acme.acme_client","msg":"trying to solve challenge","identifier":"smithbury.com","challenge_type":"dns-01","ca":"https://acme-v02.api.letsencrypt.org/directory"}

May 20 19:53:13 mercury caddy[615]: {"level":"error","ts":1716234793.1115558,"logger":"tls.issuance.acme.acme_client","msg":"cleaning up solver","identifier":"smithbury.com","challenge_type":"dns-01","error":"no memory of presenting a DNS record for \"_acme-challenge.smithbury.com\" (usually OK if presenting also failed)"}

May 20 19:53:13 mercury caddy[615]: {"level":"debug","ts":1716234793.3842466,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-v02.api.letsencrypt.org/acme/authz-v3/353256225712","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.7.6 CertMagic acmez (linux; amd64)"]},"response_headers":{"Boulder-Requester":["1734525182"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["801"],"Content-Type":["application/json"],"Date":["Mon, 20 May 2024 19:53:13 GMT"],"Link":["<https://acme-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["YEjsK0piC5FwhcI6-ouGj0mFRPp_1qbdVNXvr6RCa_0-9XfTyI0"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":200}

May 20 19:53:13 mercury caddy[615]: {"level":"error","ts":1716234793.3844786,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"smithbury.com","issuer":"acme-v02.api.letsencrypt.org-directory","error":"[smithbury.com] solving challenges: presenting for challenge: adding temporary record for zone \"smithbury.com.\": got error status: HTTP 400: [{Code:6003 Message:Invalid request headers ErrorChain:[{Code:6111 Message:Invalid format for Authorization header}]}] (order=https://acme-v02.api.letsencrypt.org/acme/order/1734525182/271102643222) (ca=https://acme-v02.api.letsencrypt.org/directory)"}

May 20 19:53:13 mercury caddy[615]: {"level":"debug","ts":1716234793.3845654,"logger":"tls.obtain","msg":"trying issuer 2/2","issuer":"acme.zerossl.com-v2-DV90"}

May 20 19:53:13 mercury caddy[615]: {"level":"info","ts":1716234793.3847327,"logger":"tls.issuance.zerossl","msg":"waiting on internal rate limiter","identifiers":["smithbury.com"],"ca":"https://acme.zerossl.com/v2/DV90","account":"caddy@zerossl.com"}

May 20 19:53:13 mercury caddy[615]: {"level":"info","ts":1716234793.3847904,"logger":"tls.issuance.zerossl","msg":"done waiting on internal rate limiter","identifiers":["smithbury.com"],"ca":"https://acme.zerossl.com/v2/DV90","account":"caddy@zerossl.com"}

May 20 19:53:14 mercury caddy[615]: {"level":"debug","ts":1716234794.5847657,"logger":"tls.issuance.zerossl.acme_client","msg":"http request","method":"HEAD","url":"https://acme.zerossl.com/v2/DV90/newNonce","headers":{"User-Agent":["Caddy/2.7.6 CertMagic acmez (linux; amd64)"]},"response_headers":{"Access-Control-Allow-Origin":["*"],"Cache-Control":["max-age=0, no-cache, no-store"],"Content-Type":["application/octet-stream"],"Date":["Mon, 20 May 2024 19:53:14 GMT"],"Link":["<https://acme.zerossl.com/v2/DV90>;rel=\"index\""],"Replay-Nonce":["4fQJ_DP0Kn50TUoc7hKjA2ADHZylAjRLYkQCb0XQmpg"],"Server":["nginx"],"Strict-Transport-Security":["max-age=15724800; includeSubDomains"]},"status_code":200}

May 20 19:53:14 mercury caddy[615]: {"level":"debug","ts":1716234794.9366267,"logger":"tls.issuance.zerossl.acme_client","msg":"http request","method":"POST","url":"https://acme.zerossl.com/v2/DV90/newOrder","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.7.6 CertMagic acmez (linux; amd64)"]},"response_headers":{"Access-Control-Allow-Origin":["*"],"Cache-Control":["max-age=0, no-cache, no-store"],"Content-Length":["275"],"Content-Type":["application/json"],"Date":["Mon, 20 May 2024 19:53:14 GMT"],"Location":["https://acme.zerossl.com/v2/DV90/order/DwoRSU1hv07Qaz0Q5CIMZA"],"Replay-Nonce":["WciqaW4YZW85ZR8U7s_jezbw8Q8M4DC9FYbVYaaAaKU"],"Server":["nginx"],"Strict-Transport-Security":["max-age=15724800; includeSubDomains"]},"status_code":201}

May 20 19:53:15 mercury caddy[615]: {"level":"debug","ts":1716234795.3002725,"logger":"tls.issuance.zerossl.acme_client","msg":"http request","method":"POST","url":"https://acme.zerossl.com/v2/DV90/authz/buvfFyOCO31db0r8WTbfEQ","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.7.6 CertMagic acmez (linux; amd64)"]},"response_headers":{"Access-Control-Allow-Origin":["*"],"Cache-Control":["max-age=0, no-cache, no-store"],"Content-Length":["443"],"Content-Type":["application/json"],"Date":["Mon, 20 May 2024 19:53:15 GMT"],"Link":["<https://acme.zerossl.com/v2/DV90>;rel=\"index\""],"Replay-Nonce":["vnVQVt2aux_Ug5Y3nhQVMk5g5N6Y-LRLpfqlJVRXc0A"],"Retry-After":["5"],"Server":["nginx"],"Strict-Transport-Security":["max-age=15724800; includeSubDomains"]},"status_code":200}

May 20 19:53:15 mercury caddy[615]: {"level":"debug","ts":1716234795.3006194,"logger":"tls.issuance.zerossl.acme_client","msg":"no solver configured","challenge_type":"http-01"}

May 20 19:53:15 mercury caddy[615]: {"level":"info","ts":1716234795.3006997,"logger":"tls.issuance.zerossl.acme_client","msg":"trying to solve challenge","identifier":"smithbury.com","challenge_type":"dns-01","ca":"https://acme.zerossl.com/v2/DV90"}

May 20 19:53:15 mercury caddy[615]: {"level":"error","ts":1716234795.528751,"logger":"tls.issuance.zerossl.acme_client","msg":"cleaning up solver","identifier":"smithbury.com","challenge_type":"dns-01","error":"no memory of presenting a DNS record for \"_acme-challenge.smithbury.com\" (usually OK if presenting also failed)"}

May 20 19:53:15 mercury caddy[615]: {"level":"debug","ts":1716234795.8438647,"logger":"tls.issuance.zerossl.acme_client","msg":"http request","method":"POST","url":"https://acme.zerossl.com/v2/DV90/authz/buvfFyOCO31db0r8WTbfEQ","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.7.6 CertMagic acmez (linux; amd64)"]},"response_headers":{"Access-Control-Allow-Origin":["*"],"Cache-Control":["max-age=0, no-cache, no-store"],"Content-Length":["125"],"Content-Type":["application/json"],"Date":["Mon, 20 May 2024 19:53:15 GMT"],"Link":["<https://acme.zerossl.com/v2/DV90>;rel=\"index\""],"Replay-Nonce":["V-t9V3_KDF151YXahOSMoq-pv0Zzh6Zphf5e2KI6Fbc"],"Server":["nginx"],"Strict-Transport-Security":["max-age=15724800; includeSubDomains"]},"status_code":200}

May 20 19:53:15 mercury caddy[615]: {"level":"error","ts":1716234795.8443263,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"smithbury.com","issuer":"acme.zerossl.com-v2-DV90","error":"[smithbury.com] solving challenges: presenting for challenge: adding temporary record for zone \"smithbury.com.\": got error status: HTTP 400: [{Code:6003 Message:Invalid request headers ErrorChain:[{Code:6111 Message:Invalid format for Authorization header}]}] (order=https://acme.zerossl.com/v2/DV90/order/DwoRSU1hv07Qaz0Q5CIMZA) (ca=https://acme.zerossl.com/v2/DV90)"}

May 20 19:53:15 mercury caddy[615]: {"level":"debug","ts":1716234795.8444223,"logger":"events","msg":"event","name":"cert_failed","id":"4fffac51-daa3-4b59-b87a-1eac51a0073b","origin":"tls","data":{"error":{},"identifier":"smithbury.com","issuers":["acme-v02.api.letsencrypt.org-directory","acme.zerossl.com-v2-DV90"],"renewal":false}}

May 20 19:53:15 mercury caddy[615]: {"level":"error","ts":1716234795.8444917,"logger":"tls.obtain","msg":"will retry","error":"[smithbury.com] Obtain: [smithbury.com] solving challenges: presenting for challenge: adding temporary record for zone \"smithbury.com.\": got error status: HTTP 400: [{Code:6003 Message:Invalid request headers ErrorChain:[{Code:6111 Message:Invalid format for Authorization header}]}] (order=https://acme.zerossl.com/v2/DV90/order/DwoRSU1hv07Qaz0Q5CIMZA) (ca=https://acme.zerossl.com/v2/DV90)","attempt":1,"retrying_in":60,"elapsed":4.449584867,"max_duration":2592000}

May 20 19:54:15 mercury caddy[615]: {"level":"info","ts":1716234855.8448598,"logger":"tls.obtain","msg":"obtaining certificate","identifier":"smithbury.com"}

May 20 19:54:15 mercury caddy[615]: {"level":"debug","ts":1716234855.8453305,"logger":"events","msg":"event","name":"cert_obtaining","id":"f23b7fd7-8978-4108-8ccf-74a88ae94c91","origin":"tls","data":{"identifier":"smithbury.com"}}

May 20 19:54:15 mercury caddy[615]: {"level":"debug","ts":1716234855.845554,"logger":"tls.obtain","msg":"trying issuer 1/2","issuer":"acme-v02.api.letsencrypt.org-directory"}

May 20 19:54:16 mercury caddy[615]: {"level":"debug","ts":1716234856.5974386,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"HEAD","url":"https://acme-staging-v02.api.letsencrypt.org/acme/new-nonce","headers":{"User-Agent":["Caddy/2.7.6 CertMagic acmez (linux; amd64)"]},"response_headers":{"Cache-Control":["public, max-age=0, no-cache"],"Date":["Mon, 20 May 2024 19:54:16 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["Ppg7MeefNa_HB99NcrrgsMyN4w0ha1h-ywJ4tqqghAXKAerxVHo"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":200}

May 20 19:54:16 mercury caddy[615]: {"level":"debug","ts":1716234856.896629,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-staging-v02.api.letsencrypt.org/acme/new-order","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.7.6 CertMagic acmez (linux; amd64)"]},"response_headers":{"Boulder-Requester":["148725774"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["352"],"Content-Type":["application/json"],"Date":["Mon, 20 May 2024 19:54:16 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Location":["https://acme-staging-v02.api.letsencrypt.org/acme/order/148725774/16656323274"],"Replay-Nonce":["Ppg7MeefSk1Llq6040vQ_W5GcR_vNbbB2fv1qIMkFsnkhC2LKcc"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":201}

May 20 19:54:17 mercury caddy[615]: {"level":"debug","ts":1716234857.1456597,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/12442343974","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.7.6 CertMagic acmez (linux; amd64)"]},"response_headers":{"Boulder-Requester":["148725774"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["818"],"Content-Type":["application/json"],"Date":["Mon, 20 May 2024 19:54:16 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["jZ8PWoQv3wUBy6LN21tVRBDYi8PrwZfiniWN8MgUvfOEXJid9xY"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":200}

May 20 19:54:17 mercury caddy[615]: {"level":"debug","ts":1716234857.1459064,"logger":"tls.issuance.acme.acme_client","msg":"no solver configured","challenge_type":"http-01"}

May 20 19:54:17 mercury caddy[615]: {"level":"debug","ts":1716234857.1459653,"logger":"tls.issuance.acme.acme_client","msg":"no solver configured","challenge_type":"tls-alpn-01"}

May 20 19:54:17 mercury caddy[615]: {"level":"info","ts":1716234857.1460023,"logger":"tls.issuance.acme.acme_client","msg":"trying to solve challenge","identifier":"smithbury.com","challenge_type":"dns-01","ca":"https://acme-staging-v02.api.letsencrypt.org/directory"}

May 20 19:54:17 mercury caddy[615]: {"level":"error","ts":1716234857.372654,"logger":"tls.issuance.acme.acme_client","msg":"cleaning up solver","identifier":"smithbury.com","challenge_type":"dns-01","error":"no memory of presenting a DNS record for \"_acme-challenge.smithbury.com\" (usually OK if presenting also failed)"}

May 20 19:54:17 mercury caddy[615]: {"level":"debug","ts":1716234857.6314173,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/12442343974","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.7.6 CertMagic acmez (linux; amd64)"]},"response_headers":{"Boulder-Requester":["148725774"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["822"],"Content-Type":["application/json"],"Date":["Mon, 20 May 2024 19:54:17 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["jZ8PWoQvWS1NJiifAEM-FHjsaEzlRGLKnobS4WDwWDfMfLKWqpA"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":200}

May 20 19:54:17 mercury caddy[615]: {"level":"error","ts":1716234857.6316683,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"smithbury.com","issuer":"acme-v02.api.letsencrypt.org-directory","error":"[smithbury.com] solving challenges: presenting for challenge: adding temporary record for zone \"smithbury.com.\": got error status: HTTP 400: [{Code:6003 Message:Invalid request headers ErrorChain:[{Code:6111 Message:Invalid format for Authorization header}]}] (order=https://acme-staging-v02.api.letsencrypt.org/acme/order/148725774/16656323274) (ca=https://acme-staging-v02.api.letsencrypt.org/directory)"}

May 20 19:54:17 mercury caddy[615]: {"level":"debug","ts":1716234857.6317358,"logger":"tls.obtain","msg":"trying issuer 2/2","issuer":"acme.zerossl.com-v2-DV90"}

May 20 19:54:17 mercury caddy[615]: {"level":"debug","ts":1716234857.9206333,"logger":"tls.issuance.zerossl.acme_client","msg":"http request","method":"HEAD","url":"https://acme.zerossl.com/v2/DV90/newNonce","headers":{"User-Agent":["Caddy/2.7.6 CertMagic acmez (linux; amd64)"]},"response_headers":{"Access-Control-Allow-Origin":["*"],"Cache-Control":["max-age=0, no-cache, no-store"],"Content-Type":["application/octet-stream"],"Date":["Mon, 20 May 2024 19:54:17 GMT"],"Link":["<https://acme.zerossl.com/v2/DV90>;rel=\"index\""],"Replay-Nonce":["KVIlvycwThiQBAygIaILc14w_eIaFkxitn6UDKTkUcM"],"Server":["nginx"],"Strict-Transport-Security":["max-age=15724800; includeSubDomains"]},"status_code":200}

May 20 19:54:18 mercury caddy[615]: {"level":"debug","ts":1716234858.2623143,"logger":"tls.issuance.zerossl.acme_client","msg":"http request","method":"POST","url":"https://acme.zerossl.com/v2/DV90/newOrder","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.7.6 CertMagic acmez (linux; amd64)"]},"response_headers":{"Access-Control-Allow-Origin":["*"],"Cache-Control":["max-age=0, no-cache, no-store"],"Content-Length":["275"],"Content-Type":["application/json"],"Date":["Mon, 20 May 2024 19:54:18 GMT"],"Location":["https://acme.zerossl.com/v2/DV90/order/i-atXUXUiKE8T3a9CJRkgg"],"Replay-Nonce":["Fz8O4JUmRBIeQYEb9Q-Y5rSqi1PYT3yqwtu191CPUvA"],"Server":["nginx"],"Strict-Transport-Security":["max-age=15724800; includeSubDomains"]},"status_code":201}

May 20 19:54:18 mercury caddy[615]: {"level":"debug","ts":1716234858.5751386,"logger":"tls.issuance.zerossl.acme_client","msg":"http request","method":"POST","url":"https://acme.zerossl.com/v2/DV90/authz/j-_cXppHoLEudZZQ_2a_jQ","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.7.6 CertMagic acmez (linux; amd64)"]},"response_headers":{"Access-Control-Allow-Origin":["*"],"Cache-Control":["max-age=0, no-cache, no-store"],"Content-Length":["443"],"Content-Type":["application/json"],"Date":["Mon, 20 May 2024 19:54:18 GMT"],"Link":["<https://acme.zerossl.com/v2/DV90>;rel=\"index\""],"Replay-Nonce":["BTa20YCA1EROpr9VRXomXDEtJxJmuGqWuuPQxPzgMMg"],"Retry-After":["5"],"Server":["nginx"],"Strict-Transport-Security":["max-age=15724800; includeSubDomains"]},"status_code":200}

May 20 19:54:18 mercury caddy[615]: {"level":"debug","ts":1716234858.575469,"logger":"tls.issuance.zerossl.acme_client","msg":"no solver configured","challenge_type":"http-01"}

May 20 19:54:18 mercury caddy[615]: {"level":"info","ts":1716234858.5755231,"logger":"tls.issuance.zerossl.acme_client","msg":"trying to solve challenge","identifier":"smithbury.com","challenge_type":"dns-01","ca":"https://acme.zerossl.com/v2/DV90"}

May 20 19:54:18 mercury caddy[615]: {"level":"error","ts":1716234858.8242736,"logger":"tls.issuance.zerossl.acme_client","msg":"cleaning up solver","identifier":"smithbury.com","challenge_type":"dns-01","error":"no memory of presenting a DNS record for \"_acme-challenge.smithbury.com\" (usually OK if presenting also failed)"}

May 20 19:54:19 mercury caddy[615]: {"level":"debug","ts":1716234859.1348228,"logger":"tls.issuance.zerossl.acme_client","msg":"http request","method":"POST","url":"https://acme.zerossl.com/v2/DV90/authz/j-_cXppHoLEudZZQ_2a_jQ","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.7.6 CertMagic acmez (linux; amd64)"]},"response_headers":{"Access-Control-Allow-Origin":["*"],"Cache-Control":["max-age=0, no-cache, no-store"],"Content-Length":["125"],"Content-Type":["application/json"],"Date":["Mon, 20 May 2024 19:54:19 GMT"],"Link":["<https://acme.zerossl.com/v2/DV90>;rel=\"index\""],"Replay-Nonce":["-8GLe3Nr65nYiIb2hadfBSIiN1owDk89epk4akThj8A"],"Server":["nginx"],"Strict-Transport-Security":["max-age=15724800; includeSubDomains"]},"status_code":200}

May 20 19:54:19 mercury caddy[615]: {"level":"error","ts":1716234859.1352487,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"smithbury.com","issuer":"acme.zerossl.com-v2-DV90","error":"[smithbury.com] solving challenges: presenting for challenge: adding temporary record for zone \"smithbury.com.\": got error status: HTTP 400: [{Code:6003 Message:Invalid request headers ErrorChain:[{Code:6111 Message:Invalid format for Authorization header}]}] (order=https://acme.zerossl.com/v2/DV90/order/i-atXUXUiKE8T3a9CJRkgg) (ca=https://acme.zerossl.com/v2/DV90)"}

May 20 19:54:19 mercury caddy[615]: {"level":"debug","ts":1716234859.1353288,"logger":"events","msg":"event","name":"cert_failed","id":"162d8bc7-e121-4ebc-a6d4-46fa10724141","origin":"tls","data":{"error":{},"identifier":"smithbury.com","issuers":["acme-v02.api.letsencrypt.org-directory","acme.zerossl.com-v2-DV90"],"renewal":false}}

May 20 19:54:19 mercury caddy[615]: {"level":"error","ts":1716234859.1353724,"logger":"tls.obtain","msg":"will retry","error":"[smithbury.com] Obtain: [smithbury.com] solving challenges: presenting for challenge: adding temporary record for zone \"smithbury.com.\": got error status: HTTP 400: [{Code:6003 Message:Invalid request headers ErrorChain:[{Code:6111 Message:Invalid format for Authorization header}]}] (order=https://acme.zerossl.com/v2/DV90/order/i-atXUXUiKE8T3a9CJRkgg) (ca=https://acme.zerossl.com/v2/DV90)","attempt":2,"retrying_in":120,"elapsed":67.740466404,"max_duration":2592000}

May 20 19:56:19 mercury caddy[615]: {"level":"info","ts":1716234979.135567,"logger":"tls.obtain","msg":"obtaining certificate","identifier":"smithbury.com"}

May 20 19:56:19 mercury caddy[615]: {"level":"debug","ts":1716234979.1359112,"logger":"events","msg":"event","name":"cert_obtaining","id":"5f9a95a9-79cb-406d-bed3-8c01f76a5ebc","origin":"tls","data":{"identifier":"smithbury.com"}}

May 20 19:56:19 mercury caddy[615]: {"level":"debug","ts":1716234979.136072,"logger":"tls.obtain","msg":"trying issuer 1/2","issuer":"acme-v02.api.letsencrypt.org-directory"}

May 20 19:56:19 mercury caddy[615]: {"level":"debug","ts":1716234979.3795156,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"HEAD","url":"https://acme-staging-v02.api.letsencrypt.org/acme/new-nonce","headers":{"User-Agent":["Caddy/2.7.6 CertMagic acmez (linux; amd64)"]},"response_headers":{"Cache-Control":["public, max-age=0, no-cache"],"Date":["Mon, 20 May 2024 19:56:19 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["Ppg7MeefrpnjDcqPi5JkHXd7_F7Zk4y7EbDmBEg5yYhGmoiQd2U"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":200}

May 20 19:56:19 mercury caddy[615]: {"level":"debug","ts":1716234979.663701,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-staging-v02.api.letsencrypt.org/acme/new-order","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.7.6 CertMagic acmez (linux; amd64)"]},"response_headers":{"Boulder-Requester":["148725774"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["352"],"Content-Type":["application/json"],"Date":["Mon, 20 May 2024 19:56:19 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Location":["https://acme-staging-v02.api.letsencrypt.org/acme/order/148725774/16656347534"],"Replay-Nonce":["Ppg7Meef-F02eJyljTkSgiFKG-U0ePbbCV8h3WBji5OpXNT4s6Q"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":201}

May 20 19:56:19 mercury caddy[615]: {"level":"debug","ts":1716234979.9162457,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/12442355084","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.7.6 CertMagic acmez (linux; amd64)"]},"response_headers":{"Boulder-Requester":["148725774"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["818"],"Content-Type":["application/json"],"Date":["Mon, 20 May 2024 19:56:19 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["Ppg7MeefEKACRCJLElwM7XCaEQbzeTkoYy4szi6MQHcHRkI4Ji8"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":200}

May 20 19:56:19 mercury caddy[615]: {"level":"debug","ts":1716234979.9165928,"logger":"tls.issuance.acme.acme_client","msg":"no solver configured","challenge_type":"http-01"}

May 20 19:56:19 mercury caddy[615]: {"level":"debug","ts":1716234979.9166682,"logger":"tls.issuance.acme.acme_client","msg":"no solver configured","challenge_type":"tls-alpn-01"}

May 20 19:56:19 mercury caddy[615]: {"level":"info","ts":1716234979.9167085,"logger":"tls.issuance.acme.acme_client","msg":"trying to solve challenge","identifier":"smithbury.com","challenge_type":"dns-01","ca":"https://acme-staging-v02.api.letsencrypt.org/directory"}

May 20 19:56:20 mercury caddy[615]: {"level":"error","ts":1716234980.1786926,"logger":"tls.issuance.acme.acme_client","msg":"cleaning up solver","identifier":"smithbury.com","challenge_type":"dns-01","error":"no memory of presenting a DNS record for \"_acme-challenge.smithbury.com\" (usually OK if presenting also failed)"}

May 20 19:56:20 mercury caddy[615]: {"level":"debug","ts":1716234980.4334671,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/12442355084","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.7.6 CertMagic acmez (linux; amd64)"]},"response_headers":{"Boulder-Requester":["148725774"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["822"],"Content-Type":["application/json"],"Date":["Mon, 20 May 2024 19:56:20 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["Ppg7MeefJfib1Kmt0Yh_EpVosPbTBZGW9_K_jHuKvCYHGwXfnfg"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":200}

May 20 19:56:20 mercury caddy[615]: {"level":"error","ts":1716234980.4338956,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"smithbury.com","issuer":"acme-v02.api.letsencrypt.org-directory","error":"[smithbury.com] solving challenges: presenting for challenge: adding temporary record for zone \"smithbury.com.\": got error status: HTTP 400: [{Code:6003 Message:Invalid request headers ErrorChain:[{Code:6111 Message:Invalid format for Authorization header}]}] (order=https://acme-staging-v02.api.letsencrypt.org/acme/order/148725774/16656347534) (ca=https://acme-staging-v02.api.letsencrypt.org/directory)"}

May 20 19:56:20 mercury caddy[615]: {"level":"debug","ts":1716234980.4340413,"logger":"tls.obtain","msg":"trying issuer 2/2","issuer":"acme.zerossl.com-v2-DV90"}

May 20 19:56:21 mercury caddy[615]: {"level":"debug","ts":1716234981.558344,"logger":"tls.issuance.zerossl.acme_client","msg":"http request","method":"HEAD","url":"https://acme.zerossl.com/v2/DV90/newNonce","headers":{"User-Agent":["Caddy/2.7.6 CertMagic acmez (linux; amd64)"]},"response_headers":{"Access-Control-Allow-Origin":["*"],"Cache-Control":["max-age=0, no-cache, no-store"],"Content-Type":["application/octet-stream"],"Date":["Mon, 20 May 2024 19:56:21 GMT"],"Link":["<https://acme.zerossl.com/v2/DV90>;rel=\"index\""],"Replay-Nonce":["SRBn-FmUZsiIx12qvsYDE2QdhVWY1jZaauyQ1wY5yFQ"],"Server":["nginx"],"Strict-Transport-Security":["max-age=15724800; includeSubDomains"]},"status_code":200}

May 20 19:56:21 mercury caddy[615]: {"level":"debug","ts":1716234981.8996916,"logger":"tls.issuance.zerossl.acme_client","msg":"http request","method":"POST","url":"https://acme.zerossl.com/v2/DV90/newOrder","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.7.6 CertMagic acmez (linux; amd64)"]},"response_headers":{"Access-Control-Allow-Origin":["*"],"Cache-Control":["max-age=0, no-cache, no-store"],"Content-Length":["275"],"Content-Type":["application/json"],"Date":["Mon, 20 May 2024 19:56:21 GMT"],"Location":["https://acme.zerossl.com/v2/DV90/order/0O3Emfqupk1v-9VKR31CHQ"],"Replay-Nonce":["AHv00OaDYRwAJ1CJwu2v8dKYChzdWj13suewBTrFh0U"],"Server":["nginx"],"Strict-Transport-Security":["max-age=15724800; includeSubDomains"]},"status_code":201}

May 20 19:56:22 mercury caddy[615]: {"level":"debug","ts":1716234982.209557,"logger":"tls.issuance.zerossl.acme_client","msg":"http request","method":"POST","url":"https://acme.zerossl.com/v2/DV90/authz/iqWJkPFdCOrcfH0BTzkWXQ","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.7.6 CertMagic acmez (linux; amd64)"]},"response_headers":{"Access-Control-Allow-Origin":["*"],"Cache-Control":["max-age=0, no-cache, no-store"],"Content-Length":["443"],"Content-Type":["application/json"],"Date":["Mon, 20 May 2024 19:56:22 GMT"],"Link":["<https://acme.zerossl.com/v2/DV90>;rel=\"index\""],"Replay-Nonce":["BiP2HSQWx5G84waJyFK44fleC2krU3fveU3-cTI2e4o"],"Retry-After":["5"],"Server":["nginx"],"Strict-Transport-Security":["max-age=15724800; includeSubDomains"]},"status_code":200}

May 20 19:56:22 mercury caddy[615]: {"level":"debug","ts":1716234982.2098854,"logger":"tls.issuance.zerossl.acme_client","msg":"no solver configured","challenge_type":"http-01"}

May 20 19:56:22 mercury caddy[615]: {"level":"info","ts":1716234982.2099695,"logger":"tls.issuance.zerossl.acme_client","msg":"trying to solve challenge","identifier":"smithbury.com","challenge_type":"dns-01","ca":"https://acme.zerossl.com/v2/DV90"}

May 20 19:56:22 mercury caddy[615]: {"level":"error","ts":1716234982.4492028,"logger":"tls.issuance.zerossl.acme_client","msg":"cleaning up solver","identifier":"smithbury.com","challenge_type":"dns-01","error":"no memory of presenting a DNS record for \"_acme-challenge.smithbury.com\" (usually OK if presenting also failed)"}

May 20 19:56:22 mercury caddy[615]: {"level":"debug","ts":1716234982.7878785,"logger":"tls.issuance.zerossl.acme_client","msg":"http request","method":"POST","url":"https://acme.zerossl.com/v2/DV90/authz/iqWJkPFdCOrcfH0BTzkWXQ","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.7.6 CertMagic acmez (linux; amd64)"]},"response_headers":{"Access-Control-Allow-Origin":["*"],"Cache-Control":["max-age=0, no-cache, no-store"],"Content-Length":["125"],"Content-Type":["application/json"],"Date":["Mon, 20 May 2024 19:56:22 GMT"],"Link":["<https://acme.zerossl.com/v2/DV90>;rel=\"index\""],"Replay-Nonce":["pgDrRIS_1Iu8MQQhX-e0RxEDiXKWYXNu4wmk7ipg2bU"],"Server":["nginx"],"Strict-Transport-Security":["max-age=15724800; includeSubDomains"]},"status_code":200}

May 20 19:56:22 mercury caddy[615]: {"level":"error","ts":1716234982.788414,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"smithbury.com","issuer":"acme.zerossl.com-v2-DV90","error":"[smithbury.com] solving challenges: presenting for challenge: adding temporary record for zone \"smithbury.com.\": got error status: HTTP 400: [{Code:6003 Message:Invalid request headers ErrorChain:[{Code:6111 Message:Invalid format for Authorization header}]}] (order=https://acme.zerossl.com/v2/DV90/order/0O3Emfqupk1v-9VKR31CHQ) (ca=https://acme.zerossl.com/v2/DV90)"}

May 20 19:56:22 mercury caddy[615]: {"level":"debug","ts":1716234982.7885568,"logger":"events","msg":"event","name":"cert_failed","id":"0bf7e715-0e22-402d-bcd6-a604db1217df","origin":"tls","data":{"error":{},"identifier":"smithbury.com","issuers":["acme-v02.api.letsencrypt.org-directory","acme.zerossl.com-v2-DV90"],"renewal":false}}

May 20 19:56:22 mercury caddy[615]: {"level":"error","ts":1716234982.7886343,"logger":"tls.obtain","msg":"will retry","error":"[smithbury.com] Obtain: [smithbury.com] solving challenges: presenting for challenge: adding temporary record for zone \"smithbury.com.\": got error status: HTTP 400: [{Code:6003 Message:Invalid request headers ErrorChain:[{Code:6111 Message:Invalid format for Authorization header}]}] (order=https://acme.zerossl.com/v2/DV90/order/0O3Emfqupk1v-9VKR31CHQ) (ca=https://acme.zerossl.com/v2/DV90)","attempt":3,"retrying_in":120,"elapsed":191.393728306,"max_duration":2592000}

Curl ouput

curl -vL https://smithbury.com
*   Trying 203.59.204.167:443...
* connect to 203.59.204.167 port 443 failed: No route to host
* Failed to connect to smithbury.com port 443 after 3016 ms: No route to host
* Closing connection 0
curl: (7) Failed to connect to smithbury.com port 443 after 3016 ms: No route to host

@jeff,

It appears that the is a firewall preventing access to Port 80 & 443;
these are what I see with nmap

For smithbury.com`

$ nmap -Pn -p80,443 smithbury.com
Starting Nmap 7.80 ( https://nmap.org ) at 2024-05-20 20:06 UTC
Nmap scan report for smithbury.com (203.59.204.167)
Host is up.
rDNS record for 203.59.204.167: 203-59-204-167.tpgi.com.au

PORT    STATE    SERVICE
80/tcp  filtered http
443/tcp filtered https

Nmap done: 1 IP address (1 host up) scanned in 3.69 seconds

For www.smithbury.com

$ nmap -Pn -p80,443 www.smithbury.com
Starting Nmap 7.80 ( https://nmap.org ) at 2024-05-20 20:06 UTC
Nmap scan report for www.smithbury.com (203.59.204.167)
Host is up.
rDNS record for 203.59.204.167: 203-59-204-167.tpgi.com.au

PORT    STATE    SERVICE
80/tcp  filtered http
443/tcp filtered https

Nmap done: 1 IP address (1 host up) scanned in 3.17 seconds

This means your Cloudflare API key is invalid. You used [ ] in what you wrote. Did you use any kind of syntax elements when putting your API key in your config? It should just be exactly what Cloudflare gave you (or something like {env.CLOUDFLARE_TOKEN} to have Caddy read it from environment variables).

2 Likes

I updated my port forwarding and reserved the internal IP for the VM. Sorry about that. Can you test again.

No brackets. Just plain like this (previous token value)

acme_dns cloudflare 27e8bb82e249392f968fd1a8280a40c6

To confirm it is the value retrieved from here

Hi @jeff,

Still see this, sorry. :frowning:

$ nmap -Pn -p80,443 smithbury.com
Starting Nmap 7.80 ( https://nmap.org ) at 2024-05-20 23:06 UTC
Nmap scan report for smithbury.com (203.59.204.167)
Host is up.
rDNS record for 203.59.204.167: 203-59-204-167.tpgi.com.au

PORT    STATE    SERVICE
80/tcp  filtered http
443/tcp filtered https

Nmap done: 1 IP address (1 host up) scanned in 3.22 seconds
$ nmap -Pn -p80,443 www.smithbury.com
Starting Nmap 7.80 ( https://nmap.org ) at 2024-05-20 23:06 UTC
Nmap scan report for www.smithbury.com (203.59.204.167)
Host is up.
rDNS record for 203.59.204.167: 203-59-204-167.tpgi.com.au

PORT    STATE    SERVICE
80/tcp  filtered http
443/tcp filtered https

Nmap done: 1 IP address (1 host up) scanned in 3.09 seconds

Thanks for testing again. I’ll be unable to change anything for next 8ish hours due to work. Will check everything at my end as soon as I get home. Thanks again for helping.

1 Like

@jeff here is an online tool, you can check the HTTP and HTTPS connectivity remotely from around the world, https://check-host.net/

1 Like

Hey again. Trying to fix/clarify this issue, I figured I’ll temp expose my API token and generate a new one once the issue is diagnosed.

So going into Cloudflare dash, select my domain then scroll down and select Get your API token. Then scroll down to Global API > View and copy.

[REDACTED]

Here is my current Caddyfile

{
	debug
	acme_dns cloudflare [REDACTED]
}

https://smithbury.com {
	# Homarr host and port
	reverse_proxy https://192.168.1.118:7575
}

With that all shared, can you see where I am messing up on the API token entry?

Troubleshooting the firewall issue I’ve gathered the following. I’ll keep searching. If this highlights an issue for anyone, please let me know.

Router with forwarded ports

Checking firewall on VM

ss -ltn
State         Recv-Q        Send-Q               Local Address:Port                Peer Address:Port       Process       
LISTEN        0             4096                     127.0.0.1:2019                     0.0.0.0:*                        
LISTEN        0             1024                     127.0.0.1:37855                    0.0.0.0:*                        
LISTEN        0             128                        0.0.0.0:22                       0.0.0.0:*                        
LISTEN        0             4096                 127.0.0.53%lo:53                       0.0.0.0:*                        
LISTEN        0             4096                             *:443                            *:*                        
LISTEN        0             4096                             *:80                             *:*                        
LISTEN        0             128                           [::]:22                          [::]:*            
sudo nmap localhost
Starting Nmap 7.80 ( https://nmap.org ) at 2024-05-21 09:19 UTC
Nmap scan report for localhost (127.0.0.1)
Host is up (0.0000010s latency).
Not shown: 997 closed ports
PORT    STATE SERVICE
22/tcp  open  ssh
80/tcp  open  http
443/tcp open  https

Nmap done: 1 IP address (1 host up) scanned in 0.02 seconds
sudo ufw status verbose
Status: inactive

The instructions are here:

2 Likes

Thank you! There is comprehensive documentation but there is also so much and it’s focused on subsets of features. I now get a blank https page and just need to figure out why the redirect part isn’t working. Thank you.

Let’s see your logs.

2 Likes

Thank you.

May 22 09:47:00 mercury systemd[1]: Reloaded Caddy.
May 22 09:47:00 mercury caddy[615]: {"level":"info","ts":1716371220.9012132,"logger":"admin","msg":"stopped previous server","address":"localhost:2019"}
May 22 09:47:00 mercury caddy[615]: {"level":"info","ts":1716371220.9029632,"logger":"tls","msg":"cleaning storage unit","storage":"FileStorage:/var/lib/caddy/.local/share/caddy"}
May 22 09:47:00 mercury caddy[615]: {"level":"info","ts":1716371220.9034016,"logger":"tls","msg":"finished cleaning storage units"}
May 22 09:49:39 mercury caddy[615]: {"level":"debug","ts":1716371379.1385624,"logger":"http.handlers.reverse_proxy","msg":"selected upstream","dial":"192.168.1.110:7878","total_upstreams":1}
May 22 09:49:39 mercury caddy[615]: {"level":"debug","ts":1716371379.1423857,"logger":"http.handlers.reverse_proxy","msg":"upstream roundtrip","upstream":"192.168.1.110:7878","duration":0.003416064,"request":{"remote_ip":"203.59.204.167","remote_port":"65107","client_ip":"203.59.204.167","proto":"HTTP/3.0","method":"GET","host":"smithbury.com","uri":"/","headers":{"X-Forwarded-Host":["smithbury.com"],"Accept-Encoding":["gzip, deflate, br, zstd"],"Accept-Language":["en-US,en;q=0.5"],"Sec-Fetch-Mode":["navigate"],"Dnt":["1"],"Alt-Used":["smithbury.com"],"Upgrade-Insecure-Requests":["1"],"Sec-Fetch-User":["?1"],"Sec-Fetch-Site":["none"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:126.0) Gecko/20100101 Firefox/126.0"],"Sec-Fetch-Dest":["document"],"X-Forwarded-For":["203.59.204.167"],"X-Forwarded-Proto":["https"],"Priority":["u=1"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8"]},"tls":{"resumed":true,"version":772,"cipher_suite":4867,"proto":"h3","server_name":"smithbury.com"}},"headers":{"Date":["Wed, 22 May 2024 09:49:38 GMT"],"Server":["Kestrel"],"Location":["https://smithbury.com/login?returnUrl=%2F"],"Content-Length":["0"]},"status":302}
May 22 09:49:52 mercury caddy[615]: {"level":"debug","ts":1716371392.5952702,"logger":"events","msg":"event","name":"tls_get_certificate","id":"d51633df-81b2-456a-b232-87b19f37e82e","origin":"tls","data":{"client_hello":{"CipherSuites":[4865,4867,4866,49195,49199,52393,52392,49196,49200,49162,49161,49171,49172,156,157,47,53],"ServerName":"plex.smithbury.com","SupportedCurves":[29,23,24,25,256,257],"SupportedPoints":"AA==","SignatureSchemes":[1027,1283,1539,2052,2053,2054,1025,1281,1537,515,513],"SupportedProtos":["h2","http/1.1"],"SupportedVersions":[772,771],"RemoteAddr":{"IP":"203.59.204.167","Port":2788,"Zone":""},"LocalAddr":{"IP":"192.168.1.121","Port":443,"Zone":""}}}}
May 22 09:49:52 mercury caddy[615]: {"level":"debug","ts":1716371392.595573,"logger":"tls.handshake","msg":"choosing certificate","identifier":"plex.smithbury.com","num_choices":1}
May 22 09:49:52 mercury caddy[615]: {"level":"debug","ts":1716371392.595584,"logger":"tls.handshake","msg":"default certificate selection results","identifier":"plex.smithbury.com","subjects":["plex.smithbury.com"],"managed":true,"issuer_key":"acme-v02.api.letsencrypt.org-directory","hash":"34f47a4fcd609140272ad6bff0a03435d825beedcbb421a51d1585d72e56130b"}
May 22 09:49:52 mercury caddy[615]: {"level":"debug","ts":1716371392.5955875,"logger":"tls.handshake","msg":"matched certificate in cache","remote_ip":"203.59.204.167","remote_port":"2788","subjects":["plex.smithbury.com"],"managed":true,"expiration":1724058901,"hash":"34f47a4fcd609140272ad6bff0a03435d825beedcbb421a51d1585d72e56130b"}
May 22 09:49:52 mercury caddy[615]: {"level":"debug","ts":1716371392.599303,"logger":"http.handlers.reverse_proxy","msg":"selected upstream","dial":"192.168.1.117:32400","total_upstreams":1}
May 22 09:49:52 mercury caddy[615]: {"level":"debug","ts":1716371392.5997515,"logger":"http.handlers.reverse_proxy","msg":"upstream roundtrip","upstream":"192.168.1.117:32400","duration":0.000434441,"request":{"remote_ip":"203.59.204.167","remote_port":"2788","client_ip":"203.59.204.167","proto":"HTTP/2.0","method":"GET","host":"plex.smithbury.com","uri":"/","headers":{"Sec-Fetch-Site":["none"],"Upgrade-Insecure-Requests":["1"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8"],"Te":["trailers"],"X-Forwarded-Host":["plex.smithbury.com"],"Priority":["u=1"],"Accept-Encoding":["gzip, deflate, br, zstd"],"Sec-Fetch-Dest":["document"],"X-Forwarded-Proto":["https"],"Sec-Fetch-User":["?1"],"Dnt":["1"],"Sec-Fetch-Mode":["navigate"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:126.0) Gecko/20100101 Firefox/126.0"],"Accept-Language":["en-US,en;q=0.5"],"X-Forwarded-For":["203.59.204.167"]},"tls":{"resumed":false,"version":772,"cipher_suite":4867,"proto":"h2","server_name":"plex.smithbury.com"}},"headers":{"Content-Encoding":["gzip"],"X-Plex-Content-Original-Length":["193"],"Content-Length":["157"],"X-Plex-Protocol":["1.0"],"Content-Type":["text/html"],"X-Plex-Content-Compressed-Length":["157"],"Cache-Control":["no-cache"],"Date":["Wed, 22 May 2024 09:49:52 GMT"]},"status":401}
May 22 09:49:52 mercury caddy[615]: {"level":"debug","ts":1716371392.6028194,"logger":"events","msg":"event","name":"tls_get_certificate","id":"9f2c86a7-49dc-423d-b9fa-c3b8d0fb6431","origin":"tls","data":{"client_hello":{"CipherSuites":[4865,4867,4866],"ServerName":"plex.smithbury.com","SupportedCurves":[29,23,24,25],"SupportedPoints":null,"SignatureSchemes":[1027,1283,1539,515,2052,2053,2054,1025,1281,1537,513],"SupportedProtos":["h3"],"SupportedVersions":[772],"RemoteAddr":{"IP":"203.59.204.167","Port":50179,"Zone":""},"LocalAddr":{"IP":"192.168.1.121","Port":443,"Zone":""}}}}
May 22 09:49:52 mercury caddy[615]: {"level":"debug","ts":1716371392.6028469,"logger":"tls.handshake","msg":"choosing certificate","identifier":"plex.smithbury.com","num_choices":1}
May 22 09:49:52 mercury caddy[615]: {"level":"debug","ts":1716371392.6028566,"logger":"tls.handshake","msg":"default certificate selection results","identifier":"plex.smithbury.com","subjects":["plex.smithbury.com"],"managed":true,"issuer_key":"acme-v02.api.letsencrypt.org-directory","hash":"34f47a4fcd609140272ad6bff0a03435d825beedcbb421a51d1585d72e56130b"}
May 22 09:49:52 mercury caddy[615]: {"level":"debug","ts":1716371392.6028605,"logger":"tls.handshake","msg":"matched certificate in cache","remote_ip":"203.59.204.167","remote_port":"50179","subjects":["plex.smithbury.com"],"managed":true,"expiration":1724058901,"hash":"34f47a4fcd609140272ad6bff0a03435d825beedcbb421a51d1585d72e56130b"}
May 22 09:49:59 mercury caddy[615]: {"level":"debug","ts":1716371399.3576162,"logger":"events","msg":"event","name":"tls_get_certificate","id":"c8f49b27-a98f-4f4d-be59-a23cd944f262","origin":"tls","data":{"client_hello":{"CipherSuites":[4865,4867,4866,49195,49199,52393,52392,49196,49200,49162,49161,49171,49172,156,157,47,53],"ServerName":"radarr.smithbury.com","SupportedCurves":[29,23,24,25,256,257],"SupportedPoints":"AA==","SignatureSchemes":[1027,1283,1539,2052,2053,2054,1025,1281,1537,515,513],"SupportedProtos":["h2","http/1.1"],"SupportedVersions":[772,771],"RemoteAddr":{"IP":"203.59.204.167","Port":2789,"Zone":""},"LocalAddr":{"IP":"192.168.1.121","Port":443,"Zone":""}}}}
May 22 09:49:59 mercury caddy[615]: {"level":"debug","ts":1716371399.3581312,"logger":"tls.handshake","msg":"choosing certificate","identifier":"radarr.smithbury.com","num_choices":1}
May 22 09:49:59 mercury caddy[615]: {"level":"debug","ts":1716371399.3582246,"logger":"tls.handshake","msg":"default certificate selection results","identifier":"radarr.smithbury.com","subjects":["radarr.smithbury.com"],"managed":true,"issuer_key":"local","hash":"65e06bc8a4446403bcea79c1e8d69b4b107d7a44c26eb87e6122c104580c59a9"}
May 22 09:49:59 mercury caddy[615]: {"level":"debug","ts":1716371399.3583035,"logger":"tls.handshake","msg":"matched certificate in cache","remote_ip":"203.59.204.167","remote_port":"2789","subjects":["radarr.smithbury.com"],"managed":true,"expiration":1716331855,"hash":"65e06bc8a4446403bcea79c1e8d69b4b107d7a44c26eb87e6122c104580c59a9"}
May 22 09:49:59 mercury caddy[615]: {"level":"debug","ts":1716371399.362469,"logger":"http.stdlib","msg":"http: TLS handshake error from 203.59.204.167:2789: remote error: tls: unknown certificate authority"}
May 22 09:50:03 mercury caddy[615]: {"level":"debug","ts":1716371403.3752131,"logger":"events","msg":"event","name":"tls_get_certificate","id":"50c35087-febd-48bc-a9d5-07e572091cda","origin":"tls","data":{"client_hello":{"CipherSuites":[4865,4867,4866,49195,49199,52393,52392,49196,49200,49162,49161,49171,49172,156,157,47,53],"ServerName":"radarr.smithbury.com","SupportedCurves":[29,23,24,25,256,257],"SupportedPoints":"AA==","SignatureSchemes":[1027,1283,1539,2052,2053,2054,1025,1281,1537,515,513],"SupportedProtos":["h2","http/1.1"],"SupportedVersions":[772,771],"RemoteAddr":{"IP":"203.59.204.167","Port":2794,"Zone":""},"LocalAddr":{"IP":"192.168.1.121","Port":443,"Zone":""}}}}
May 22 09:50:03 mercury caddy[615]: {"level":"debug","ts":1716371403.3756926,"logger":"tls.handshake","msg":"choosing certificate","identifier":"radarr.smithbury.com","num_choices":1}
May 22 09:50:03 mercury caddy[615]: {"level":"debug","ts":1716371403.3757937,"logger":"tls.handshake","msg":"default certificate selection results","identifier":"radarr.smithbury.com","subjects":["radarr.smithbury.com"],"managed":true,"issuer_key":"local","hash":"65e06bc8a4446403bcea79c1e8d69b4b107d7a44c26eb87e6122c104580c59a9"}
May 22 09:50:03 mercury caddy[615]: {"level":"debug","ts":1716371403.3758745,"logger":"tls.handshake","msg":"matched certificate in cache","remote_ip":"203.59.204.167","remote_port":"2794","subjects":["radarr.smithbury.com"],"managed":true,"expiration":1716331855,"hash":"65e06bc8a4446403bcea79c1e8d69b4b107d7a44c26eb87e6122c104580c59a9"}
May 22 09:50:03 mercury caddy[615]: {"level":"debug","ts":1716371403.3809605,"logger":"http.handlers.reverse_proxy","msg":"selected upstream","dial":"radarr/:80","total_upstreams":1}
May 22 09:50:03 mercury caddy[615]: {"level":"debug","ts":1716371403.381313,"logger":"http.handlers.reverse_proxy","msg":"upstream roundtrip","upstream":"192.168.1.110:7878/radarr/:80","duration":0.000111619,"request":{"remote_ip":"203.59.204.167","remote_port":"2794","client_ip":"203.59.204.167","proto":"HTTP/2.0","method":"GET","host":"radarr.smithbury.com","uri":"/","headers":{"Priority":["u=1"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:126.0) Gecko/20100101 Firefox/126.0"],"Sec-Fetch-Mode":["navigate"],"X-Forwarded-For":["203.59.204.167"],"Accept-Language":["en-US,en;q=0.5"],"Sec-Fetch-Site":["none"],"Sec-Fetch-User":["?1"],"Te":["trailers"],"Accept-Encoding":["gzip, deflate, br, zstd"],"X-Forwarded-Proto":["https"],"X-Forwarded-Host":["radarr.smithbury.com"],"Sec-Fetch-Dest":["document"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8"],"Upgrade-Insecure-Requests":["1"],"Dnt":["1"]},"tls":{"resumed":false,"version":772,"cipher_suite":4867,"proto":"h2","server_name":"radarr.smithbury.com"}},"error":"dial tcp: lookup radarr/: no such host"}
May 22 09:50:03 mercury caddy[615]: {"level":"error","ts":1716371403.3814723,"logger":"http.log.error","msg":"dial tcp: lookup radarr/: no such host","request":{"remote_ip":"203.59.204.167","remote_port":"2794","client_ip":"203.59.204.167","proto":"HTTP/2.0","method":"GET","host":"radarr.smithbury.com","uri":"/","headers":{"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8"],"Accept-Language":["en-US,en;q=0.5"],"Upgrade-Insecure-Requests":["1"],"Sec-Fetch-Dest":["document"],"Priority":["u=1"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:126.0) Gecko/20100101 Firefox/126.0"],"Accept-Encoding":["gzip, deflate, br, zstd"],"Dnt":["1"],"Sec-Fetch-Mode":["navigate"],"Sec-Fetch-Site":["none"],"Sec-Fetch-User":["?1"],"Te":["trailers"]},"tls":{"resumed":false,"version":772,"cipher_suite":4867,"proto":"h2","server_name":"radarr.smithbury.com"}},"duration":0.000547578,"status":502,"err_id":"0eeatah2m","err_trace":"reverseproxy.statusError (reverseproxy.go:1267)"}
May 22 09:50:03 mercury caddy[615]: {"level":"debug","ts":1716371403.4041154,"logger":"events","msg":"event","name":"tls_get_certificate","id":"25a323a7-9a66-4b40-8805-335b53aa8e5f","origin":"tls","data":{"client_hello":{"CipherSuites":[4865,4867,4866],"ServerName":"radarr.smithbury.com","SupportedCurves":[29,23,24,25],"SupportedPoints":null,"SignatureSchemes":[1027,1283,1539,515,2052,2053,2054,1025,1281,1537,513],"SupportedProtos":["h3"],"SupportedVersions":[772],"RemoteAddr":{"IP":"203.59.204.167","Port":62651,"Zone":""},"LocalAddr":{"IP":"192.168.1.121","Port":443,"Zone":""}}}}
May 22 09:50:03 mercury caddy[615]: {"level":"debug","ts":1716371403.404398,"logger":"tls.handshake","msg":"choosing certificate","identifier":"radarr.smithbury.com","num_choices":1}
May 22 09:50:03 mercury caddy[615]: {"level":"debug","ts":1716371403.404493,"logger":"tls.handshake","msg":"default certificate selection results","identifier":"radarr.smithbury.com","subjects":["radarr.smithbury.com"],"managed":true,"issuer_key":"local","hash":"65e06bc8a4446403bcea79c1e8d69b4b107d7a44c26eb87e6122c104580c59a9"}
May 22 09:50:03 mercury caddy[615]: {"level":"debug","ts":1716371403.404586,"logger":"tls.handshake","msg":"matched certificate in cache","remote_ip":"203.59.204.167","remote_port":"62651","subjects":["radarr.smithbury.com"],"managed":true,"expiration":1716331855,"hash":"65e06bc8a4446403bcea79c1e8d69b4b107d7a44c26eb87e6122c104580c59a9"}

Caddyfile

{
	debug
	acme_dns cloudflare [TOKEN]
}

smithbury.com/ {
	# Homarr host and port
	reverse_proxy 192.168.1.110:7878
}

plex.smithbury.com/ {
	reverse_proxy 192.168.1.117:32400
	# URL should be 192.168.1.117:32400/web/index.html#!/
}

radarr.smithbury.com/ {
	reverse_proxy 192.168.1.110:7878/radarr/
}

This is your problem. Remove the / here. This is setting up a path matcher which only matches exactly / and nothing else. (Same for each of your sites)

If you look at your logs before this line, you should see warnings about that in your config, path matchers in the site address is deprecated and will be removed (turned into an error) at some point.

Similarly here, you can’t have a path in an upstream address. Remove /radarr/ here. If you need to add a path prefix (you probably don’t need to), use a rewrite directive to do that.

2 Likes

You sir, are a scholar and a gentleman. I can now use smithbury.com on my network. Thank you very much :slight_smile:

Oddly though I found the domain and direct WAN IP address are inaccessible from off my network. E.g. using mobile network on phone. Any ideas what I’ve overlooked?