Caddy-docker-proxy: Connection Refused: "type":"urn:ietf:params:acme:error:connection"

the-bort-the · November 14, 2024, 7:47pm

1. The problem I’m having:

I’m trying to complete the setup and configuration of caddy-docker-proxy I found here. I have used the exact same docker-compose.yml and Dockerfile.

I have used my own domain in place of course. The caddy container is running, updating my A record within Cloudflare, but ultimately shows a few errors as seen in the title of the post with more presented below.

I have used their Caddyfile as mine as well. Lastly I have implemented the adguard container as shown in the same github repo.

Basically I want to use my domain and caddy-docker-proxy to get a certificate for any other applications that are started using a docker-compose.yml (adguard, pihole, jellyfin, etc.)

2. Error messages and/or full log output:

debug mode was added to get this

{"level":"info","ts":1731612998.6600852,"logger":"docker-proxy","msg":"Running caddy proxy server"}
{"level":"info","ts":1731612998.660854,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
{"level":"info","ts":1731612998.6610358,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
{"level":"info","ts":1731612998.6610794,"logger":"docker-proxy","msg":"Running caddy proxy controller"}
{"level":"info","ts":1731612998.6616914,"logger":"docker-proxy","msg":"Start","CaddyfilePath":"/etc/caddy/Caddyfile","EnvFile":"","LabelPrefix":"caddy","PollingInterval":30,"ProxyServiceTasks":true,"ProcessCaddyfile":true,"ScanStoppedContainers":false,"IngressNetworks":"[caddy_network]","DockerSockets":[""],"DockerCertsPath":[""],"DockerAPIsVersion":[""]}
{"level":"info","ts":1731612998.6625135,"logger":"docker-proxy","msg":"Connecting to docker events","DockerSocket":""}
{"level":"info","ts":1731612998.6628373,"logger":"docker-proxy","msg":"IngressNetworksMap","ingres":"map[9a3cf9acc92eaf24a584234ca35e0f68e22ea5686ad4ace6b94b83dda81b172b:true caddy_network:true]"}
{"level":"info","ts":1731612998.6697774,"logger":"docker-proxy","msg":"Swarm is available","new":false}
{"level":"info","ts":1731612998.6795292,"logger":"docker-proxy","msg":"New Caddyfile","caddyfile":"{\n\tadmin localhost:2019\n\tdebug\n\temail my@email.com\n\tdynamic_dns {\n\t\tprovider cloudflare {$CLOUDFLARE_API_TOKEN}\n\t\tdomains {\n\t\t\titsacomputer.com *\n\t\t}\n\t\tcheck_interval 1m\n\t\tversions ipv4\n\t}\n}\nadguard.itsacomputer.com {\n\troute {\n\t\treverse_proxy 192.168.112.3:80\n\t}\n}\n"}
{"level":"info","ts":1731612998.6801991,"logger":"docker-proxy","msg":"New Config JSON","json":"{\"admin\":{\"listen\":\"localhost:2019\"},\"logging\":{\"logs\":{\"default\":{\"level\":\"DEBUG\"}}},\"apps\":{\"dynamic_dns\":{\"dns_provider\":{\"api_token\":\"/run/secrets/CLOUDFLARE_API_TOKEN\",\"name\":\"cloudflare\"},\"domains\":{\"itsacomputer.com\":[\"*\"]},\"versions\":{\"ipv4\":true,\"ipv6\":false},\"check_interval\":60000000000},\"http\":{\"servers\":{\"srv0\":{\"listen\":[\":443\"],\"routes\":[{\"match\":[{\"host\":[\"adguard.itsacomputer.com\"]}],\"handle\":[{\"handler\":\"subroute\",\"routes\":[{\"handle\":[{\"handler\":\"subroute\",\"routes\":[{\"handle\":[{\"handler\":\"reverse_proxy\",\"upstreams\":[{\"dial\":\"192.168.112.3:80\"}]}]}]}]}]}],\"terminal\":true}]}}},\"tls\":{\"automation\":{\"policies\":[{\"subjects\":[\"adguard.itsacomputer.com\"],\"issuers\":[{\"email\":\"my@email.com\",\"module\":\"acme\"},{\"ca\":\"https://acme.zerossl.com/v2/DV90\",\"email\":\"my@email.com\",\"module\":\"acme\"}]}]}}}}"}
{"level":"info","ts":1731612998.680274,"logger":"docker-proxy","msg":"Sending configuration to","server":"localhost"}
{"level":"info","ts":1731612998.681048,"logger":"admin.api","msg":"received request","method":"POST","host":"localhost:2019","uri":"/load","remote_ip":"127.0.0.1","remote_port":"41090","headers":{"Accept-Encoding":["gzip"],"Content-Length":["830"],"Content-Type":["application/json"],"User-Agent":["Go-http-client/1.1"]}}
{"level":"info","ts":1731612998.6819322,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
{"level":"info","ts":1731612998.6821437,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc00073ef80"}
{"level":"info","ts":1731612998.6823535,"logger":"http.auto_https","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443}
{"level":"info","ts":1731612998.6823757,"logger":"http.auto_https","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
{"level":"debug","ts":1731612998.6824129,"logger":"http.auto_https","msg":"adjusted config","tls":{"automation":{"policies":[{"subjects":["adguard.itsacomputer.com"]},{}]}},"http":{"servers":{"remaining_auto_https_redirects":{"listen":[":80"],"routes":[{},{}]},"srv0":{"listen":[":443"],"routes":[{"handle":[{"handler":"subroute","routes":[{"handle":[{"handler":"subroute","routes":[{"handle":[{"handler":"reverse_proxy","upstreams":[{"dial":"192.168.112.3:80"}]}]}]}]}]}],"terminal":true}],"tls_connection_policies":[{}],"automatic_https":{}}}}}
{"level":"info","ts":1731612998.6828141,"logger":"http","msg":"enabling HTTP/3 listener","addr":":443"}
{"level":"info","ts":1731612998.6828966,"msg":"failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 7168 kiB, got: 416 kiB). See https://github.com/quic-go/quic-go/wiki/UDP-Buffer-Sizes for details."}
{"level":"debug","ts":1731612998.6829934,"logger":"http","msg":"starting server loop","address":"[::]:443","tls":true,"http3":true}
{"level":"info","ts":1731612998.683009,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2","h3"]}
{"level":"debug","ts":1731612998.6830597,"logger":"http","msg":"starting server loop","address":"[::]:80","tls":false,"http3":false}
{"level":"info","ts":1731612998.6830668,"logger":"http.log","msg":"server running","name":"remaining_auto_https_redirects","protocols":["h1","h2","h3"]}
{"level":"info","ts":1731612998.6830711,"logger":"http","msg":"enabling automatic TLS certificate management","domains":["adguard.itsacomputer.com"]}
{"level":"info","ts":1731612998.6833167,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
{"level":"info","ts":1731612998.6833448,"logger":"admin.api","msg":"load complete"}
{"level":"debug","ts":1731612998.6835122,"logger":"dynamic_dns","msg":"beginning IP address check"}
{"level":"info","ts":1731612998.683674,"logger":"tls.obtain","msg":"acquiring lock","identifier":"adguard.itsacomputer.com"}
{"level":"info","ts":1731612998.6844203,"logger":"admin","msg":"stopped previous server","address":"localhost:2019"}
{"level":"info","ts":1731612998.6846125,"logger":"docker-proxy","msg":"Successfully configured","server":"localhost"}
{"level":"info","ts":1731612998.685395,"logger":"tls","msg":"storage cleaning happened too recently; skipping for now","storage":"FileStorage:/data/caddy","instance":"4cabd726-13fb-4d9b-af70-d5ec01dbdbec","try_again":1731699398.6853921,"try_again_in":86399.999999477}
{"level":"info","ts":1731612998.6854987,"logger":"tls","msg":"finished cleaning storage units"}
{"level":"info","ts":1731612998.6863532,"logger":"tls.obtain","msg":"lock acquired","identifier":"adguard.itsacomputer.com"}
{"level":"info","ts":1731612998.6864974,"logger":"tls.obtain","msg":"obtaining certificate","identifier":"adguard.itsacomputer.com"}
{"level":"debug","ts":1731612998.686545,"logger":"events","msg":"event","name":"cert_obtaining","id":"bf569b85-6c72-49e9-bc83-383521f4e593","origin":"tls","data":{"identifier":"adguard.itsacomputer.com"}}
{"level":"debug","ts":1731612998.6867695,"logger":"tls.obtain","msg":"trying issuer 1/2","issuer":"acme-v02.api.letsencrypt.org-directory"}
{"level":"info","ts":1731612998.686985,"logger":"tls.issuance.acme","msg":"waiting on internal rate limiter","identifiers":["adguard.itsacomputer.com"],"ca":"https://acme-v02.api.letsencrypt.org/directory","account":"my@email.com"}
{"level":"info","ts":1731612998.6869924,"logger":"tls.issuance.acme","msg":"done waiting on internal rate limiter","identifiers":["adguard.itsacomputer.com"],"ca":"https://acme-v02.api.letsencrypt.org/directory","account":"my@email.com"}
{"level":"info","ts":1731612998.687002,"logger":"tls.issuance.acme","msg":"using ACME account","account_id":"https://acme-v02.api.letsencrypt.org/acme/acct/2056631987","account_contact":["mailto:my@email.com"]}
{"level":"debug","ts":1731612998.8089836,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"GET","url":"https://acme-v02.api.letsencrypt.org/directory","headers":{"User-Agent":["Caddy/2.8.4 CertMagic acmez (linux; amd64)"]},"response_headers":{"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["746"],"Content-Type":["application/json"],"Date":["Thu, 14 Nov 2024 19:36:38 GMT"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":200}
{"level":"debug","ts":1731612998.8092709,"logger":"tls.issuance.acme.acme_client","msg":"creating order","account":"https://acme-v02.api.letsencrypt.org/acme/acct/2056631987","identifiers":["adguard.itsacomputer.com"]}
{"level":"debug","ts":1731612998.8461518,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"HEAD","url":"https://acme-v02.api.letsencrypt.org/acme/new-nonce","headers":{"User-Agent":["Caddy/2.8.4 CertMagic acmez (linux; amd64)"]},"response_headers":{"Cache-Control":["public, max-age=0, no-cache"],"Date":["Thu, 14 Nov 2024 19:36:38 GMT"],"Link":["<https://acme-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["usf-jMgXVlQpJ8pN76XWUcYiHcQuwTVD8nLhenYXCn1DLBhP7Ik"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":200}
{"level":"error","ts":1731612998.8827329,"logger":"dynamic_dns","msg":"unable to lookup current IPs from DNS records","error":"got error status: HTTP 400: [{Code:6003 Message:Invalid request headers ErrorChain:[{Code:6111 Message:Invalid format for Authorization header}]}]"}
{"level":"debug","ts":1731612998.8827896,"logger":"dynamic_dns","msg":"looked up current IPs from DNS","lastIPs":null}
{"level":"debug","ts":1731612998.954999,"logger":"dynamic_dns.ip_sources.simple_http","msg":"lookup","type":"IPv4","endpoint":"https://icanhazip.com","ip":"REDACTED PUBLIC IP"}
{"level":"info","ts":1731612998.9550593,"logger":"dynamic_dns","msg":"updating DNS record","zone":"itsacomputer.com","type":"A","name":"*","value":"REDACTED PUBLIC IP","ttl":0}
{"level":"debug","ts":1731612999.0329607,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-v02.api.letsencrypt.org/acme/new-order","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.8.4 CertMagic acmez (linux; amd64)"]},"response_headers":{"Boulder-Requester":["2056631987"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["350"],"Content-Type":["application/json"],"Date":["Thu, 14 Nov 2024 19:36:39 GMT"],"Link":["<https://acme-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Location":["https://acme-v02.api.letsencrypt.org/acme/order/2056631987/323125871607"],"Replay-Nonce":["usf-jMgX8zzUtRUERsakdrEa_TU_rzN7nI6VQewA5QtoTLpq6-o"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":201}
{"level":"debug","ts":1731612999.0812092,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-v02.api.letsencrypt.org/acme/authz-v3/430160425037","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.8.4 CertMagic acmez (linux; amd64)"]},"response_headers":{"Boulder-Requester":["2056631987"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["808"],"Content-Type":["application/json"],"Date":["Thu, 14 Nov 2024 19:36:39 GMT"],"Link":["<https://acme-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["JNlwFRqrftYzoObxlwdhKmCzhoeeTfLi-9xCcj1W8T7EgmJ-PY8"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":200}
{"level":"info","ts":1731612999.0817454,"logger":"tls.issuance.acme.acme_client","msg":"trying to solve challenge","identifier":"adguard.itsacomputer.com","challenge_type":"tls-alpn-01","ca":"https://acme-v02.api.letsencrypt.org/directory"}
{"level":"error","ts":1731612999.0828407,"logger":"dynamic_dns","msg":"failed setting DNS record(s) with new IP address(es)","zone":"itsacomputer.com","error":"got error status: HTTP 400: [{Code:6003 Message:Invalid request headers ErrorChain:[{Code:6111 Message:Invalid format for Authorization header}]}]"}
{"level":"info","ts":1731612999.0828986,"logger":"dynamic_dns","msg":"finished updating DNS","current_ips":["REDACTED PUBLIC IP"]}
{"level":"debug","ts":1731612999.0842018,"logger":"tls.issuance.acme.acme_client","msg":"waiting for solver before continuing","identifier":"adguard.itsacomputer.com","challenge_type":"tls-alpn-01"}
{"level":"debug","ts":1731612999.0842378,"logger":"tls.issuance.acme.acme_client","msg":"done waiting for solver","identifier":"adguard.itsacomputer.com","challenge_type":"tls-alpn-01"}
{"level":"debug","ts":1731612999.0843594,"logger":"http.stdlib","msg":"http: TLS handshake error from 127.0.0.1:35260: EOF"}
{"level":"debug","ts":1731612999.134357,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-v02.api.letsencrypt.org/acme/chall-v3/430160425037/pz2RBw","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.8.4 CertMagic acmez (linux; amd64)"]},"response_headers":{"Boulder-Requester":["2056631987"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["191"],"Content-Type":["application/json"],"Date":["Thu, 14 Nov 2024 19:36:39 GMT"],"Link":["<https://acme-v02.api.letsencrypt.org/directory>;rel=\"index\"","<https://acme-v02.api.letsencrypt.org/acme/authz-v3/430160425037>;rel=\"up\""],"Location":["https://acme-v02.api.letsencrypt.org/acme/chall-v3/430160425037/pz2RBw"],"Replay-Nonce":["usf-jMgXay4ID_744W79VnO1kifRcQ41jQj2fu9u9Wcuga7JE28"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":200}
{"level":"debug","ts":1731612999.134531,"logger":"tls.issuance.acme.acme_client","msg":"challenge accepted","identifier":"adguard.itsacomputer.com","challenge_type":"tls-alpn-01"}
{"level":"debug","ts":1731612999.4309814,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-v02.api.letsencrypt.org/acme/authz-v3/430160425037","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.8.4 CertMagic acmez (linux; amd64)"]},"response_headers":{"Boulder-Requester":["2056631987"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["833"],"Content-Type":["application/json"],"Date":["Thu, 14 Nov 2024 19:36:39 GMT"],"Link":["<https://acme-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["JNlwFRqrKyYmgBr1wmWliojvgYiBKkZgoZqmO8LOiHAJv2aJRmg"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":200}
{"level":"error","ts":1731612999.4315207,"logger":"tls.issuance.acme.acme_client","msg":"challenge failed","identifier":"adguard.itsacomputer.com","challenge_type":"tls-alpn-01","problem":{"type":"urn:ietf:params:acme:error:connection","title":"","detail":"REDACTED PUBLIC IP: Connection refused","instance":"","subproblems":[]}}
{"level":"error","ts":1731612999.431581,"logger":"tls.issuance.acme.acme_client","msg":"validating authorization","identifier":"adguard.itsacomputer.com","problem":{"type":"urn:ietf:params:acme:error:connection","title":"","detail":"REDACTED PUBLIC IP: Connection refused","instance":"","subproblems":[]},"order":"https://acme-v02.api.letsencrypt.org/acme/order/2056631987/323125871607","attempt":1,"max_attempts":3}
{"level":"debug","ts":1731613000.431826,"logger":"tls.issuance.acme.acme_client","msg":"creating order","account":"https://acme-v02.api.letsencrypt.org/acme/acct/2056631987","identifiers":["adguard.itsacomputer.com"]}
{"level":"debug","ts":1731613000.671462,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-v02.api.letsencrypt.org/acme/new-order","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.8.4 CertMagic acmez (linux; amd64)"]},"response_headers":{"Boulder-Requester":["2056631987"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["350"],"Content-Type":["application/json"],"Date":["Thu, 14 Nov 2024 19:36:40 GMT"],"Link":["<https://acme-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Location":["https://acme-v02.api.letsencrypt.org/acme/order/2056631987/323125876577"],"Replay-Nonce":["JNlwFRqr3gqh6zywQ_TUVN6wC19kVrLkK7KKOtkz3SiCsyUtZ-8"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":201}
{"level":"debug","ts":1731613000.7143564,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-v02.api.letsencrypt.org/acme/authz-v3/430160432677","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.8.4 CertMagic acmez (linux; amd64)"]},"response_headers":{"Boulder-Requester":["2056631987"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["808"],"Content-Type":["application/json"],"Date":["Thu, 14 Nov 2024 19:36:40 GMT"],"Link":["<https://acme-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["usf-jMgX8vPhVic-TO9ZIAtHPln7_g4_G24mAAdOSCSnXjbeta0"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":200}
{"level":"info","ts":1731613000.714614,"logger":"tls.issuance.acme.acme_client","msg":"trying to solve challenge","identifier":"adguard.itsacomputer.com","challenge_type":"http-01","ca":"https://acme-v02.api.letsencrypt.org/directory"}
{"level":"debug","ts":1731613000.715501,"logger":"tls.issuance.acme.acme_client","msg":"waiting for solver before continuing","identifier":"adguard.itsacomputer.com","challenge_type":"http-01"}
{"level":"debug","ts":1731613000.7155783,"logger":"tls.issuance.acme.acme_client","msg":"done waiting for solver","identifier":"adguard.itsacomputer.com","challenge_type":"http-01"}
{"level":"debug","ts":1731613000.766722,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-v02.api.letsencrypt.org/acme/chall-v3/430160432677/GwRkzw","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.8.4 CertMagic acmez (linux; amd64)"]},"response_headers":{"Boulder-Requester":["2056631987"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["187"],"Content-Type":["application/json"],"Date":["Thu, 14 Nov 2024 19:36:40 GMT"],"Link":["<https://acme-v02.api.letsencrypt.org/directory>;rel=\"index\"","<https://acme-v02.api.letsencrypt.org/acme/authz-v3/430160432677>;rel=\"up\""],"Location":["https://acme-v02.api.letsencrypt.org/acme/chall-v3/430160432677/GwRkzw"],"Replay-Nonce":["usf-jMgXASSxOC907jUvn0cJ1z5lNYkulMUjZAQmwue3vnVGBgc"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":200}
{"level":"debug","ts":1731613000.7670054,"logger":"tls.issuance.acme.acme_client","msg":"challenge accepted","identifier":"adguard.itsacomputer.com","challenge_type":"http-01"}
{"level":"debug","ts":1731613001.072433,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-v02.api.letsencrypt.org/acme/authz-v3/430160432677","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.8.4 CertMagic acmez (linux; amd64)"]},"response_headers":{"Boulder-Requester":["2056631987"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["1064"],"Content-Type":["application/json"],"Date":["Thu, 14 Nov 2024 19:36:41 GMT"],"Link":["<https://acme-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["usf-jMgXeejbjBKTEzIeb2sln4gBdqOi1pCtCjV1W3vishnUFt4"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":200}
{"level":"error","ts":1731613001.0729275,"logger":"tls.issuance.acme.acme_client","msg":"challenge failed","identifier":"adguard.itsacomputer.com","challenge_type":"http-01","problem":{"type":"urn:ietf:params:acme:error:connection","title":"","detail":"REDACTED PUBLIC IP: Fetching http://adguard.itsacomputer.com/.well-known/acme-challenge/wWj13Yl89YUWhJkKn_qyQiahgTXiSM5ukmfJuzLR7JQ: Connection refused","instance":"","subproblems":[]}}
{"level":"error","ts":1731613001.0729866,"logger":"tls.issuance.acme.acme_client","msg":"validating authorization","identifier":"adguard.itsacomputer.com","problem":{"type":"urn:ietf:params:acme:error:connection","title":"","detail":"REDACTED PUBLIC IP: Fetching http://adguard.itsacomputer.com/.well-known/acme-challenge/wWj13Yl89YUWhJkKn_qyQiahgTXiSM5ukmfJuzLR7JQ: Connection refused","instance":"","subproblems":[]},"order":"https://acme-v02.api.letsencrypt.org/acme/order/2056631987/323125876577","attempt":2,"max_attempts":3}
{"level":"error","ts":1731613001.0730493,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"adguard.itsacomputer.com","issuer":"acme-v02.api.letsencrypt.org-directory","error":"HTTP 400 urn:ietf:params:acme:error:connection - REDACTED PUBLIC IP: Fetching http://adguard.itsacomputer.com/.well-known/acme-challenge/wWj13Yl89YUWhJkKn_qyQiahgTXiSM5ukmfJuzLR7JQ: Connection refused"}
{"level":"debug","ts":1731613001.073088,"logger":"tls.obtain","msg":"trying issuer 2/2","issuer":"acme.zerossl.com-v2-DV90"}
{"level":"info","ts":1731613001.0737467,"logger":"tls.issuance.acme","msg":"waiting on internal rate limiter","identifiers":["adguard.itsacomputer.com"],"ca":"https://acme.zerossl.com/v2/DV90","account":"my@email.com"}
{"level":"info","ts":1731613001.0738647,"logger":"tls.issuance.acme","msg":"done waiting on internal rate limiter","identifiers":["adguard.itsacomputer.com"],"ca":"https://acme.zerossl.com/v2/DV90","account":"my@email.com"}
{"level":"info","ts":1731613001.0739105,"logger":"tls.issuance.acme","msg":"using ACME account","account_id":"https://acme.zerossl.com/v2/DV90/account/15MuYn1W-kXvPtieW3JGxQ","account_contact":["mailto:my@email.com"]}
{"level":"debug","ts":1731613001.2852602,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"GET","url":"https://acme.zerossl.com/v2/DV90","headers":{"User-Agent":["Caddy/2.8.4 CertMagic acmez (linux; amd64)"]},"response_headers":{"Access-Control-Allow-Origin":["*"],"Content-Length":["678"],"Content-Type":["application/json"],"Date":["Thu, 14 Nov 2024 19:36:41 GMT"],"Server":["nginx"],"Strict-Transport-Security":["max-age=15724800; includeSubDomains"]},"status_code":200}
{"level":"debug","ts":1731613001.285486,"logger":"tls.issuance.acme.acme_client","msg":"creating order","account":"https://acme.zerossl.com/v2/DV90/account/15MuYn1W-kXvPtieW3JGxQ","identifiers":["adguard.itsacomputer.com"]}
{"level":"debug","ts":1731613001.9733987,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"HEAD","url":"https://acme.zerossl.com/v2/DV90/newNonce","headers":{"User-Agent":["Caddy/2.8.4 CertMagic acmez (linux; amd64)"]},"response_headers":{"Access-Control-Allow-Origin":["*"],"Cache-Control":["max-age=0, no-cache, no-store"],"Content-Type":["application/octet-stream"],"Date":["Thu, 14 Nov 2024 19:36:41 GMT"],"Link":["<https://acme.zerossl.com/v2/DV90>;rel=\"index\""],"Replay-Nonce":["D9JXByJeuqh2vve3vN0oD8K4FvA2e6sG6E4HXQI0vy4"],"Server":["nginx"],"Strict-Transport-Security":["max-age=15724800; includeSubDomains"]},"status_code":200}
{"level":"debug","ts":1731613002.4633021,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme.zerossl.com/v2/DV90/newOrder","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.8.4 CertMagic acmez (linux; amd64)"]},"response_headers":{"Access-Control-Allow-Origin":["*"],"Cache-Control":["max-age=0, no-cache, no-store"],"Content-Length":["286"],"Content-Type":["application/json"],"Date":["Thu, 14 Nov 2024 19:36:42 GMT"],"Location":["https://acme.zerossl.com/v2/DV90/order/lK4D-iPb3x0iHWrdEsUJIg"],"Replay-Nonce":["-N1xrICCwSaMWNbSvvw4YB-7JOU7gT4SOKJv-hovpFE"],"Server":["nginx"],"Strict-Transport-Security":["max-age=15724800; includeSubDomains"]},"status_code":201}
{"level":"debug","ts":1731613002.7626367,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme.zerossl.com/v2/DV90/authz/G0BlNI4wa3oDAjrRkXewbg","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.8.4 CertMagic acmez (linux; amd64)"]},"response_headers":{"Access-Control-Allow-Origin":["*"],"Cache-Control":["max-age=0, no-cache, no-store"],"Content-Length":["457"],"Content-Type":["application/json"],"Date":["Thu, 14 Nov 2024 19:36:42 GMT"],"Link":["<https://acme.zerossl.com/v2/DV90>;rel=\"index\""],"Replay-Nonce":["N78CaGZHEa1MvK7aSvwXfGgh8gxRSY8rGYqwiCCTK6Y"],"Retry-After":["86400"],"Server":["nginx"],"Strict-Transport-Security":["max-age=15724800; includeSubDomains"]},"status_code":200}
{"level":"debug","ts":1731613002.7628794,"logger":"tls.issuance.acme.acme_client","msg":"no solver configured","challenge_type":"dns-01"}
{"level":"info","ts":1731613002.76299,"logger":"tls.issuance.acme.acme_client","msg":"trying to solve challenge","identifier":"adguard.itsacomputer.com","challenge_type":"http-01","ca":"https://acme.zerossl.com/v2/DV90"}
{"level":"debug","ts":1731613002.7640533,"logger":"tls.issuance.acme.acme_client","msg":"waiting for solver before continuing","identifier":"adguard.itsacomputer.com","challenge_type":"http-01"}
{"level":"debug","ts":1731613002.764194,"logger":"tls.issuance.acme.acme_client","msg":"done waiting for solver","identifier":"adguard.itsacomputer.com","challenge_type":"http-01"}
{"level":"debug","ts":1731613002.9077835,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme.zerossl.com/v2/DV90/chall/wt1IZw9__acFiruBgDgdOQ","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.8.4 CertMagic acmez (linux; amd64)"]},"response_headers":{"Access-Control-Allow-Origin":["*"],"Cache-Control":["max-age=0, no-cache, no-store"],"Content-Length":["164"],"Content-Type":["application/json"],"Date":["Thu, 14 Nov 2024 19:36:42 GMT"],"Link":["<https://acme.zerossl.com/v2/DV90/authz/G0BlNI4wa3oDAjrRkXewbg>;rel=\"up\""],"Replay-Nonce":["xm73Vu_-sXYDEMxXRz36D3cIsJuf_kX8pGGKEKp1dG0"],"Retry-After":["60"],"Server":["nginx"],"Strict-Transport-Security":["max-age=15724800; includeSubDomains"]},"status_code":200}
{"level":"debug","ts":1731613002.9079459,"logger":"tls.issuance.acme.acme_client","msg":"challenge accepted","identifier":"adguard.itsacomputer.com","challenge_type":"http-01"}
{"level":"debug","ts":1731613003.4398038,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme.zerossl.com/v2/DV90/authz/G0BlNI4wa3oDAjrRkXewbg","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.8.4 CertMagic acmez (linux; amd64)"]},"response_headers":{"Access-Control-Allow-Origin":["*"],"Cache-Control":["max-age=0, no-cache, no-store"],"Content-Length":["457"],"Content-Type":["application/json"],"Date":["Thu, 14 Nov 2024 19:36:43 GMT"],"Link":["<https://acme.zerossl.com/v2/DV90>;rel=\"index\""],"Replay-Nonce":["OovUmEjFarT_oz94QKVHuM87-bfCuWfa_EQTODHRlUI"],"Retry-After":["86400"],"Server":["nginx"],"Strict-Transport-Security":["max-age=15724800; includeSubDomains"]},"status_code":200}
{"level":"info","ts":1731613008.720393,"logger":"admin.api","msg":"received request","method":"GET","host":"127.0.0.1:2019","uri":"/config","remote_ip":"127.0.0.1","remote_port":"51352","headers":{"Accept":["*/*"],"User-Agent":["curl/8.9.1"]}}

3. Caddy version:

v2.8.4 h1:q3pe0wpBj1OcHFZ3n/1nl4V4bxBrYoSoab7rL9BMYNk=

4. How I installed and ran Caddy:

Using the Dockerfile I built the docker image and then used docker compose up -d to start the container

a. System environment:

Debian x86, Docker, docker compose

b. Command:

docker build -t caddy:cloudflare .
docker compose up -d

c. Service/unit/compose file:

secrets:
  CLOUDFLARE_API_TOKEN:
    file: ./secrets/CLOUDFLARE_API_TOKEN.secret

services:
  caddy:
    build:
      context: ./
      dockerfile: Dockerfile
    image: caddy:cloudflare
    container_name: caddy
    ports:
      - "80:80"
      - "443:443"
      - "443:443/udp"
      - "2019:2019"
    secrets:
      - CLOUDFLARE_API_TOKEN
    environment:
      - CADDY_INGRESS_NETWORKS=caddy_network
      - CLOUDFLARE_API_TOKEN=/run/secrets/CLOUDFLARE_API_TOKEN
      - CADDY_DOCKER_CADDYFILE_PATH=/etc/caddy/Caddyfile
    networks:
      - caddy_network
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - caddy_data:/data
      - caddy_config:/config
      - ./Caddyfile:/etc/caddy/
    restart: unless-stopped
    dns:
     - 1.1.1.1
     - 1.0.0.1

networks:
  caddy_network:
    name: caddy_network
    external: true

volumes:
  caddy_data:
    name: caddy_data
  caddy_config:
    name: caddy_config

d. My complete Caddy config:

{
  debug
}
{
	admin localhost:2019
	email my@email.com
	dynamic_dns {
		provider cloudflare {$CLOUDFLARE_API_TOKEN}
		domains {
			itsacomputer.com *
		}
		check_interval 1m
        versions ipv4
	}
}

5. Links to relevant resources:

Github example I followed

Also including Dockerfile used

ARG GOLANG_VERSION=1.22.1
ARG ALPINE_VERSION=3.19

FROM golang:${GOLANG_VERSION}-alpine${ALPINE_VERSION} as gobuild
ARG GOLANG_VERSION
ARG ALPINE_VERSION

WORKDIR /go/src/github.com/caddyserver/xcaddy/cmd/xcaddy

RUN apk add --no-cache git gcc build-base; \
    go install github.com/caddyserver/xcaddy/cmd/xcaddy@latest

RUN xcaddy build \
    --output /go/src/github.com/caddyserver/xcaddy/cmd/caddy \
    --with github.com/lucaslorentz/caddy-docker-proxy/v2 \
    --with github.com/mholt/caddy-dynamicdns \
    --with github.com/caddy-dns/cloudflare

FROM alpine:${ALPINE_VERSION}

RUN apk add --no-cache ca-certificates curl tzdata; \
    rm -rf /var/cache/apk/*;

EXPOSE 80 443 2019

ENV XDG_CONFIG_HOME /config
ENV XDG_DATA_HOME /data

COPY --from=gobuild /go/src/github.com/caddyserver/xcaddy/cmd/caddy /usr/bin/

HEALTHCHECK --interval=10s --timeout=5s --start-period=5s CMD curl -fsS http://127.0.0.1:2019/config -o /dev/null || exit 1

ENTRYPOINT ["/usr/bin/caddy"]
CMD ["docker-proxy"]

Also from within the container running caddy docker-proxy:

2024/11/14 19:52:37.984	INFO	docker-proxy	Running caddy proxy server
2024/11/14 19:52:37.984	INFO	admin	admin endpoint started	{"address": "localhost:2019", "enforce_origin": false, "origins": ["//localhost:2019", "//[::1]:2019", "//127.0.0.1:2019"]}
2024/11/14 19:52:37.985	INFO	autosaved config (load with --resume flag)	{"file": "/config/caddy/autosave.json"}
2024/11/14 19:52:37.985	INFO	docker-proxy	Running caddy proxy controller
2024/11/14 19:52:37.985	INFO	docker-proxy	Start	{"CaddyfilePath": "/etc/caddy/Caddyfile", "EnvFile": "", "LabelPrefix": "caddy", "PollingInterval": 30, "ProxyServiceTasks": true, "ProcessCaddyfile": true, "ScanStoppedContainers": false, "IngressNetworks": "[caddy_network]", "DockerSockets": [""], "DockerCertsPath": [""], "DockerAPIsVersion": [""]}
2024/11/14 19:52:37.986	INFO	docker-proxy	Connecting to docker events	{"DockerSocket": ""}
2024/11/14 19:52:37.987	INFO	docker-proxy	IngressNetworksMap	{"ingres": "map[9a3cf9acc92eaf24a584234ca35e0f68e22ea5686ad4ace6b94b83dda81b172b:true caddy_network:true]"}
2024/11/14 19:52:37.995	INFO	docker-proxy	Swarm is available	{"new": false}
2024/11/14 19:52:38.004	INFO	docker-proxy	New Caddyfile	{"caddyfile": "{\n\tadmin localhost:2019\n\tdebug\n\temail my@email.com\n\tdynamic_dns {\n\t\tprovider cloudflare {$CLOUDFLARE_API_TOKEN}\n\t\tdomains {\n\t\t\titsacomputer.com *\n\t\t}\n\t\tcheck_interval 1m\n\t\tversions ipv4\n\t}\n}\nadguard.itsacomputer.com {\n\troute {\n\t\treverse_proxy 192.168.112.3:80\n\t}\n}\n"}
2024/11/14 19:52:38.005	INFO	docker-proxy	New Config JSON	{"json": "{\"admin\":{\"listen\":\"localhost:2019\"},\"logging\":{\"logs\":{\"default\":{\"level\":\"DEBUG\"}}},\"apps\":{\"dynamic_dns\":{\"dns_provider\":{\"api_token\":\"/run/secrets/CLOUDFLARE_API_TOKEN\",\"name\":\"cloudflare\"},\"domains\":{\"itsacomputer.com\":[\"*\"]},\"versions\":{\"ipv4\":true,\"ipv6\":false},\"check_interval\":60000000000},\"http\":{\"servers\":{\"srv0\":{\"listen\":[\":443\"],\"routes\":[{\"match\":[{\"host\":[\"adguard.itsacomputer.com\"]}],\"handle\":[{\"handler\":\"subroute\",\"routes\":[{\"handle\":[{\"handler\":\"subroute\",\"routes\":[{\"handle\":[{\"handler\":\"reverse_proxy\",\"upstreams\":[{\"dial\":\"192.168.112.3:80\"}]}]}]}]}]}],\"terminal\":true}]}}},\"tls\":{\"automation\":{\"policies\":[{\"subjects\":[\"adguard.itsacomputer.com\"],\"issuers\":[{\"email\":\"my@email.com\",\"module\":\"acme\"},{\"ca\":\"https://acme.zerossl.com/v2/DV90\",\"email\":\"my@email.com\",\"module\":\"acme\"}]}]}}}}"}
2024/11/14 19:52:38.005	INFO	docker-proxy	Sending configuration to	{"server": "localhost"}
2024/11/14 19:52:38.006	INFO	docker-proxy	Successfully configured	{"server": "localhost"}
2024/11/14 19:52:38.255	INFO	admin.api	received request	{"method": "GET", "host": "127.0.0.1:2019", "uri": "/config", "remote_ip": "127.0.0.1", "remote_port": "42820", "headers": {"Accept":["*/*"],"User-Agent":["curl/8.9.1"]}}
2024/11/14 19:52:48.385	INFO	admin.api	received request	{"method": "GET", "host": "127.0.0.1:2019", "uri": "/config", "remote_ip": "127.0.0.1", "remote_port": "60884", "headers": {"Accept":["*/*"],"User-Agent":["curl/8.9.1"]}}

francislavoie · November 19, 2024, 6:17pm

You can only have one global options block. See Caddyfile Concepts — Caddy Documentation You have two there. Merge the debug with the other one.

You can greatly simplify it by using this one instead: GitHub - lucaslorentz/caddy-docker-proxy: Caddy as a reverse proxy for Docker

Are you sure that’s correct? Or is that just setting the env var to be a path to a file?

Caddy does have a {file.*} placeholder now which would let you read from a secrets file (but keep in mind that you need to make sure the file has no trailing newline for now, but that’s fixed in 2.9.0 where the newline is stripped out).

the-bort-the · November 20, 2024, 5:32am

Thanks for the feedback. I proceeded to add the acme_dns directive to my Caddyfile because this part of the docs said the server need not be externally available. I see certs have been downloaded successfully, but I’m still not able to land on the page. I suppose I’m still doing something incorrectly. I can see the certs have been pulled today and let’s debug shows success for DNS-01, but not the others.

updated Caddy file to include:

acme_dns cloudflare {secret}

logs

{"level":"info","ts":1732079749.6034534,"logger":"tls.issuance.acme.acme_client","msg":"successfully downloaded available certificate chains","count":2,"first_url":"https://acme-v02.api.letsencrypt.org/acme/cert/"}
{"level":"debug","ts":1732079749.6048412,"logger":"tls","msg":"loading managed certificate","domain":

francislavoie · November 20, 2024, 5:48am

What do you mean “but not the others”? Show evidence of the problem.

the-bort-the · November 20, 2024, 10:48am

I suspect I just can’t get away from not exposing the port. I was hopeful in that the DNS challenge portion of the docs stating a certificate can be obtained without exposing ports or the server was made possible with the recent change of implementing the DNS providers and satisfying the DNS challenge.

HTTP-01 returns with:

ANotWorking
Error
Domain.com has an A (IPv4) record (public ip) but a request to this address over port 80 did not succeed. Your web server must have at least one working IPv4 or IPv6 address.
Get "http://domain.com/.well-known/acme-challenge/letsdebug-test": dial tcp public ip:80: connect: connection refused

Trace:
@0ms: Making a request to http://domain.com/.well-known/acme-challenge/letsdebug-test (using initial IP )
@0ms: Dialing
@129ms: Experienced error: dial tcp :80: connect: connection refused
IssueFromLetsEncrypt
Error
A test authorization for domin.com to the Let's Encrypt staging service has revealed issues that may prevent any certificate for this domain being issued.
: Fetching http://domain.com/.well-known/acme-challenge/jYD6WZvF0VBgtXal2y4ffVl75CCZ_3EePTkvZkmeqi0: Connection refused

And then for tls-alpn-01:

IssueFromLetsEncrypt
Error
A test authorization for domain.com to the Let's Encrypt staging service has revealed issues that may prevent any certificate for this domain being issued.
Public IP: Connection refused

francislavoie · November 21, 2024, 8:35am

Yes you can. It’s probably just a config issue.

But you haven’t really shared your full config at any point yet. Please show your full config. CDP emits it in the /config directory of the container so you can pull it up.

the-bort-the · November 21, 2024, 2:01pm

Two files within /config/caddy:

/config/caddy # ls
Caddyfile.autosave  autosave.json
/config/caddy # cat Caddyfile.autosave
{
        debug
        admin localhost:2019
        email my@email.com
        acme_dns cloudflare REDACTED_SECRET
}
adguard.itsacomputer.com {
        route {
                reverse_proxy 192.168.160.3:80
        }
}
pihole.itsacomputer.com {
        reverse_proxy 192.168.160.4:80
}
/config/caddy # cat autosave.json

{"admin":{"listen":"tcp/localhost:2019"},"apps":{"http":{"servers":{"srv0":{"listen":[":443"],"routes":[{"handle":[{"handler":"subroute","routes":[{"handle":[{"handler":"subroute","routes":[{"handle":[{"handler":"reverse_proxy","upstreams":[{"dial":"192.168.160.3:80"}]}]}]}]}]}],"match":[{"host":["adguard.domain.com"]}],"terminal":true},{"handle":[{"handler":"subroute","routes":[{"handle":[{"handler":"reverse_proxy","upstreams":[{"dial":"192.168.160.4:80"}]}]}]}],"match":[{"host":["pihole.itsacomputer.com"]}],"terminal":true}]}}},"tls":{"automation":{"policies":[{"issuers":[{"challenges":{"dns":{"provider":{"api_token":"REDACTED_SECRET","name":"cloudflare"}}},"email":"my@email.com","module":"acme"},{"ca":"https://acme.zerossl.com/v2/DV90","challenges":{"dns":{"provider":{"api_token":"REDACTED_SECRET","name":"cloudflare"}}},"email":"my@email.com","module":"acme"}],"subjects":["adguard.itsacomputer.com","pihole.itsacomputer.com"]}]}}},"logging":{"logs":{"default":{"level":"DEBUG"}}}}

/config/caddy #

francislavoie · November 21, 2024, 4:34pm

Yeah that looks fine, Caddy should be using the ACME DNS challenge for both of those domains. Please show your full logs, it should mention where it stops doing it.

the-bort-the · November 21, 2024, 4:57pm

$ docker logs caddy 
{"level":"info","ts":1732207894.290905,"logger":"docker-proxy","msg":"Running caddy proxy server"}
{"level":"info","ts":1732207894.2921524,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
{"level":"info","ts":1732207894.2923968,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
{"level":"info","ts":1732207894.292437,"logger":"docker-proxy","msg":"Running caddy proxy controller"}
{"level":"info","ts":1732207894.2959232,"logger":"docker-proxy","msg":"Start","CaddyfilePath":"/etc/caddy/Caddyfile","EnvFile":"","LabelPrefix":"caddy","PollingInterval":30,"ProxyServiceTasks":true,"ProcessCaddyfile":true,"ScanStoppedContainers":false,"IngressNetworks":"[]","DockerSockets":[""],"DockerCertsPath":[""],"DockerAPIsVersion":[""]}
{"level":"info","ts":1732207894.2966497,"logger":"docker-proxy","msg":"Caddy ContainerID","ID":"403d6906d37066d6be2fe6ef3167b51c481ae0a162626ed731341f910891820f"}
{"level":"info","ts":1732207894.2967296,"logger":"docker-proxy","msg":"Connecting to docker events","DockerSocket":""}
{"level":"info","ts":1732207894.2987003,"logger":"docker-proxy","msg":"IngressNetworksMap","ingres":"map[0c7053beb9c97a06fb08f843096591d44bb7ae11844f770deef9b59e49e6e90c:true caddy_network:true]"}
{"level":"info","ts":1732207894.3053148,"logger":"docker-proxy","msg":"Swarm is available","new":false}
{"level":"info","ts":1732207894.3148084,"logger":"docker-proxy","msg":"New Caddyfile","caddyfile":"{\n\tdebug\n\tadmin localhost:2019\n\temail my@email.com\n\tacme_dns cloudflare REDACTED_TOKEN\n}\nadguard.itsacomputer.com {\n\troute {\n\t\treverse_proxy 192.168.160.3:80\n\t}\n}\npihole.itsacomputer.com {\n\treverse_proxy 192.168.160.4:80\n}\n"}
{"level":"info","ts":1732207894.3153863,"logger":"docker-proxy","msg":"New Config JSON","json":"{\"admin\":{\"listen\":\"localhost:2019\"},\"logging\":{\"logs\":{\"default\":{\"level\":\"DEBUG\"}}},\"apps\":{\"http\":{\"servers\":{\"srv0\":{\"listen\":[\":443\"],\"routes\":[{\"match\":[{\"host\":[\"adguard.itsacomputer.com\"]}],\"handle\":[{\"handler\":\"subroute\",\"routes\":[{\"handle\":[{\"handler\":\"subroute\",\"routes\":[{\"handle\":[{\"handler\":\"reverse_proxy\",\"upstreams\":[{\"dial\":\"192.168.160.3:80\"}]}]}]}]}]}],\"terminal\":true},{\"match\":[{\"host\":[\"pihole.itsacomputer.com\"]}],\"handle\":[{\"handler\":\"subroute\",\"routes\":[{\"handle\":[{\"handler\":\"reverse_proxy\",\"upstreams\":[{\"dial\":\"192.168.160.4:80\"}]}]}]}],\"terminal\":true}]}}},\"tls\":{\"automation\":{\"policies\":[{\"subjects\":[\"adguard.itsacomputer.com\",\"pihole.itsacomputer.com\"],\"issuers\":[{\"challenges\":{\"dns\":{\"provider\":{\"api_token\":\"REDACTED_TOKEN\",\"name\":\"cloudflare\"}}},\"email\":\"my@email.com\",\"module\":\"acme\"},{\"ca\":\"https://acme.zerossl.com/v2/DV90\",\"challenges\":{\"dns\":{\"provider\":{\"api_token\":\"REDACTED_TOKEN\",\"name\":\"cloudflare\"}}},\"email\":\"my@email.com\",\"module\":\"acme\"}]}]}}}}"}
{"level":"info","ts":1732207894.3154254,"logger":"docker-proxy","msg":"Sending configuration to","server":"localhost"}
{"level":"info","ts":1732207894.3159795,"logger":"admin.api","msg":"received request","method":"POST","host":"localhost:2019","uri":"/load","remote_ip":"127.0.0.1","remote_port":"46294","headers":{"Accept-Encoding":["gzip"],"Content-Length":["1066"],"Content-Type":["application/json"],"User-Agent":["Go-http-client/1.1"]}}
{"level":"info","ts":1732207894.316592,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
{"level":"info","ts":1732207894.3167992,"logger":"http.auto_https","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443}
{"level":"info","ts":1732207894.3168128,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc00082e180"}
{"level":"info","ts":1732207894.316816,"logger":"http.auto_https","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
{"level":"debug","ts":1732207894.3168488,"logger":"http.auto_https","msg":"adjusted config","tls":{"automation":{"policies":[{"subjects":["adguard.itsacomputer.com","pihole.itsacomputer.com"]},{}]}},"http":{"servers":{"remaining_auto_https_redirects":{"listen":[":80"],"routes":[{},{}]},"srv0":{"listen":[":443"],"routes":[{"handle":[{"handler":"subroute","routes":[{"handle":[{"handler":"subroute","routes":[{"handle":[{"handler":"reverse_proxy","upstreams":[{"dial":"192.168.160.3:80"}]}]}]}]}]}],"terminal":true},{"handle":[{"handler":"subroute","routes":[{"handle":[{"handler":"reverse_proxy","upstreams":[{"dial":"192.168.160.4:80"}]}]}]}],"terminal":true}],"tls_connection_policies":[{}],"automatic_https":{}}}}}
{"level":"info","ts":1732207894.3172226,"logger":"http","msg":"enabling HTTP/3 listener","addr":":443"}
{"level":"info","ts":1732207894.317329,"msg":"failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 7168 kiB, got: 416 kiB). See https://github.com/quic-go/quic-go/wiki/UDP-Buffer-Sizes for details."}
{"level":"debug","ts":1732207894.317509,"logger":"http","msg":"starting server loop","address":"[::]:443","tls":true,"http3":true}
{"level":"info","ts":1732207894.317546,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2","h3"]}
{"level":"debug","ts":1732207894.3176231,"logger":"http","msg":"starting server loop","address":"[::]:80","tls":false,"http3":false}
{"level":"info","ts":1732207894.317651,"logger":"http.log","msg":"server running","name":"remaining_auto_https_redirects","protocols":["h1","h2","h3"]}
{"level":"info","ts":1732207894.317669,"logger":"http","msg":"enabling automatic TLS certificate management","domains":["adguard.itsacomputer.com","pihole.itsacomputer.com"]}
{"level":"debug","ts":1732207894.31796,"logger":"tls","msg":"loading managed certificate","domain":"adguard.itsacomputer.com","expiration":1739852237,"issuer_key":"acme-v02.api.letsencrypt.org-directory","storage":"FileStorage:/data/caddy"}
{"level":"debug","ts":1732207894.3182745,"logger":"tls.cache","msg":"added certificate to cache","subjects":["adguard.itsacomputer.com"],"expiration":1739852237,"managed":true,"issuer_key":"acme-v02.api.letsencrypt.org-directory","hash":"19323e29ad4ca54222b6632793ba1120d0f757960fba182f1da17d019a0b1fc6","cache_size":1,"cache_capacity":10000}
{"level":"debug","ts":1732207894.3183346,"logger":"events","msg":"event","name":"cached_managed_cert","id":"dae91bba-428a-4101-8846-3068f90c41eb","origin":"tls","data":{"sans":["adguard.itsacomputer.com"]}}
{"level":"debug","ts":1732207894.3185444,"logger":"tls","msg":"loading managed certificate","domain":"pihole.itsacomputer.com","expiration":1739852237,"issuer_key":"acme-v02.api.letsencrypt.org-directory","storage":"FileStorage:/data/caddy"}
{"level":"info","ts":1732207894.318707,"logger":"tls","msg":"storage cleaning happened too recently; skipping for now","storage":"FileStorage:/data/caddy","instance":"4cabd726-13fb-4d9b-af70-d5ec01dbdbec","try_again":1732294294.318705,"try_again_in":86399.999999657}
{"level":"debug","ts":1732207894.3187566,"logger":"tls.cache","msg":"added certificate to cache","subjects":["pihole.itsacomputer.com"],"expiration":1739852237,"managed":true,"issuer_key":"acme-v02.api.letsencrypt.org-directory","hash":"01c075a9954fb232b44689acc56a35242412c2945b40bdbb0f09b5aff6c3bffb","cache_size":2,"cache_capacity":10000}
{"level":"info","ts":1732207894.3187711,"logger":"tls","msg":"finished cleaning storage units"}
{"level":"debug","ts":1732207894.3187764,"logger":"events","msg":"event","name":"cached_managed_cert","id":"841ba1da-4c38-479d-8d0e-1aa5ed222d2e","origin":"tls","data":{"sans":["pihole.itsacomputer.com"]}}
{"level":"info","ts":1732207894.3188932,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
{"level":"info","ts":1732207894.3189008,"logger":"admin.api","msg":"load complete"}
{"level":"info","ts":1732207894.3190486,"logger":"docker-proxy","msg":"Successfully configured","server":"localhost"}
{"level":"info","ts":1732207894.320076,"logger":"admin","msg":"stopped previous server","address":"localhost:2019"}
{"level":"info","ts":1732207904.2943392,"logger":"admin.api","msg":"received request","method":"GET","host":"127.0.0.1:2019","uri":"/config","remote_ip":"127.0.0.1","remote_port":"51622","headers":{"Accept":["*/*"],"User-Agent":["curl/8.9.1"]}}
{"level":"info","ts":1732207914.4043934,"logger":"admin.api","msg":"received request","method":"GET","host":"127.0.0.1:2019","uri":"/config","remote_ip":"127.0.0.1","remote_port":"55138","headers":{"Accept":["*/*"],"User-Agent":["curl/8.9.1"]}}
{"level":"debug","ts":1732207924.2972527,"logger":"docker-proxy","msg":"Skipping swarm config caddyfiles because swarm is not available"}
{"level":"debug","ts":1732207924.307535,"logger":"docker-proxy","msg":"Skipping swarm services because swarm is not available"}
{"level":"info","ts":1732207924.5121646,"logger":"admin.api","msg":"received request","method":"GET","host":"127.0.0.1:2019","uri":"/config","remote_ip":"127.0.0.1","remote_port":"41760","headers":{"Accept":["*/*"],"User-Agent":["curl/8.9.1"]}}
{"level":"info","ts":1732207934.5875986,"logger":"admin.api","msg":"received request","method":"GET","host":"127.0.0.1:2019","uri":"/config","remote_ip":"127.0.0.1","remote_port":"60470","headers":{"Accept":["*/*"],"User-Agent":["curl/8.9.1"]}}
{"level":"info","ts":1732207944.673132,"logger":"admin.api","msg":"received request","method":"GET","host":"127.0.0.1:2019","uri":"/config","remote_ip":"127.0.0.1","remote_port":"34942","headers":{"Accept":["*/*"],"User-Agent":["curl/8.9.1"]}}
{"level":"debug","ts":1732207954.297652,"logger":"docker-proxy","msg":"Skipping swarm config caddyfiles because swarm is not available"}
{"level":"debug","ts":1732207954.3074107,"logger":"docker-proxy","msg":"Skipping swarm services because swarm is not available"}
{"level":"info","ts":1732207954.7798202,"logger":"admin.api","msg":"received request","method":"GET","host":"127.0.0.1:2019","uri":"/config","remote_ip":"127.0.0.1","remote_port":"49950","headers":{"Accept":["*/*"],"User-Agent":["curl/8.9.1"]}}
{"level":"info","ts":1732207964.9016387,"logger":"admin.api","msg":"received request","method":"GET","host":"127.0.0.1:2019","uri":"/config","remote_ip":"127.0.0.1","remote_port":"53706","headers":{"Accept":["*/*"],"User-Agent":["curl/8.9.1"]}}
{"level":"info","ts":1732207975.009549,"logger":"admin.api","msg":"received request","method":"GET","host":"127.0.0.1:2019","uri":"/config","remote_ip":"127.0.0.1","remote_port":"53294","headers":{"Accept":["*/*"],"User-Agent":["curl/8.9.1"]}}
{"level":"debug","ts":1732207984.3115592,"logger":"docker-proxy","msg":"Skipping swarm config caddyfiles because swarm is not available"}
{"level":"debug","ts":1732207984.3239396,"logger":"docker-proxy","msg":"Skipping swarm services because swarm is not available"}
{"level":"info","ts":1732207985.1262395,"logger":"admin.api","msg":"received request","method":"GET","host":"127.0.0.1:2019","uri":"/config","remote_ip":"127.0.0.1","remote_port":"35456","headers":{"Accept":["*/*"],"User-Agent":["curl/8.9.1"]}}
{"level":"info","ts":1732207995.2593017,"logger":"admin.api","msg":"received request","method":"GET","host":"127.0.0.1:2019","uri":"/config","remote_ip":"127.0.0.1","remote_port":"41958","headers":{"Accept":["*/*"],"User-Agent":["curl/8.9.1"]}}
{"level":"info","ts":1732208005.3639455,"logger":"admin.api","msg":"received request","method":"GET","host":"127.0.0.1:2019","uri":"/config","remote_ip":"127.0.0.1","remote_port":"34116","headers":{"Accept":["*/*"],"User-Agent":["curl/8.9.1"]}}
{"level":"debug","ts":1732208014.2997506,"logger":"docker-proxy","msg":"Skipping swarm config caddyfiles because swarm is not available"}
{"level":"debug","ts":1732208014.3097286,"logger":"docker-proxy","msg":"Skipping swarm services because swarm is not available"}
{"level":"info","ts":1732208015.4533515,"logger":"admin.api","msg":"received request","method":"GET","host":"127.0.0.1:2019","uri":"/config","remote_ip":"127.0.0.1","remote_port":"57588","headers":{"Accept":["*/*"],"User-Agent":["curl/8.9.1"]}}
{"level":"info","ts":1732208025.5858638,"logger":"admin.api","msg":"received request","method":"GET","host":"127.0.0.1:2019","uri":"/config","remote_ip":"127.0.0.1","remote_port":"36716","headers":{"Accept":["*/*"],"User-Agent":["curl/8.9.1"]}}

francislavoie · November 21, 2024, 5:33pm

Your certs are already issued and get loaded properly. So I don’t see a problem here.

What do you think is the problem at this point, exactly?

the-bort-the · November 21, 2024, 5:58pm

I’m not sure of the problem other than I cannot access these sites from any browser and instead get a forbidden error. When looking at the cert, in the browser for either of those sites, it’s showing my openwrt router.

forbidden

OpenWRT

Whitestrake · November 22, 2024, 12:00am

That looks like DNS rebind protection in action if OpenWRT is responding.

I’m guessing your DNS is pointed at your router’s external IP address and you’re getting this error from a host inside your router’s LAN.

If that’s the case, you probably don’t want to disable DNS rebind protection, so you’ll want to look into enabling hairpin NAT instead so that OpenWRT forwards the internal → external → internal packets properly according to port forwards. I think on OpenWRT they refer to it as “loopback NAT”.

the-bort-the · November 25, 2024, 2:29pm

I think we have a winner! Adding the port forwards within OpenWRT finally allowed access to the sites. I will explore more on the OpenWRT side to see if other options are available as I was hoping to avoid exposing ports. Caddy docs said DNS challenge would not need to expose ports.

“This challenge does not require any open ports, and the server requesting a certificate does not need to be externally accessible.”

config redirect
	option name 'HTTP-Forward'
	option src 'wan'
	option src_dport '80'
	option dest 'lan'
	option dest_ip 'x.x.x.x'
	option dest_port '80'
	option target 'DNAT'
	option proto 'tcp'

config redirect
	option name 'HTTPS-Forward'
	option src 'wan'
	option src_dport '443'
	option dest 'lan'
	option dest_ip 'x.x.x.x'
	option dest_port '443'
	option target 'DNAT'
	option proto 'tcp'

Additional screen from advanced settings within OpenWRT port forward GUI:
nat_loopback

Whitestrake · November 25, 2024, 11:47pm

The DNS challenge doesn’t need ports open.

You only need these ports (and hairpin NAT) if you want to access Caddy via your router. Typically, because you want external access.

If you don’t want external access, you could instead configure your site address to resolve to the Caddy host’s LAN IP directly and skip the detour through your router entirely.

the-bort-the · November 27, 2024, 7:30pm

Just to clarify, this would involve inactivating the port forwards within OpenWRT, but also not changing the Caddyfile reverse proxy definitions to that of the LAN IP of the machine hosting the suite of containers, which includes the caddy container? Caddy overwrites any changes with the IP of the running container.

{
        email my@email.com
        acme_dns cloudflare {file./var/run/secrets/CLOUDFLARE_API_TOKEN}
}
app1.example.com {
        route {
                reverse_proxy 192.168.0.1:1023
        }
}
app2.example.com {
        route {
                reverse_proxy 192.168.0.1:80
        }
}
app3.example.com {
        route {
                reverse_proxy 192.168.0.1:1024
        }
}

Bruce5051 · November 27, 2024, 8:45pm

Supplemental information:
example.com is the proper domain name to use if you redacted the actual domain name.

As https://example.com/ shows the intended usages

Example Domain
This domain is for use in illustrative examples in documents. You may use this domain in literature without prior coordination or asking for permission.

Also see:

Whitestrake · November 27, 2024, 11:25pm

I’m a little bit confused as to exactly what you’re asking, here.

Whatever reverse-proxy definitions you were using before should still work - you were having Caddy proxy within your Docker network, right?

Right now, example.com points to your EXTERNAL IP address. That means requests from within your LAN to Caddy have to be routed out to your external IP address and back into your LAN, which requires OpenWRT to hairpin those packets and then port forward them to the right server.

All you need to do is cut out the middleman. Configure your DNS to resolve example.com domains directly to your Caddy host’s LAN IP so requests go directly to it. Then OpenWRT doesn’t need to be involved at all and it doesn’t matter whether it’s hairpinning or port forwarding, so you can turn those off if you like and disable external access.

I’m assuming you mean Caddy-Docker-Proxy?

All of that depends on how you’ve configured your Docker networking, and the Compose stack of your other services, which I don’t think you’ve shared.

the-bort-the · November 28, 2024, 2:25am

I’m going to mark the answer for exposing the ports externally as the solution because this is getting out of scope for the original ask/error. I appreciate the help! I have been reviewing the etc/config/firewall and etc/config/dhcp settings and will likely take the question over the OpenWRT community as I have nslookup showing my domain connecting to the LAN IP of the desired host.