How to configure Cloudflare?

simsrw73 · August 7, 2022, 7:54pm

1. Output of `caddy version`:

2.5.2

2. How I run Caddy:

a. System environment:

Docker running on Debian VM on Proxmox

Proxmox (OS: Proxmox VE 7.2-7 x86_64; Kernel: 5.15.39-2-pve)
Debian GNU/Linux 11 (bullseye) x86_64 (Host: KVM/QEMU (Standard PC (i440FX + PIIX, 1996) pc-i440fx-6.2); Kernel: 5.10.0-16-amd64)
Docker in Swarm Mode version 20.10.17

b. Command:

docker stack deploy -c docker-compose.yml caddy

c. Service/unit/compose file:

docker network create --driver overlay --opt encrypted --attachable proxy

Dockerfile:

FROM caddy:2.5.2-builder AS builder

RUN xcaddy build \
    --with github.com/caddy-dns/cloudflare

FROM caddy:2.5.2

COPY --from=builder /usr/bin/caddy /usr/bin/caddy

docker-compose.yml:

version: "3.7"

services:
  caddy:
    image: caddy-network_me:latest
    ports:
      - "80:80"
      - "443:443"
    networks:
      - proxy
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - /docker-services/caddy/data:/data
      - /docker-services/caddy/config:/config
      - /docker-services/caddy/logs:/var/log/caddy
      - /docker-services/caddy/Caddyfile:/etc/caddy/Caddyfile
    deploy:
      placement:
        constraints:
          - node.role == manager
      replicas: 1
      update_config:
	parallelism: 2
        delay: 10s
      restart_policy:
        condition: on-failure

networks:
  proxy:
    external: true
    attachable: true

d. My complete Caddy config:

Caddyfile:

{
    debug
    email admin@network.me
    acme_ca https://acme-staging-v02.api.letsencrypt.org/directory
}

netdata.network.me {
    reverse_proxy netdata:19999

	tls {
		protocols tls1.3
		dns cloudflare {env.CF_API_TOKEN}
		resolvers 1.1.1.1
	}
}

3. The problem I’m having:

Trying to learn Caddy and failing. I can’t seem to find any examples or tutorials. I want to use Caddy as reverse proxy on my small home network. I have Cloudflare dns configured and working (I’ve had this working with Nginx Proxy Manager, which is too simple, and somewhat with Traefik, which I somewhat despise). Ultimately, I’d like to have it configured for wildcard certs, which seems the recommended approach, but I can’t even get a simple config running.

Really, three issues:

The posted config fails.
How to do wildcard certs?
Is there a way to set global tls options, instead of repeating for each subdomain as above?

I would love recommendations on tutorials, examples, books, courses, anything to keep me from posting dumb questions.

4. Error messages and/or full log output:

{"level":"info","ts":1659899093.5184858,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":"caddyfile"}
{"level":"warn","ts":1659899093.5191138,"msg":"Caddyfile input is not formatted; run the 'caddy fmt' command to fix inconsistencies","adapter":"caddyfile","file":"/etc/caddy/Caddyfile","line":2}
{"level":"info","ts":1659899093.5195644,"logger":"admin","msg":"admin endpoint started","address":"tcp/localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
{"level":"info","ts":1659899093.519707,"logger":"http","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
{"level":"debug","ts":1659899093.5198865,"logger":"http","msg":"starting server loop","address":"[::]:443","http3":false,"tls":true}
{"level":"debug","ts":1659899093.519922,"logger":"http","msg":"starting server loop","address":"[::]:80","http3":false,"tls":false}
{"level":"info","ts":1659899093.5199273,"logger":"http","msg":"enabling automatic TLS certificate management","domains":["netdata.network.me"]}
{"level":"info","ts":1659899093.5200484,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
{"level":"info","ts":1659899093.520149,"msg":"serving initial configuration"}
{"level":"info","ts":1659899093.5201187,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc0003ec230"}
{"level":"info","ts":1659899093.52024,"logger":"tls","msg":"cleaning storage unit","description":"FileStorage:/data/caddy"}
{"level":"info","ts":1659899093.5202775,"logger":"tls","msg":"finished cleaning storage units"}
{"level":"info","ts":1659899093.5204654,"logger":"tls.obtain","msg":"acquiring lock","identifier":"netdata.network.me"}
{"level":"info","ts":1659899093.5236504,"logger":"tls.obtain","msg":"lock acquired","identifier":"netdata.network.me"}
{"level":"debug","ts":1659899093.5239449,"logger":"tls.obtain","msg":"trying issuer 1/2","issuer":"acme-v02.api.letsencrypt.org-directory"}
{"level":"info","ts":1659899093.5240948,"logger":"tls.issuance.acme","msg":"waiting on internal rate limiter","identifiers":["netdata.network.me"],"ca":"https://acme-v02.api.letsencrypt.org/directory","account":"admin@network.me"}
{"level":"info","ts":1659899093.5241046,"logger":"tls.issuance.acme","msg":"done waiting on internal rate limiter","identifiers":["netdata.network.me"],"ca":"https://acme-v02.api.letsencrypt.org/directory","account":"admin@network.me"}
{"level":"debug","ts":1659899095.5628524,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"GET","url":"https://acme-v02.api.letsencrypt.org/directory","headers":{"User-Agent":["Caddy/2.5.2 CertMagic acmez (linux; amd64)"]},"response_headers":{"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["658"],"Content-Type":["application/json"],"Date":["Sun, 07 Aug 2022 19:04:54 GMT"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":200}
{"level":"debug","ts":1659899096.210765,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"HEAD","url":"https://acme-v02.api.letsencrypt.org/acme/new-nonce","headers":{"User-Agent":["Caddy/2.5.2 CertMagic acmez (linux; amd64)"]},"response_headers":{"Cache-Control":["public, max-age=0, no-cache"],"Date":["Sun, 07 Aug 2022 19:04:55 GMT"],"Link":["<https://acme-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["0101pTmfo8waY_lHsM93Plwi7XLJi2bB1pmC9rY8cXiZjCI"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":200}
{"level":"debug","ts":1659899097.1108932,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-v02.api.letsencrypt.org/acme/new-order","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.5.2 CertMagic acmez (linux; amd64)"]},"response_headers":{"Boulder-Requester":["666481256"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["352"],"Content-Type":["application/json"],"Date":["Sun, 07 Aug 2022 19:04:56 GMT"],"Link":["<https://acme-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Location":["https://acme-v02.api.letsencrypt.org/acme/order/666481256/114051344106"],"Replay-Nonce":["0101coPHbJQ5QiLqgIi9sU7DnGaDInVNqftA4dNi6-UnWPQ"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":201}
{"level":"debug","ts":1659899097.6656504,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-v02.api.letsencrypt.org/acme/authz-v3/139450001606","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.5.2 CertMagic acmez (linux; amd64)"]},"response_headers":{"Boulder-Requester":["666481256"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["811"],"Content-Type":["application/json"],"Date":["Sun, 07 Aug 2022 19:04:57 GMT"],"Link":["<https://acme-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["0101kjYxjS03oUCduuqdaIjX2m8n6sJd8cZGRg7uu6UYQBo"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":200}
{"level":"debug","ts":1659899097.6657627,"logger":"tls.issuance.acme.acme_client","msg":"no solver configured","challenge_type":"tls-alpn-01"}
{"level":"debug","ts":1659899097.6657715,"logger":"tls.issuance.acme.acme_client","msg":"no solver configured","challenge_type":"http-01"}
{"level":"info","ts":1659899097.6657758,"logger":"tls.issuance.acme.acme_client","msg":"trying to solve challenge","identifier":"netdata.network.me","challenge_type":"dns-01","ca":"https://acme-v02.api.letsencrypt.org/directory"}
{"level":"error","ts":1659899099.7435234,"logger":"tls.issuance.acme.acme_client","msg":"cleaning up solver","identifier":"netdata.network.me","challenge_type":"dns-01","error":"no memory of presenting a DNS record for netdata.network.me (probably OK if presenting failed)"}
{"level":"debug","ts":1659899100.529625,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-v02.api.letsencrypt.org/acme/authz-v3/139450001606","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.5.2 CertMagic acmez (linux; amd64)"]},"response_headers":{"Boulder-Requester":["666481256"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["815"],"Content-Type":["application/json"],"Date":["Sun, 07 Aug 2022 19:04:59 GMT"],"Link":["<https://acme-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["0101trigv2fXymQy96cOdHHPWlCi4vNVACTCCE96Q58pc48"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":200}
{"level":"error","ts":1659899100.5297222,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"netdata.network.me","issuer":"acme-v02.api.letsencrypt.org-directory","error":"[netdata.network.me] solving challenges: presenting for challenge: adding temporary record for zone network.me.: got error status: HTTP 400: [{Code:6003 Message:Invalid request headers}] (order=https://acme-v02.api.letsencrypt.org/acme/order/666481256/114051344106) (ca=https://acme-v02.api.letsencrypt.org/directory)"}
{"level":"debug","ts":1659899100.5297325,"logger":"tls.obtain","msg":"trying issuer 2/2","issuer":"acme.zerossl.com-v2-DV90"}
{"level":"info","ts":1659899102.574443,"logger":"tls.issuance.zerossl","msg":"generated EAB credentials","key_id":"pTlmIoltndgdcENucddpNg"}
{"level":"debug","ts":1659899108.2629573,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"GET","url":"https://acme.zerossl.com/v2/DV90","headers":{"User-Agent":["Caddy/2.5.2 CertMagic acmez (linux; amd64)"]},"response_headers":{"Access-Control-Allow-Origin":["*"],"Cache-Control":["max-age=-1"],"Content-Length":["645"],"Content-Type":["application/json"],"Date":["Sun, 07 Aug 2022 19:05:08 GMT"],"Server":["nginx"],"Strict-Transport-Security":["max-age=15552000"]},"status_code":200}
{"level":"debug","ts":1659899113.2738614,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"HEAD","url":"https://acme.zerossl.com/v2/DV90/newNonce","headers":{"User-Agent":["Caddy/2.5.2 CertMagic acmez (linux; amd64)"]},"response_headers":{"Access-Control-Allow-Origin":["*"],"Cache-Control":["max-age=-1"],"Content-Type":["application/octet-stream"],"Date":["Sun, 07 Aug 2022 19:05:13 GMT"],"Link":["<https://acme.zerossl.com/v2/DV90>;rel=\"index\""],"Replay-Nonce":["AqDioNhH3MvrjA1DbtiZDJGjkTrLshI8d_UVcsIU2NU"],"Server":["nginx"],"Strict-Transport-Security":["max-age=15552000"]},"status_code":200}

5. What I already tried:

I’ve spent the last 4 hours Googling and trying to find anything relevant and keep striking out.

francislavoie · August 7, 2022, 9:37pm

Probably because you didn’t actually provide the CF_API_TOKEN environment variable to your container. The error message says that authentication with Cloudflare failed.

Why do you need wildcards, exactly? If you’re trying to use them “just because it’d be nice”, you probably shouldn’t, because it complicates things a bit.

Yes, with the acme_dns global option.

Don’t set this. The defaults of TLS1.2 minimum and TLS 1.3 maximum are good.

I don’t think you need this. Your Caddy container isn’t communicating with the Docker socket.

simsrw73 · August 8, 2022, 12:54am

Thanks. I set the global option acme_dns and it is now acquiring the cert. It shows success in the logfile and I can see it in the data directory.

{"level":"info","ts":1659919480.01905,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":"caddyfile"}
{"level":"warn","ts":1659919480.0201318,"msg":"Caddyfile input is not formatted; run the 'caddy fmt' command to fix inconsistencies","adapter":"caddyfile","file":"/etc/caddy/Caddyfile","line":2}
{"level":"info","ts":1659919480.0206802,"logger":"admin","msg":"admin endpoint started","address":"tcp/localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
{"level":"info","ts":1659919480.0209112,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc00044e930"}
{"level":"info","ts":1659919480.0210173,"logger":"http","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443}
{"level":"info","ts":1659919480.0210314,"logger":"http","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
{"level":"debug","ts":1659919480.0212455,"logger":"http","msg":"starting server loop","address":"[::]:80","http3":false,"tls":false}
{"level":"debug","ts":1659919480.0213063,"logger":"http","msg":"starting server loop","address":"[::]:443","http3":false,"tls":true}
{"level":"info","ts":1659919480.0213594,"logger":"http","msg":"enabling automatic TLS certificate management","domains":["netdata.network.me"]}
{"level":"debug","ts":1659919480.0216527,"logger":"tls","msg":"loading managed certificate","domain":"netdata.network.me","expiration":1667679064,"issuer_key":"acme-v02.api.letsencrypt.org-directory","storage":"FileStorage:/data/caddy"}
{"level":"debug","ts":1659919480.0218184,"logger":"tls.cache","msg":"added certificate to cache","subjects":["netdata.network.me"],"expiration":1667679064,"managed":true,"issuer_key":"acme-v02.api.letsencrypt.org-directory","hash":"60de715e838a7ec70a8a95c5960cf88f035f05e3d850fca7c5c2a9d6bdf12109","cache_size":1,"cache_capacity":10000}
{"level":"info","ts":1659919480.0219269,"logger":"tls","msg":"cleaning storage unit","description":"FileStorage:/data/caddy"}
{"level":"info","ts":1659919480.0222201,"logger":"tls","msg":"finished cleaning storage units"}
{"level":"info","ts":1659919480.0222878,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
{"level":"info","ts":1659919480.0222971,"msg":"serving initial configuration"}

The problem now is that it’s failing because of the way I’m referencing the container, I think. The log doesn’t show anything after acquiring the cert. Should it be logging when I try to access the url?

I’ve tried variations on

netdata.network.me {
    reverse_proxy localhost:19999
}

How do I reference a service that is deployed in the swarm? Or otherwise debug this?

Using netdata as a test container, and abbreviated here:

services:
  netdata:
    image: netdata/netdata
    ports:
      - 19999:19999
    networks:
      - proxy
    deploy:
      mode:
        global

networks:
  proxy:
    external: true

I’m eventually going to look at caddy-docker-proxy, but I’m trying to wrap my head around vanilla Caddy first.

francislavoie · August 8, 2022, 1:19am

Don’t use localhost, use the name of the service in Docker. The Docker DNS resolver will resolve container and service names to the IP address of the container in the docker network.

Using localhost basically tells Caddy "try to connect to something inside the same container which is impossible because the only thing running in that container is Caddy itself.

simsrw73 · August 15, 2022, 9:32pm

I’m not sure if I should post back here or start a new topic. After your previous help all has gone well. Silly mistakes on my part. I’ve been exploring some of the other features of Caddy and everything went well until I tried to use the caddy-docker-proxy plugin. I’m getting a similar error failing to get cert, I think. I always felt like a fairly competent programmer and tech nerd, but networking, dev-ops, tls stuff makes me feel like an idiot. Maybe age is catching up to me.

It’s creating the dns entries on cloudflare, but failing after that. Brave browser shows:

whoami.mysmarthome.network sent an invalid response.
ERR_SSL_PROTOCOL_ERROR

Dockerfile:

ARG CADDY_VERSION=2.5.2
FROM caddy:${CADDY_VERSION}-builder AS builder

RUN xcaddy build \
    --with github.com/lucaslorentz/caddy-docker-proxy/plugin \
    --with github.com/caddy-dns/cloudflare

FROM caddy:${CADDY_VERSION}-alpine

COPY --from=builder /usr/bin/caddy /usr/bin/caddy

CMD ["caddy", "docker-proxy"]

Caddy docker-compose.yml

version: "3.7"

services:
  caddy:
    image: caddy-docker-proxy--mysmarthome_network:latest
    ports:
      - "80:80"
      - "443:443"
    networks:
      - proxy
    env_file: .env
    environment:
      - CADDY_INGRESS_NETWORKS=proxy
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - /docker-services/caddy/data:/data
      - /docker-services/caddy/config:/config
      - /docker-services/caddy/logs:/var/log/caddy
    deploy:
      labels:
        caddy.debug:
        caddy.log.output: file /var/log/caddy/access.log
        caddy.acme_dns: "cloudflare {env.CF_API_TOKEN}"
        caddy.email: "{env.EMAIL}"
      placement:
        constraints:
          - node.role == manager
      replicas: 1
      resources:
        reservations:
          cpus: "0.1"
          memory: 200M
      restart_policy:
        condition: any

networks:
  proxy:
    external: true
  cloudflared:
    external: true

whoami docker-compose.yml (test container)

version: "3.7"

services:

  whoami:
    image: jwilder/whoami
    networks:
      - proxy
    deploy:
      labels:
        caddy: whoami.mysmarthome.network
        caddy.reverse_proxy: "{{upstreams 8000}}"
        caddy.tls.ca: https://acme-staging-v02.api.letsencrypt.org/directory

networks:
  proxy:
    external: true

logs:

{"level":"info","ts":1660598417.0931878,"logger":"docker-proxy","msg":"Running caddy proxy server"}
{"level":"info","ts":1660598417.0945556,"logger":"admin","msg":"admin endpoint started","address":"tcp/localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
{"level":"info","ts":1660598417.0946932,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
{"level":"info","ts":1660598417.0947223,"logger":"docker-proxy","msg":"Running caddy proxy controller"}
{"level":"info","ts":1660598417.0950556,"logger":"docker-proxy","msg":"Start","CaddyfilePath":"","LabelPrefix":"caddy","PollingInterval":30,"ProcessCaddyfile":true,"ProxyServiceTasks":true,"IngressNetworks":"[proxy]","DockerSockets":[""],"DockerCertsPath":[""],"DockerAPIsVersion":[""]}
{"level":"info","ts":1660598417.095542,"logger":"docker-proxy","msg":"Connecting to docker events","DockerSocket":""}
{"level":"info","ts":1660598417.095826,"logger":"docker-proxy","msg":"IngressNetworksMap","ingres":"map[8ajj8n5wc7vunwk13xvln2ot9:true]"}
{"level":"info","ts":1660598417.101687,"logger":"docker-proxy","msg":"Swarm is available","new":true}
{"level":"info","ts":1660598417.1036057,"logger":"docker-proxy","msg":"New Caddyfile","caddyfile":"{\n\tacme_dns cloudflare {env.CF_API_TOKEN}\n\tdebug\n\temail {env.EMAIL}\n\tlog {\n\t\toutput file /var/log/caddy/access.log\n\t}\n}\n"}
{"level":"info","ts":1660598417.103688,"logger":"docker-proxy","msg":"New Config JSON","json":"{\"logging\":{\"logs\":{\"default\":{\"writer\":{\"filename\":\"/var/log/caddy/access.log\",\"output\":\"file\"},\"level\":\"DEBUG\"}}}}"}
{"level":"info","ts":1660598417.1037083,"logger":"docker-proxy","msg":"Sending configuration to","server":"localhost"}
{"level":"info","ts":1660598417.1042159,"logger":"admin.api","msg":"received request","method":"POST","host":"localhost:2019","uri":"/load","remote_ip":"127.0.0.1","remote_port":"57026","headers":{"Accept-Encoding":["gzip"],"Content-Length":["156"],"Content-Type":["application/json"],"User-Agent":["Go-http-client/1.1"]}}
{"level":"info","ts":1660598417.1043327,"msg":"redirected default logger","from":"stderr","to":"/var/log/caddy/access.log"}
{"level":"info","ts":1660598417.104711,"logger":"docker-proxy","msg":"Successfully configured","server":"localhost"}

---

{"level":"info","ts":1660598417.1044695,"logger":"admin","msg":"admin endpoint started","address":"tcp/localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
{"level":"info","ts":1660598417.1046367,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
{"level":"info","ts":1660598417.104645,"logger":"admin.api","msg":"load complete"}
{"level":"info","ts":1660598417.1059852,"logger":"admin","msg":"stopped previous server","address":"tcp/localhost:2019"}
{"level":"debug","ts":1660598424.96045,"logger":"docker-proxy","msg":"Skipping default Caddyfile because no path is set"}
{"level":"debug","ts":1660598424.9618394,"logger":"docker-proxy","msg":"Swarm service","service":"caddy_caddy"}
{"level":"debug","ts":1660598424.9619174,"logger":"docker-proxy","msg":"Swarm service","service":"whoami_whoami"}
{"level":"warn","ts":1660598424.9626446,"logger":"docker-proxy","msg":"Service has no tasks in running state","service":"whoami_whoami","serviceId":"s2atx8cw4idbgj315qezxpn22"}
{"level":"info","ts":1660598424.9632492,"logger":"docker-proxy","msg":"New Caddyfile","caddyfile":"{\n\tacme_dns cloudflare {env.CF_API_TOKEN}\n\tdebug\n\temail {env.EMAIL}\n\tlog {\n\t\toutput file /var/log/caddy/access.log\n\t}\n}\nwhoami.mysmarthome.network {\n\treverse_proxy\n}\n"}
{"level":"info","ts":1660598424.9633925,"logger":"docker-proxy","msg":"New Config JSON","json":"{\"logging\":{\"logs\":{\"default\":{\"writer\":{\"filename\":\"/var/log/caddy/access.log\",\"output\":\"file\"},\"level\":\"DEBUG\"}}},\"apps\":{\"http\":{\"servers\":{\"srv0\":{\"listen\":[\":443\"],\"routes\":[{\"match\":[{\"host\":[\"whoami.mysmarthome.network\"]}],\"handle\":[{\"handler\":\"subroute\",\"routes\":[{\"handle\":[{\"handler\":\"reverse_proxy\"}]}]}],\"terminal\":true}]}}},\"tls\":{\"automation\":{\"policies\":[{\"subjects\":[\"whoami.mysmarthome.network\"],\"issuers\":[{\"challenges\":{\"dns\":{\"provider\":{\"api_token\":\"{env.CF_API_TOKEN}\",\"name\":\"cloudflare\"}}},\"email\":\"{env.EMAIL}\",\"module\":\"acme\"},{\"challenges\":{\"dns\":{\"provider\":{\"api_token\":\"{env.CF_API_TOKEN}\",\"name\":\"cloudflare\"}}},\"email\":\"{env.EMAIL}\",\"module\":\"zerossl\"}]}]}}}}"}
{"level":"info","ts":1660598424.9634173,"logger":"docker-proxy","msg":"Sending configuration to","server":"localhost"}
{"level":"info","ts":1660598424.9637525,"logger":"admin.api","msg":"received request","method":"POST","host":"localhost:2019","uri":"/load","remote_ip":"127.0.0.1","remote_port":"37246","headers":{"Accept-Encoding":["gzip"],"Content-Length":["731"],"Content-Type":["application/json"],"User-Agent":["Go-http-client/1.1"]}}
{"level":"info","ts":1660598424.9639575,"logger":"admin","msg":"admin endpoint started","address":"tcp/localhost:2019","enforce_origin":false,"origins":["//127.0.0.1:2019","//localhost:2019","//[::1]:2019"]}
{"level":"info","ts":1660598424.9640658,"logger":"http","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443}
{"level":"info","ts":1660598424.9640777,"logger":"http","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
{"level":"debug","ts":1660598424.964238,"logger":"http","msg":"starting server loop","address":"[::]:443","http3":false,"tls":true}
{"level":"debug","ts":1660598424.9642696,"logger":"http","msg":"starting server loop","address":"[::]:80","http3":false,"tls":false}
{"level":"info","ts":1660598424.9642766,"logger":"http","msg":"enabling automatic TLS certificate management","domains":["whoami.mysmarthome.network"]}
{"level":"info","ts":1660598424.9643984,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
{"level":"info","ts":1660598424.9644065,"logger":"admin.api","msg":"load complete"}
{"level":"info","ts":1660598424.964569,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc000102af0"}
{"level":"info","ts":1660598424.9646544,"logger":"tls","msg":"cleaning storage unit","description":"FileStorage:/data/caddy"}
{"level":"info","ts":1660598424.9646788,"logger":"tls","msg":"finished cleaning storage units"}
{"level":"info","ts":1660598424.964751,"logger":"docker-proxy","msg":"Successfully configured","server":"localhost"}
{"level":"info","ts":1660598424.964874,"logger":"tls.obtain","msg":"acquiring lock","identifier":"whoami.mysmarthome.network"}
{"level":"info","ts":1660598424.9657602,"logger":"admin","msg":"stopped previous server","address":"tcp/localhost:2019"}
{"level":"info","ts":1660598424.9682083,"logger":"tls.obtain","msg":"lock acquired","identifier":"whoami.mysmarthome.network"}
{"level":"debug","ts":1660598424.968486,"logger":"tls.obtain","msg":"trying issuer 1/2","issuer":"acme-v02.api.letsencrypt.org-directory"}
{"level":"debug","ts":1660598425.1854303,"logger":"docker-proxy","msg":"Skipping default Caddyfile because no path is set"}
{"level":"debug","ts":1660598425.1871543,"logger":"docker-proxy","msg":"Swarm service","service":"caddy_caddy"}
{"level":"debug","ts":1660598425.1872456,"logger":"docker-proxy","msg":"Swarm service","service":"whoami_whoami"}
{"level":"warn","ts":1660598425.1881094,"logger":"docker-proxy","msg":"Service has no tasks in running state","service":"whoami_whoami","serviceId":"s2atx8cw4idbgj315qezxpn22"}
{"level":"debug","ts":1660598425.2049625,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"GET","url":"https://acme-v02.api.letsencrypt.org/directory","headers":{"User-Agent":["Caddy/2.5.2 CertMagic acmez (linux; amd64)"]},"response_headers":{"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["658"],"Content-Type":["application/json"],"Date":["Mon, 15 Aug 2022 21:20:25 GMT"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":200}
{"level":"debug","ts":1660598425.2546258,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"HEAD","url":"https://acme-v02.api.letsencrypt.org/acme/new-nonce","headers":{"User-Agent":["Caddy/2.5.2 CertMagic acmez (linux; amd64)"]},"response_headers":{"Cache-Control":["public, max-age=0, no-cache"],"Date":["Mon, 15 Aug 2022 21:20:25 GMT"],"Link":["<https://acme-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["0102zYynDAW1qtd7kPoYbQKzwwX70u0cqSzPBnHxZsWWlAs"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":200}
{"level":"debug","ts":1660598425.3852293,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-v02.api.letsencrypt.org/acme/new-acct","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.5.2 CertMagic acmez (linux; amd64)"]},"response_headers":{"Boulder-Requester":["679699397"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["328"],"Content-Type":["application/json"],"Date":["Mon, 15 Aug 2022 21:20:25 GMT"],"Link":["<https://acme-v02.api.letsencrypt.org/directory>;rel=\"index\"","<https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf>;rel=\"terms-of-service\""],"Location":["https://acme-v02.api.letsencrypt.org/acme/acct/679699397"],"Replay-Nonce":["0101JgFsPTn3w6joJmSyF4dhWzVsYY90oYnhr2RuHRoDRyM"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":201}
{"level":"info","ts":1660598425.3855195,"logger":"tls.issuance.acme","msg":"waiting on internal rate limiter","identifiers":["whoami.mysmarthome.network"],"ca":"https://acme-v02.api.letsencrypt.org/directory","account":"admin@mysmarthome.network"}
{"level":"info","ts":1660598425.3855283,"logger":"tls.issuance.acme","msg":"done waiting on internal rate limiter","identifiers":["whoami.mysmarthome.network"],"ca":"https://acme-v02.api.letsencrypt.org/directory","account":"admin@mysmarthome.network"}
{"level":"debug","ts":1660598425.5352948,"logger":"docker-proxy","msg":"Skipping default Caddyfile because no path is set"}
{"level":"debug","ts":1660598425.5367262,"logger":"docker-proxy","msg":"Swarm service","service":"caddy_caddy"}
{"level":"debug","ts":1660598425.5367835,"logger":"docker-proxy","msg":"Swarm service","service":"whoami_whoami"}
{"level":"warn","ts":1660598425.5373206,"logger":"docker-proxy","msg":"Service has no tasks in running state","service":"whoami_whoami","serviceId":"s2atx8cw4idbgj315qezxpn22"}
{"level":"debug","ts":1660598425.6926916,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-v02.api.letsencrypt.org/acme/new-order","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.5.2 CertMagic acmez (linux; amd64)"]},"response_headers":{"Boulder-Requester":["679699397"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["351"],"Content-Type":["application/json"],"Date":["Mon, 15 Aug 2022 21:20:25 GMT"],"Link":["<https://acme-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Location":["https://acme-v02.api.letsencrypt.org/acme/order/679699397/116535060277"],"Replay-Nonce":["0102Lt_F788GlmMicY7perCvccHy01Jdm5T74C558-C1Joo"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":201}
{"level":"debug","ts":1660598425.7548785,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-v02.api.letsencrypt.org/acme/authz-v3/142455956147","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.5.2 CertMagic acmez (linux; amd64)"]},"response_headers":{"Boulder-Requester":["679699397"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["810"],"Content-Type":["application/json"],"Date":["Mon, 15 Aug 2022 21:20:25 GMT"],"Link":["<https://acme-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["0101QmIVaJnqeb2uPWHb6yAn6gMM0ZVY8-Qn8_GBr1s7YXs"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":200}
{"level":"debug","ts":1660598425.754954,"logger":"tls.issuance.acme.acme_client","msg":"no solver configured","challenge_type":"http-01"}
{"level":"debug","ts":1660598425.7549617,"logger":"tls.issuance.acme.acme_client","msg":"no solver configured","challenge_type":"tls-alpn-01"}
{"level":"info","ts":1660598425.7549663,"logger":"tls.issuance.acme.acme_client","msg":"trying to solve challenge","identifier":"whoami.mysmarthome.network","challenge_type":"dns-01","ca":"https://acme-v02.api.letsencrypt.org/directory"}
{"level":"debug","ts":1660598455.535497,"logger":"docker-proxy","msg":"Skipping default Caddyfile because no path is set"}
{"level":"debug","ts":1660598455.5372446,"logger":"docker-proxy","msg":"Swarm service","service":"caddy_caddy"}
{"level":"debug","ts":1660598455.5386999,"logger":"docker-proxy","msg":"Swarm service","service":"whoami_whoami"}
{"level":"info","ts":1660598455.539633,"logger":"docker-proxy","msg":"New Caddyfile","caddyfile":"{\n\tacme_dns cloudflare {env.CF_API_TOKEN}\n\tdebug\n\temail {env.EMAIL}\n\tlog {\n\t\toutput file /var/log/caddy/access.log\n\t}\n}\nwhoami.mysmarthome.network {\n\treverse_proxy 10.0.1.146:8000\n}\n"}
{"level":"info","ts":1660598455.5397563,"logger":"docker-proxy","msg":"New Config JSON","json":"{\"logging\":{\"logs\":{\"default\":{\"writer\":{\"filename\":\"/var/log/caddy/access.log\",\"output\":\"file\"},\"level\":\"DEBUG\"}}},\"apps\":{\"http\":{\"servers\":{\"srv0\":{\"listen\":[\":443\"],\"routes\":[{\"match\":[{\"host\":[\"whoami.mysmarthome.network\"]}],\"handle\":[{\"handler\":\"subroute\",\"routes\":[{\"handle\":[{\"handler\":\"reverse_proxy\",\"upstreams\":[{\"dial\":\"10.0.1.146:8000\"}]}]}]}],\"terminal\":true}]}}},\"tls\":{\"automation\":{\"policies\":[{\"subjects\":[\"whoami.mysmarthome.network\"],\"issuers\":[{\"challenges\":{\"dns\":{\"provider\":{\"api_token\":\"{env.CF_API_TOKEN}\",\"name\":\"cloudflare\"}}},\"email\":\"{env.EMAIL}\",\"module\":\"acme\"},{\"challenges\":{\"dns\":{\"provider\":{\"api_token\":\"{env.CF_API_TOKEN}\",\"name\":\"cloudflare\"}}},\"email\":\"{env.EMAIL}\",\"module\":\"zerossl\"}]}]}}}}"}

francislavoie · August 15, 2022, 10:38pm

This label isn’t writing just access logs there, it’s writing all of Caddy’s runtime logs there. Access logs must be configured per-site with the log directive (different from the log global option).

So your logs for TLS issuance are probably in that file, and not in the docker container’s stdout, because you configured that.

I’m not seeing any issue with what you posted in particular.

simsrw73 · August 15, 2022, 10:55pm

Ahh, yeah. This config has been through many revisions and then scaled back to the bare config that demonstrates the issue and I didn’t rename some of files, etc. The logs above contain the docker logs up to the — and the rest is the content of the access.log file. Or up to a point where it seemed to be repeating. Then entire content wasn’t allowed in a single post. Below is the continuation of that file to the end. It never allows me to access the subdomain through the browser.

{"level":"info","ts":1660598455.5397563,"logger":"docker-proxy","msg":"New Config JSON","json":"{\"logging\":{\"logs\":{\"default\":{\"writer\":{\"filename\":\"/var/log/caddy/access.log\",\"output\":\"file\"},\"level\":\"DEBUG\"}}},\"apps\":{\"http\":{\"servers\":{\"srv0\":{\"listen\":[\":443\"],\"routes\":[{\"match\":[{\"host\":[\"whoami.mysmarthome.network\"]}],\"handle\":[{\"handler\":\"subroute\",\"routes\":[{\"handle\":[{\"handler\":\"reverse_proxy\",\"upstreams\":[{\"dial\":\"10.0.1.146:8000\"}]}]}]}],\"terminal\":true}]}}},\"tls\":{\"automation\":{\"policies\":[{\"subjects\":[\"whoami.mysmarthome.network\"],\"issuers\":[{\"challenges\":{\"dns\":{\"provider\":{\"api_token\":\"{env.CF_API_TOKEN}\",\"name\":\"cloudflare\"}}},\"email\":\"{env.EMAIL}\",\"module\":\"acme\"},{\"challenges\":{\"dns\":{\"provider\":{\"api_token\":\"{env.CF_API_TOKEN}\",\"name\":\"cloudflare\"}}},\"email\":\"{env.EMAIL}\",\"module\":\"zerossl\"}]}]}}}}"}
{"level":"info","ts":1660598455.539781,"logger":"docker-proxy","msg":"Sending configuration to","server":"localhost"}
{"level":"info","ts":1660598455.5401676,"logger":"admin.api","msg":"received request","method":"POST","host":"localhost:2019","uri":"/load","remote_ip":"127.0.0.1","remote_port":"54792","headers":{"Accept-Encoding":["gzip"],"Content-Length":["772"],"Content-Type":["application/json"],"User-Agent":["Go-http-client/1.1"]}}
{"level":"info","ts":1660598455.5404184,"logger":"admin","msg":"admin endpoint started","address":"tcp/localhost:2019","enforce_origin":false,"origins":["//127.0.0.1:2019","//localhost:2019","//[::1]:2019"]}
{"level":"info","ts":1660598455.5405028,"logger":"http","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443}
{"level":"info","ts":1660598455.5405138,"logger":"http","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
{"level":"debug","ts":1660598455.5405896,"logger":"http","msg":"starting server loop","address":"[::]:443","http3":false,"tls":true}
{"level":"info","ts":1660598455.540599,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc00079e620"}
{"level":"debug","ts":1660598455.540602,"logger":"http","msg":"starting server loop","address":"[::]:80","http3":false,"tls":false}
{"level":"info","ts":1660598455.5406153,"logger":"http","msg":"enabling automatic TLS certificate management","domains":["whoami.mysmarthome.network"]}
{"level":"info","ts":1660598455.5408356,"logger":"tls.obtain","msg":"acquiring lock","identifier":"whoami.mysmarthome.network"}
{"level":"info","ts":1660598455.5429823,"logger":"tls.cache.maintenance","msg":"stopped background certificate maintenance","cache":"0xc000102af0"}
{"level":"error","ts":1660598455.5430634,"logger":"tls.issuance.acme.acme_client","msg":"cleaning up solver","identifier":"whoami.mysmarthome.network","challenge_type":"dns-01","error":"deleting temporary record for zone mysmarthome.network.: Delete \"https://api.cloudflare.com/client/v4/zones/4c38d8a7f0598c0cc42ab69d65eeb822/dns_records/f0ea0543888554b387b9812f0d4f8051\": context canceled"}
{"level":"info","ts":1660598455.5430856,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
{"level":"info","ts":1660598455.5431697,"logger":"admin.api","msg":"load complete"}
{"level":"warn","ts":1660598455.543176,"logger":"tls.issuance.acme.acme_client","msg":"HTTP request failed; retrying","url":"https://acme-v02.api.letsencrypt.org/acme/authz-v3/142455956147","error":"performing request: Post \"https://acme-v02.api.letsencrypt.org/acme/authz-v3/142455956147\": context canceled"}
{"level":"error","ts":1660598455.5431905,"logger":"tls.issuance.acme.acme_client","msg":"deactivating authorization","identifier":"whoami.mysmarthome.network","authz":"https://acme-v02.api.letsencrypt.org/acme/authz-v3/142455956147","error":"attempt 1: https://acme-v02.api.letsencrypt.org/acme/authz-v3/142455956147: context canceled"}
{"level":"error","ts":1660598455.5432036,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"whoami.mysmarthome.network","issuer":"acme-v02.api.letsencrypt.org-directory","error":"[whoami.mysmarthome.network] solving challenges: waiting for solver certmagic.solverWrapper to be ready: context canceled (order=https://acme-v02.api.letsencrypt.org/acme/order/679699397/116535060277) (ca=https://acme-v02.api.letsencrypt.org/directory)"}
{"level":"debug","ts":1660598455.5432258,"logger":"tls.obtain","msg":"trying issuer 2/2","issuer":"acme.zerossl.com-v2-DV90"}
{"level":"info","ts":1660598455.5432696,"logger":"docker-proxy","msg":"Successfully configured","server":"localhost"}
{"level":"error","ts":1660598455.5432904,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"whoami.mysmarthome.network","issuer":"acme.zerossl.com-v2-DV90","error":"account pre-registration callback: performing EAB credentials request: Post \"https://api.zerossl.com/acme/eab-credentials-email\": context canceled"}
{"level":"error","ts":1660598455.5433016,"logger":"tls.obtain","msg":"will retry","error":"[whoami.mysmarthome.network] Obtain: account pre-registration callback: performing EAB credentials request: Post \"https://api.zerossl.com/acme/eab-credentials-email\": context canceled","attempt":1,"retrying_in":60,"elapsed":30.575082114,"max_duration":2592000}
{"level":"info","ts":1660598455.543314,"logger":"tls.obtain","msg":"releasing lock","identifier":"whoami.mysmarthome.network"}
{"level":"error","ts":1660598455.5433521,"logger":"tls","msg":"job failed","error":"whoami.mysmarthome.network: obtaining certificate: context canceled"}
{"level":"info","ts":1660598455.5444603,"logger":"admin","msg":"stopped previous server","address":"tcp/localhost:2019"}
{"level":"info","ts":1660598456.5660381,"logger":"tls.obtain","msg":"lock acquired","identifier":"whoami.mysmarthome.network"}
{"level":"debug","ts":1660598456.566291,"logger":"tls.obtain","msg":"trying issuer 1/2","issuer":"acme-v02.api.letsencrypt.org-directory"}
{"level":"info","ts":1660598456.56641,"logger":"tls.issuance.acme","msg":"waiting on internal rate limiter","identifiers":["whoami.mysmarthome.network"],"ca":"https://acme-v02.api.letsencrypt.org/directory","account":"admin@mysmarthome.network"}
{"level":"info","ts":1660598456.5664287,"logger":"tls.issuance.acme","msg":"done waiting on internal rate limiter","identifiers":["whoami.mysmarthome.network"],"ca":"https://acme-v02.api.letsencrypt.org/directory","account":"admin@mysmarthome.network"}
{"level":"debug","ts":1660598456.7121646,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"HEAD","url":"https://acme-v02.api.letsencrypt.org/acme/new-nonce","headers":{"User-Agent":["Caddy/2.5.2 CertMagic acmez (linux; amd64)"]},"response_headers":{"Cache-Control":["public, max-age=0, no-cache"],"Date":["Mon, 15 Aug 2022 21:20:56 GMT"],"Link":["<https://acme-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["0101KYTOnby2v52XACCm8O9v9Ipi6uOGjcAyZmvTSw1VJco"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":200}
{"level":"debug","ts":1660598456.9836736,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-v02.api.letsencrypt.org/acme/new-order","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.5.2 CertMagic acmez (linux; amd64)"]},"response_headers":{"Boulder-Requester":["679699397"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["351"],"Content-Type":["application/json"],"Date":["Mon, 15 Aug 2022 21:20:56 GMT"],"Link":["<https://acme-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Location":["https://acme-v02.api.letsencrypt.org/acme/order/679699397/116535151757"],"Replay-Nonce":["0101NEidqqeDyYYwKh_J8pEiG-mfJcvWfrW5E3GtfBqsX60"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":201}
{"level":"debug","ts":1660598457.0605226,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-v02.api.letsencrypt.org/acme/authz-v3/142455956147","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.5.2 CertMagic acmez (linux; amd64)"]},"response_headers":{"Boulder-Requester":["679699397"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["810"],"Content-Type":["application/json"],"Date":["Mon, 15 Aug 2022 21:20:57 GMT"],"Link":["<https://acme-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["0101w0qmgCt56aS_eDIFu6TswFZDgrr1ySCq0vfJSbRlEyI"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":200}
{"level":"debug","ts":1660598457.0606472,"logger":"tls.issuance.acme.acme_client","msg":"no solver configured","challenge_type":"http-01"}
{"level":"debug","ts":1660598457.0606794,"logger":"tls.issuance.acme.acme_client","msg":"no solver configured","challenge_type":"tls-alpn-01"}
{"level":"info","ts":1660598457.0607073,"logger":"tls.issuance.acme.acme_client","msg":"trying to solve challenge","identifier":"whoami.mysmarthome.network","challenge_type":"dns-01","ca":"https://acme-v02.api.letsencrypt.org/directory"}
{"level":"error","ts":1660598457.3811948,"logger":"tls.issuance.acme.acme_client","msg":"cleaning up solver","identifier":"whoami.mysmarthome.network","challenge_type":"dns-01","error":"no memory of presenting a DNS record for whoami.mysmarthome.network (probably OK if presenting failed)"}
{"level":"debug","ts":1660598457.4746213,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-v02.api.letsencrypt.org/acme/authz-v3/142455956147","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.5.2 CertMagic acmez (linux; amd64)"]},"response_headers":{"Boulder-Requester":["679699397"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["814"],"Content-Type":["application/json"],"Date":["Mon, 15 Aug 2022 21:20:57 GMT"],"Link":["<https://acme-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["0102TOg72nB-BV2abfdSoIHWGyEHv1o9TSdRIF43W-DWD2g"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":200}
{"level":"error","ts":1660598457.4747033,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"whoami.mysmarthome.network","issuer":"acme-v02.api.letsencrypt.org-directory","error":"[whoami.mysmarthome.network] solving challenges: presenting for challenge: adding temporary record for zone mysmarthome.network.: got error status: HTTP 400: [{Code:81057 Message:Record already exists.}] (order=https://acme-v02.api.letsencrypt.org/acme/order/679699397/116535151757) (ca=https://acme-v02.api.letsencrypt.org/directory)"}
{"level":"debug","ts":1660598457.4747162,"logger":"tls.obtain","msg":"trying issuer 2/2","issuer":"acme.zerossl.com-v2-DV90"}
{"level":"info","ts":1660598458.1617756,"logger":"tls.issuance.zerossl","msg":"generated EAB credentials","key_id":"vaiVcubR0CZOjWa_CEgFJw"}
{"level":"debug","ts":1660598461.9560862,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"GET","url":"https://acme.zerossl.com/v2/DV90","headers":{"User-Agent":["Caddy/2.5.2 CertMagic acmez (linux; amd64)"]},"response_headers":{"Access-Control-Allow-Origin":["*"],"Cache-Control":["max-age=-1"],"Content-Length":["645"],"Content-Type":["application/json"],"Date":["Mon, 15 Aug 2022 21:21:01 GMT"],"Server":["nginx"],"Strict-Transport-Security":["max-age=15552000"]},"status_code":200}
{"level":"debug","ts":1660598469.4432127,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"HEAD","url":"https://acme.zerossl.com/v2/DV90/newNonce","headers":{"User-Agent":["Caddy/2.5.2 CertMagic acmez (linux; amd64)"]},"response_headers":{"Access-Control-Allow-Origin":["*"],"Cache-Control":["max-age=-1"],"Content-Type":["application/octet-stream"],"Date":["Mon, 15 Aug 2022 21:21:09 GMT"],"Link":["<https://acme.zerossl.com/v2/DV90>;rel=\"index\""],"Replay-Nonce":["xQDV8V3-jWbdVHIa_j2nE4FI8WW4D6I3OS5CXqvbVpc"],"Server":["nginx"],"Strict-Transport-Security":["max-age=15552000"]},"status_code":200}
{"level":"debug","ts":1660598470.1008034,"logger":"tls.handshake","msg":"no matching certificates and no custom selection logic","identifier":"whoami.mysmarthome.network"}
{"level":"debug","ts":1660598470.1008255,"logger":"tls.handshake","msg":"no matching certificates and no custom selection logic","identifier":"*.mysmarthome.network"}
{"level":"debug","ts":1660598470.1008322,"logger":"tls.handshake","msg":"no matching certificates and no custom selection logic","identifier":"*.*.network"}
{"level":"debug","ts":1660598470.1008384,"logger":"tls.handshake","msg":"no matching certificates and no custom selection logic","identifier":"*.*.*"}
{"level":"debug","ts":1660598470.1008437,"logger":"tls.handshake","msg":"all external certificate managers yielded no certificates and no errors","sni":"whoami.mysmarthome.network"}
{"level":"debug","ts":1660598470.100851,"logger":"tls.handshake","msg":"no certificate matching TLS ClientHello","server_name":"whoami.mysmarthome.network","remote":"10.0.0.2:41134","identifier":"whoami.mysmarthome.network","cipher_suites":[43690,4865,4866,4867,49195,49199,49196,49200,52393,52392,49171,49172,156,157,47,53],"cert_cache_fill":0,"load_if_necessary":true,"obtain_if_necessary":true,"on_demand":false}
{"level":"debug","ts":1660598470.1009433,"logger":"http.stdlib","msg":"http: TLS handshake error from 10.0.0.2:41134: no certificate available for 'whoami.mysmarthome.network'"}
{"level":"debug","ts":1660598470.1029634,"logger":"tls.handshake","msg":"no matching certificates and no custom selection logic","identifier":"whoami.mysmarthome.network"}
{"level":"debug","ts":1660598470.1030004,"logger":"tls.handshake","msg":"no matching certificates and no custom selection logic","identifier":"*.mysmarthome.network"}
{"level":"debug","ts":1660598470.1030219,"logger":"tls.handshake","msg":"no matching certificates and no custom selection logic","identifier":"*.*.network"}
{"level":"debug","ts":1660598470.1030397,"logger":"tls.handshake","msg":"no matching certificates and no custom selection logic","identifier":"*.*.*"}
{"level":"debug","ts":1660598470.1030571,"logger":"tls.handshake","msg":"all external certificate managers yielded no certificates and no errors","sni":"whoami.mysmarthome.network"}
{"level":"debug","ts":1660598470.1030912,"logger":"tls.handshake","msg":"no certificate matching TLS ClientHello","server_name":"whoami.mysmarthome.network","remote":"10.0.0.2:41135","identifier":"whoami.mysmarthome.network","cipher_suites":[31354,4865,4866,4867,49195,49199,49196,49200,52393,52392,49171,49172,156,157,47,53],"cert_cache_fill":0,"load_if_necessary":true,"obtain_if_necessary":true,"on_demand":false}
{"level":"debug","ts":1660598470.1031673,"logger":"http.stdlib","msg":"http: TLS handshake error from 10.0.0.2:41135: no certificate available for 'whoami.mysmarthome.network'"}
{"level":"debug","ts":1660598475.8276005,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme.zerossl.com/v2/DV90/newAccount","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.5.2 CertMagic acmez (linux; amd64)"]},"response_headers":{"Access-Control-Allow-Origin":["*"],"Cache-Control":["max-age=0, no-cache, no-store","max-age=-1"],"Content-Length":["587"],"Content-Type":["application/json"],"Date":["Mon, 15 Aug 2022 21:21:15 GMT"],"Link":["<https://acme.zerossl.com/v2/DV90>;rel=\"index\""],"Location":["https://acme.zerossl.com/v2/DV90/account/vaiVcubR0CZOjWa_CEgFJw"],"Replay-Nonce":["soWJaWoVBKVB3OIZ1KYxK4u0wlkXl1QzIluTvd2PCUo"],"Server":["nginx"],"Status":[""],"Strict-Transport-Security":["max-age=15552000"]},"status_code":201}
{"level":"info","ts":1660598475.8279445,"logger":"tls.issuance.acme","msg":"waiting on internal rate limiter","identifiers":["whoami.mysmarthome.network"],"ca":"https://acme.zerossl.com/v2/DV90","account":"admin@mysmarthome.network"}
{"level":"info","ts":1660598475.8279595,"logger":"tls.issuance.acme","msg":"done waiting on internal rate limiter","identifiers":["whoami.mysmarthome.network"],"ca":"https://acme.zerossl.com/v2/DV90","account":"admin@mysmarthome.network"}
{"level":"debug","ts":1660598479.075288,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme.zerossl.com/v2/DV90/newOrder","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.5.2 CertMagic acmez (linux; amd64)"]},"response_headers":{"Access-Control-Allow-Origin":["*"],"Cache-Control":["max-age=0, no-cache, no-store","max-age=-1"],"Content-Length":["288"],"Content-Type":["application/json"],"Date":["Mon, 15 Aug 2022 21:21:19 GMT"],"Location":["https://acme.zerossl.com/v2/DV90/order/a3VnjdjYTLrJttV4SOF2yA"],"Replay-Nonce":["RQQ_vy4fC5URFKiBn2LH2_NIsrQ--uDSL5QCCd-7Lw8"],"Server":["nginx"],"Status":[""],"Strict-Transport-Security":["max-age=15552000"]},"status_code":201}
{"level":"debug","ts":1660598482.102841,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme.zerossl.com/v2/DV90/authz/VEDdb7XsfTKhreSi2deyvw","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.5.2 CertMagic acmez (linux; amd64)"]},"response_headers":{"Access-Control-Allow-Origin":["*"],"Cache-Control":["max-age=-1"],"Content-Length":["456"],"Content-Type":["application/json"],"Date":["Mon, 15 Aug 2022 21:21:22 GMT"],"Link":["<https://acme.zerossl.com/v2/DV90>;rel=\"index\""],"Replay-Nonce":["BKr72iGM3hWhcuus93gI8GhfySJXgwAzkAcFcZgD9U4"],"Retry-After":["5"],"Server":["nginx"],"Strict-Transport-Security":["max-age=15552000"]},"status_code":200}
{"level":"debug","ts":1660598482.1029394,"logger":"tls.issuance.acme.acme_client","msg":"no solver configured","challenge_type":"http-01"}
{"level":"info","ts":1660598482.1029649,"logger":"tls.issuance.acme.acme_client","msg":"trying to solve challenge","identifier":"whoami.mysmarthome.network","challenge_type":"dns-01","ca":"https://acme.zerossl.com/v2/DV90"}
{"level":"debug","ts":1660598482.6167088,"logger":"tls.handshake","msg":"no matching certificates and no custom selection logic","identifier":"whoami.mysmarthome.network"}
{"level":"debug","ts":1660598482.616745,"logger":"tls.handshake","msg":"no matching certificates and no custom selection logic","identifier":"*.mysmarthome.network"}
{"level":"debug","ts":1660598482.616763,"logger":"tls.handshake","msg":"no matching certificates and no custom selection logic","identifier":"*.*.network"}
{"level":"debug","ts":1660598482.6167798,"logger":"tls.handshake","msg":"no matching certificates and no custom selection logic","identifier":"*.*.*"}
{"level":"debug","ts":1660598482.616806,"logger":"tls.handshake","msg":"all external certificate managers yielded no certificates and no errors","sni":"whoami.mysmarthome.network"}
{"level":"debug","ts":1660598482.6168134,"logger":"tls.handshake","msg":"no certificate matching TLS ClientHello","server_name":"whoami.mysmarthome.network","remote":"10.0.0.2:41205","identifier":"whoami.mysmarthome.network","cipher_suites":[31354,4865,4866,4867,49195,49199,49196,49200,52393,52392,49171,49172,156,157,47,53],"cert_cache_fill":0,"load_if_necessary":true,"obtain_if_necessary":true,"on_demand":false}
{"level":"debug","ts":1660598482.6168528,"logger":"http.stdlib","msg":"http: TLS handshake error from 10.0.0.2:41205: no certificate available for 'whoami.mysmarthome.network'"}
{"level":"debug","ts":1660598482.618903,"logger":"tls.handshake","msg":"no matching certificates and no custom selection logic","identifier":"whoami.mysmarthome.network"}
{"level":"debug","ts":1660598482.618937,"logger":"tls.handshake","msg":"no matching certificates and no custom selection logic","identifier":"*.mysmarthome.network"}
{"level":"debug","ts":1660598482.6189442,"logger":"tls.handshake","msg":"no matching certificates and no custom selection logic","identifier":"*.*.network"}
{"level":"debug","ts":1660598482.6189494,"logger":"tls.handshake","msg":"no matching certificates and no custom selection logic","identifier":"*.*.*"}
{"level":"debug","ts":1660598482.6189544,"logger":"tls.handshake","msg":"all external certificate managers yielded no certificates and no errors","sni":"whoami.mysmarthome.network"}
{"level":"debug","ts":1660598482.6189609,"logger":"tls.handshake","msg":"no certificate matching TLS ClientHello","server_name":"whoami.mysmarthome.network","remote":"10.0.0.2:41206","identifier":"whoami.mysmarthome.network","cipher_suites":[23130,4865,4866,4867,49195,49199,49196,49200,52393,52392,49171,49172,156,157,47,53],"cert_cache_fill":0,"load_if_necessary":true,"obtain_if_necessary":true,"on_demand":false}
{"level":"debug","ts":1660598482.6189961,"logger":"http.stdlib","msg":"http: TLS handshake error from 10.0.0.2:41206: no certificate available for 'whoami.mysmarthome.network'"}
{"level":"debug","ts":1660598485.541939,"logger":"docker-proxy","msg":"Skipping default Caddyfile because no path is set"}
{"level":"debug","ts":1660598485.5433612,"logger":"docker-proxy","msg":"Swarm service","service":"caddy_caddy"}
{"level":"debug","ts":1660598485.5434368,"logger":"docker-proxy","msg":"Swarm service","service":"whoami_whoami"}
{"level":"debug","ts":1660598504.33116,"logger":"docker-proxy","msg":"Skipping default Caddyfile because no path is set"}
{"level":"debug","ts":1660598504.3325257,"logger":"docker-proxy","msg":"Swarm service","service":"caddy_caddy"}
{"level":"info","ts":1660598504.3326678,"logger":"docker-proxy","msg":"New Caddyfile","caddyfile":"{\n\tacme_dns cloudflare {env.CF_API_TOKEN}\n\tdebug\n\temail {env.EMAIL}\n\tlog {\n\t\toutput file /var/log/caddy/access.log\n\t}\n}\n"}
{"level":"info","ts":1660598504.3327222,"logger":"docker-proxy","msg":"New Config JSON","json":"{\"logging\":{\"logs\":{\"default\":{\"writer\":{\"filename\":\"/var/log/caddy/access.log\",\"output\":\"file\"},\"level\":\"DEBUG\"}}}}"}
{"level":"info","ts":1660598504.3327355,"logger":"docker-proxy","msg":"Sending configuration to","server":"localhost"}
{"level":"info","ts":1660598504.3330293,"logger":"admin.api","msg":"received request","method":"POST","host":"localhost:2019","uri":"/load","remote_ip":"127.0.0.1","remote_port":"60444","headers":{"Accept-Encoding":["gzip"],"Content-Length":["156"],"Content-Type":["application/json"],"User-Agent":["Go-http-client/1.1"]}}
{"level":"info","ts":1660598504.3331783,"logger":"admin","msg":"admin endpoint started","address":"tcp/localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
{"level":"info","ts":1660598504.3354015,"logger":"tls.cache.maintenance","msg":"stopped background certificate maintenance","cache":"0xc00079e620"}
{"level":"info","ts":1660598504.3354926,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
{"level":"info","ts":1660598504.3355007,"logger":"admin.api","msg":"load complete"}
{"level":"error","ts":1660598504.3355718,"logger":"tls.issuance.acme.acme_client","msg":"cleaning up solver","identifier":"whoami.mysmarthome.network","challenge_type":"dns-01","error":"deleting temporary record for zone mysmarthome.network.: Delete \"https://api.cloudflare.com/client/v4/zones/4c38d8a7f0598c0cc42ab69d65eeb822/dns_records/d850a62731e1a775173bcf71be609ebb\": context canceled"}
{"level":"warn","ts":1660598504.335651,"logger":"tls.issuance.acme.acme_client","msg":"HTTP request failed; retrying","url":"https://acme.zerossl.com/v2/DV90/authz/VEDdb7XsfTKhreSi2deyvw","error":"performing request: Post \"https://acme.zerossl.com/v2/DV90/authz/VEDdb7XsfTKhreSi2deyvw\": context canceled"}
{"level":"error","ts":1660598504.3356633,"logger":"tls.issuance.acme.acme_client","msg":"deactivating authorization","identifier":"whoami.mysmarthome.network","authz":"https://acme.zerossl.com/v2/DV90/authz/VEDdb7XsfTKhreSi2deyvw","error":"attempt 1: https://acme.zerossl.com/v2/DV90/authz/VEDdb7XsfTKhreSi2deyvw: context canceled"}
{"level":"info","ts":1660598504.3357298,"logger":"docker-proxy","msg":"Successfully configured","server":"localhost"}
{"level":"error","ts":1660598504.3357797,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"whoami.mysmarthome.network","issuer":"acme.zerossl.com-v2-DV90","error":"[whoami.mysmarthome.network] solving challenges: waiting for solver certmagic.solverWrapper to be ready: context canceled (order=https://acme.zerossl.com/v2/DV90/order/a3VnjdjYTLrJttV4SOF2yA) (ca=https://acme.zerossl.com/v2/DV90)"}
{"level":"info","ts":1660598504.3357904,"logger":"tls.obtain","msg":"releasing lock","identifier":"whoami.mysmarthome.network"}
{"level":"error","ts":1660598504.335832,"logger":"tls","msg":"job failed","error":"whoami.mysmarthome.network: obtaining certificate: [whoami.mysmarthome.network] Obtain: [whoami.mysmarthome.network] solving challenges: waiting for solver certmagic.solverWrapper to be ready: context canceled (order=https://acme.zerossl.com/v2/DV90/order/a3VnjdjYTLrJttV4SOF2yA) (ca=https://acme.zerossl.com/v2/DV90)"}
{"level":"info","ts":1660598504.3367987,"logger":"admin","msg":"stopped previous server","address":"tcp/localhost:2019"}
{"level":"debug","ts":1660598507.0168848,"logger":"docker-proxy","msg":"Skipping default Caddyfile because no path is set"}
{"level":"info","ts":1660598507.0184,"logger":"docker-proxy","msg":"New Caddyfile","caddyfile":"# Empty caddyfile"}
{"level":"warn","ts":1660598507.0185142,"logger":"docker-proxy","msg":"Caddyfile to json warning","warn":"[Caddyfile:1: Caddyfile input is not formatted; run the 'caddy fmt' command to fix inconsistencies]"}
{"level":"info","ts":1660598507.0185258,"logger":"docker-proxy","msg":"New Config JSON","json":"{}"}
{"level":"info","ts":1660598507.018539,"logger":"docker-proxy","msg":"Sending configuration to","server":"localhost"}
{"level":"info","ts":1660598507.0188575,"logger":"admin.api","msg":"received request","method":"POST","host":"localhost:2019","uri":"/load","remote_ip":"127.0.0.1","remote_port":"60460","headers":{"Accept-Encoding":["gzip"],"Content-Length":["41"],"Content-Type":["application/json"],"User-Agent":["Go-http-client/1.1"]}}
{"level":"info","ts":1660598507.0189083,"msg":"redirected default logger","from":"/var/log/caddy/access.log","to":"stderr"}
{"level":"info","ts":1660598507.0192502,"logger":"docker-proxy","msg":"Successfully configured","server":"localhost"}

francislavoie · August 15, 2022, 11:17pm

Well, this seems like the key. Try clearing out your DNS TXT records, clearing out Caddy’s storage (/data dir) and restarting Caddy, to get a clean slate. It might be stuck in a bad state because of DNS.

The attempts with ZeroSSL are failing too but it seems like for a different reason (ZeroSSL tends to be flakier, sometimes it has very high latency for some reason), that can probably be ignored.

Let’s see what your logs look like after that.

simsrw73 · August 16, 2022, 12:30am

Strange. It worked exactly once. Tried to recreate it a dozen times. And it won’t work. No idea why it worked the once. Cleared dns, removed & recreated directories, restarted containers. No Cloudflare proxy on anything. I don’t have a ZeroSSL account and haven’t tried configuring for that yet.

Full logs: https://pastebin.com/Kdv9WWpt

francislavoie · August 16, 2022, 12:39am

There’s a lot of context cancelled stuff but that’s from the config being reloaded, and when that happens, anything currently in-progress gets cancelled so that the new config can take effect (don’t want to keep doing things that the old config wanted to do because it could now be wrong, knowing there’s a new config).

{"level":"info","ts":1660609070.7409632,"logger":"tls.issuance.acme.acme_client","msg":"successfully downloaded available certificate chains","count":2,"first_url":"https://acme-v02.api.letsencrypt.org/acme/cert/04fa2cb56e03a77cb69b61017867d47068e8"}
{"level":"info","ts":1660609070.7412431,"logger":"tls.obtain","msg":"certificate obtained successfully","identifier":"whoami.mysmarthome.network"}

It looks like it worked, eventually. I can hit your domain now and I get a response.

simsrw73 · August 16, 2022, 1:34am

Oh, no. It works. I suppose I must have been a tad impatient. Before switching to this caddy-docker-proxy plugin, it was much quicker and I assumed it should be similar. Is it configurable? It sounds like it is itself getting impatient and retrying before it has had a chance to take affect?

BTW, extremely grateful for your assistance and patience. I’ve dabbled with some other proxies before Caddy. Eg Traefik was terribly complicated in this scenario and I posted on 7-8 forums, discords, reddits, without getting any help. Caddy is much more sensible, simple and it has been a pleasure reading and posting here. Thank you for that.

francislavoie · August 16, 2022, 1:48am

I think you just got unlucky, frankly. But also some “frequent” docker container up/downs may have killed off some of the cert issuance attempts that would have otherwise succeeded. Typically issuance takes just a few seconds (like 3-10s) and anything longer is very unusual.

From your logs, no, I don’t think that was the case at all. Retries were all after failing the previous try.

simsrw73 · August 17, 2022, 8:17pm

I hate to reopen this, but I am still having troubles. Something is failing with the caddy-docker-proxy plugin that does not fail running with a static config. It’s not taking seconds, it’s taking many minutes and certs are not registered or downloaded at all. The day I posted here last, I left Caddy and whoami running continually over night. The next morning I tried launching wikijs. Just added the two lines of config to the docker-compose file I previously used with Caddy without the plugin and launched it, but after at least 20 mins or so it didn’t come up. While still running, I looked on cloudflare and saw the dns entries, deleted them, and then it came up, but that was probably luck because I can’t repeat that scenario. Launched a cat demo container, and like whoami, it took maybe 5-10 mins or so before I could access it on my network. It also left the dns entries on cloudflare. Sometimes domains come up and sometimes they never do. Certs that have already be registered will sometimes re-download and sometimes not.

I’ve tried recreating my Cloudflare access token and that didn’t change anything.

I created an account on ZeroSSL. Haven’t really looked to see what other config is needed, but I noticed that a couple of domains did register there, but when run again it won’t download the certs nor create new ones.

I ran these same containers manually with Caddy before using the plugin and they came right up. And just ran them again manually without the plugin and they pull the cert and are up as quick as I can type in the url.

Maybe I should post something on the github for caddy-docker-proxy? Just following up here in case you might see something else I’m missing here. Otherwise, just close this and I’ll move my query over to github for the plugin.

Full logs:
https://zerobin.net/?502c321b3dc65965#hzMBiSM7ziWinwD6kxwfENvV918/1NoY734HNq/xARI=

matt · August 17, 2022, 9:21pm

What’s your Caddy config? Do you only have the one domain whoami.mysmarthome.network?

Also please don’t redact your domains, it makes it exceptionally difficult for us to help you, and in many cases people make errors when redacting making the logs and other reports unreliable.

simsrw73 · August 17, 2022, 10:28pm

My domain is mysmarthome.network and I’m trying to create the subdomains of whoami.mysmarthome.network, wiki.mysmarthome.network, etc. Sorry for redacting that in the earlier posts: in other forums, reddits, etc. I had read many posts saying it’s bad to post your domain for everyone to see.

Here are my configs:

Dockerfile

ARG CADDY_VERSION=2.5.2
FROM caddy:${CADDY_VERSION}-builder AS builder

RUN xcaddy build \
    --with github.com/lucaslorentz/caddy-docker-proxy/plugin \
    --with github.com/caddy-dns/cloudflare

FROM caddy:${CADDY_VERSION}-alpine

COPY --from=builder /usr/bin/caddy /usr/bin/caddy

CMD ["caddy", "docker-proxy"]

Built with: docker build -t caddy-docker-proxy--mysmarthome_network .

docker-compose.yml for caddy

version: "3.7"

services:
  caddy:
    image: caddy-docker-proxy--mysmarthome_network:latest
    ports:
      - "80:80"
      - "443:443"
    networks:
      - proxy
    env_file: .env
    environment:
      - CADDY_INGRESS_NETWORKS=proxy
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - /docker-services/caddy/data:/data
      - /docker-services/caddy/config:/config
      - /docker-services/caddy/logs:/var/log/caddy
    deploy:
      labels:
        caddy.debug:
        caddy.log.output: file /var/log/caddy/caddy.log
        caddy.acme_dns: "cloudflare {env.CF_API_TOKEN}"
        caddy.email: "{env.EMAIL}"
      placement:
        constraints:
          - node.role == manager
      replicas: 1
      resources:
        reservations:
          cpus: "0.1"
          memory: 200M
      restart_policy:
        condition: any

networks:
  proxy:
    external: true

Deployed with: docker stack deploy -c docker-compose.yml caddy

A couple test containers are:

docker-compose.yml for whoami

version: "3.7"

services:

  whoami:
    image: jwilder/whoami
    networks:
      - proxy
    deploy:
      labels:
        caddy: whoami.mysmarthome.network
        caddy.reverse_proxy: "{{upstreams 8000}}"

networks:
  proxy:
    external: true

docker-compose.yml for wikijs

version: "3.7"

services:
  wikijs:
    image: lscr.io/linuxserver/wikijs:latest
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=America/New_York
    volumes:
      - /docker-services/wikijs/config:/config
      - /docker-services/wikijs/data:/data
    ports:
      - 3000:3000
    networks:
      - proxy
    deploy:
      mode: replicated
      replicas: 1
      placement:
        constraints: [node.role == manager]
      labels:
         caddy: wiki.mysmarthome.network
         caddy.reverse_proxy: "{{upstreams 3000}}"

networks:
  proxy:
    external: true

matt · August 17, 2022, 10:49pm

Thanks to @francislavoie for also pointing out to me that caddy-docker-proxy spits out the config to the logs

Appreciate all the useful info.

Do you have any wildcard domains you’re using / getting certs for? It doesn’t seem like it from this, but I want to check because I recently fixed a bug where getting certs for a wildcard domain and a subdomain at the same time could cause a timeout: go.mod: Upgrade CertMagic and acmez · caddyserver/caddy@63c7720 · GitHub

I’d be curious if you upgraded to use the latest commits on master if the error still occurs.

But I don’t think it would actually affect your use case. (Btw, for any more subdomains I’d just recommend using a wildcard cert, since Let’s Encrypt rate limits subdomains quite heavily.)

matt · August 17, 2022, 11:08pm

Oh, I see the problem now.

deleting temporary record for zone mysmarthome.network.: Delete "https://api.cloudflare.com/client/v4/zones/4c38d8a7f0598c0cc42ab69d65eeb822/dns_records/8a21f345e0b0bd0d4fd7136d8cf185e3\": context canceled

Basically, the config is unloaded and it tries to clean up the challenge but we use the same context that was cancelled by the config reload, so the underlying HTTP client aborts the request to clean up the DNS record.

I mean, this should be the right thing to do to avoid leaking resources, in theory. But maybe we need to use a context that isn’t tied to the config and use one with a timeout instead. I’ll look into this. Francis and I are chatting about this in Slack.

simsrw73 · August 18, 2022, 1:23am

Thanks for looking in to this. If it matters about the certs I’ve got, I guess I got a lot, incl wild cards.

https://search.censys.io/certificates?q=mysmarthome.network
https://crt.sh/?q=mysmarthome.network

Many of those were created while trying different proxies: npm, traefik, etc. Esp the wildcards. I haven’t learned about wildcards in Caddy yet. And I don’t know enough about certs to understand why there are multiple copies issued instead of fetching the existing ones??? I’ve still got a ton to learn about all the under the hood stuff, while right now I’m just trying to get something up and running.

matt · August 18, 2022, 1:41am

I’m not sure about caddy-docker-proxy and if that affects wildcards, but if I were you I’d build Caddy from the latest master because it has a likely-relevant patch, since it seems like you’d benefit if there are indeed wildcard certs in play.

francislavoie · August 18, 2022, 2:19am

To do that, you can just add another parameter after xcaddy build to specify the git tag/branch/ref, e.g. xcaddy build master --with <etc>. You can probably just put master there for now, or better, get the commit hash of the current master and use that for now.