Caddy reverse proxy not reachable by subdomain

Help
1

1. Caddy version (caddy version):

2.4.6

2. How I run Caddy:

a. System environment:

Ubuntu Server 20.04.3 LTS x86_64
Docker version 20.10.12, build e91ed57
docker-compose version 1.29.2, build 5becea4c
DNS managed by Namecheap with a simple A record @ → IPv4

b. Command:

Dockerfile
ARG CADDY_VERSION=2.4.6
FROM caddy:${CADDY_VERSION}-builder AS builder

RUN xcaddy build \
    --with github.com/lucaslorentz/caddy-docker-proxy/plugin \
    --with github.com/caddy-dns/namecheap

FROM caddy:${CADDY_VERSION}

COPY --from=builder /usr/bin/caddy /usr/bin/caddy

CMD ["caddy", "docker-proxy"]
docker-compose.yml
version: '3.7'

services:
  caddy:
    build: .
    container_name: caddy
    ports:
      - 80:80
      - 443:443
    environment:
      - CADDY_INGRESS_NETWORKS=caddy
      - ACME_AGREE=true
    networks:
      - caddy
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - caddy_data:/data
    restart: unless-stopped

networks:
  caddy:
    external: true

volumes:
  caddy_data: {}

c. Service/unit/compose file:

docker-compose.yml (for a simple whoami test image)
version: '3.7'
services:
  whoami:
    image: jwilder/whoami
    networks:
      - caddy
    labels:
      caddy: hello.waloschek.me
      caddy.reverse_proxy: "{{upstreams 8000}}"
      caddy.tls.dns: "namecheap"
      caddy.tls.dns.user: "${NAMECHEAP_USER}"
      caddy.tls.dns.api_key: "${NAMECHEAP_API_KEY}"
      
networks:
  caddy:
    external: true

d. My complete Caddyfile or JSON config:

Not really applicable, gets build internally by caddy-docker-proxy, but here is the auto generated Caddyfile:

hello.waloschek.me {
    reverse_proxy 172.19.0.2:8000
    tls {
        dns namecheap {
            api_key ......its_a_secret.......
            user ....same....
        }
    }
}

3. The problem I’m having:

I am not able to even resolve the host hello.waloschek.me let alone get a HTTP GET response via https://hello.waloschek.me using curl:

$ curl -v https://hello.waloschek.me/
* Could not resolve host: hello.waloschek.me
* Closing connection 0
curl: (6) Could not resolve host: hello.waloschek.me

4. Error messages and/or full log output:

{"level":"info","ts":1641241596.4617794,"logger":"docker-proxy","msg":"Running caddy proxy server"}
{"level":"info","ts":1641241596.4627314,"logger":"admin","msg":"admin endpoint started","address":"tcp/localhost:2019","enforce_origin":false,"origins":["127.0.0.1:2019","localhost:2019","[::1]:2019"]}
{"level":"info","ts":1641241596.46288,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
{"level":"info","ts":1641241596.462898,"logger":"docker-proxy","msg":"Running caddy proxy controller"}
{"level":"info","ts":1641241596.4635024,"logger":"docker-proxy","msg":"Start","CaddyfilePath":"","LabelPrefix":"caddy","PollingInterval":30,"ProcessCaddyfile":true,"ProxyServiceTasks":true,"IngressNetworks":"[caddy]","DockerSockets":[""],"DockerCertsPath":[""],"DockerAPIsVersion":[""]}
{"level":"info","ts":1641241596.4641638,"logger":"docker-proxy","msg":"Connecting to docker events","DockerSocket":""}
{"level":"info","ts":1641241596.4645758,"logger":"docker-proxy","msg":"IngressNetworksMap","ingres":"map[572518288bd77fcd99fc3522669f167c883d1abd1677ed00da1856db6c5962c4:true]"}
{"level":"info","ts":1641241596.4733803,"logger":"docker-proxy","msg":"Swarm is available","new":false}
{"level":"info","ts":1641241596.473398,"logger":"docker-proxy","msg":"Skipping default Caddyfile because no path is set"}
{"level":"info","ts":1641241596.4734013,"logger":"docker-proxy","msg":"Skipping swarm config caddyfiles because swarm is not available"}
{"level":"info","ts":1641241596.474661,"logger":"docker-proxy","msg":"Skipping swarm services because swarm is not available"}
{"level":"info","ts":1641241596.4755554,"logger":"docker-proxy","msg":"New Caddyfile","caddyfile":"hello.waloschek.me {\n\treverse_proxy 172.19.0.2:8000\n\ttls {\n\t\tdns namecheap {\n\t\t\tapi_key ......its_a_secret.......\n\t\t\tuser ....same....\n\t\t}\n\t}\n}\n"}
{"level":"info","ts":1641241596.4762275,"logger":"docker-proxy","msg":"New Config JSON","json":"{\"apps\":{\"http\":{\"servers\":{\"srv0\":{\"listen\":[\":443\"],\"routes\":[{\"match\":[{\"host\":[\"hello.waloschek.me\"]}],\"handle\":[{\"handler\":\"subroute\",\"routes\":[{\"handle\":[{\"handler\":\"reverse_proxy\",\"upstreams\":[{\"dial\":\"172.19.0.2:8000\"}]}]}]}],\"terminal\":true}]}}},\"tls\":{\"automation\":{\"policies\":[{\"subjects\":[\"hello.waloschek.me\"],\"issuers\":[{\"challenges\":{\"dns\":{\"provider\":{\"api_key\":\"......its_a_secret.......\",\"name\":\"namecheap\",\"user\":\"....same....\"}}},\"module\":\"acme\"},{\"challenges\":{\"dns\":{\"provider\":{\"api_key\":\"......its_a_secret.......\",\"name\":\"namecheap\",\"user\":\"....same....\"}}},\"module\":\"zerossl\"}]}]}}}}"}
{"level":"info","ts":1641241596.4763904,"logger":"docker-proxy","msg":"Sending configuration to","server":"localhost"}
{"level":"info","ts":1641241596.477473,"logger":"admin.api","msg":"received request","method":"POST","host":"localhost:2019","uri":"/load","remote_addr":"127.0.0.1:43136","headers":{"Accept-Encoding":["gzip"],"Content-Length":["657"],"Content-Type":["application/json"],"User-Agent":["Go-http-client/1.1"]}}
{"level":"info","ts":1641241596.4778223,"logger":"admin","msg":"admin endpoint started","address":"tcp/localhost:2019","enforce_origin":false,"origins":["localhost:2019","[::1]:2019","127.0.0.1:2019"]}
{"level":"info","ts":1641241596.4779835,"logger":"http","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443}
{"level":"info","ts":1641241596.4779944,"logger":"http","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
{"level":"info","ts":1641241596.4779925,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc000444690"}
{"level":"info","ts":1641241596.4782763,"logger":"tls","msg":"cleaning storage unit","description":"FileStorage:/data/caddy"}
{"level":"info","ts":1641241596.4783366,"logger":"http","msg":"enabling automatic TLS certificate management","domains":["hello.waloschek.me"]}
{"level":"info","ts":1641241596.4787734,"logger":"tls","msg":"finished cleaning storage units"}
{"level":"info","ts":1641241596.481878,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
{"level":"info","ts":1641241596.481893,"logger":"admin.api","msg":"load complete"}
{"level":"info","ts":1641241596.4820142,"logger":"docker-proxy","msg":"Successfully configured","server":"localhost"}
{"level":"info","ts":1641241596.4872177,"logger":"admin","msg":"stopped previous server","address":"tcp/localhost:2019"}
{"level":"info","ts":1641241626.4836822,"logger":"docker-proxy","msg":"Skipping default Caddyfile because no path is set"}
{"level":"info","ts":1641241626.4837227,"logger":"docker-proxy","msg":"Skipping swarm config caddyfiles because swarm is not available"}
{"level":"info","ts":1641241626.4867892,"logger":"docker-proxy","msg":"Skipping swarm services because swarm is not available"}

5. What I already tried:

Accessing the whoami container directly by its docker IP works without any problems:

$ curl -v 172.19.0.2:8000
*   Trying 172.19.0.2:8000...
* TCP_NODELAY set
* Connected to 172.19.0.2 (172.19.0.2) port 8000 (#0)
> GET / HTTP/1.1
> Host: 172.19.0.2:8000
> User-Agent: curl/7.68.0
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Mon, 03 Jan 2022 20:32:11 GMT
< Content-Length: 17
< Content-Type: text/plain; charset=utf-8
< 
I'm 90faf37f8afd
* Connection #0 to host 172.19.0.2 left intact

Also, exposing whoami:8000 on http://waloschek.me:8000 with a simple docker run [...] works as well.

The log (see above) shows no signs of problems whatsoever. I’m at loss.

6. Links to relevant resources:

2

You don’t need this anymore, this was only necessary for Caddy v1. You can remove it.

Caddy will not configure DNS A records for you, that’s your responsibility. Caddy only creates TXT records used by ACME issuers to solve the ACME DNS challenge.

Make sure your DNS is configured to point that subdomain to your server’s IP address.

1 Like
3

Thank you very much for your answer, francislavoie! I never had to deal with DNS before so I was under the impression, that a single @ → IP would be sufficient. Is there some sort of PlugIn for Caddy that might automate generating A Records for each subdomain for some DNS providers?

4

Yes, there’s GitHub - mholt/caddy-dynamicdns: Caddy app that keeps your DNS records (A/AAAA) pointed at itself., but it’s really meant for situations where your server’s IP address may change because your ISP didn’t provision you with a static IP.

In this case, you probably want to just make a * DNS entry that points to your server’s IP address. @ is for the apex domain (no subdomain) and * is for all subdomains.

2 Likes
5

Thank you again. Adding a catch-all A record solved the problem, but only for a couple of hours. I added another service to the Caddyfile like this:

herten.waloschek.me {
    reverse_proxy 172.19.0.6:80
    tls {
        dns namecheap {
            api_key ......its_a_secret.......
            user ....same....
        }
    }
}

It worked for a couple of hours, but now nslookup herten.waloschek.me returns nothing:

Server:		127.0.0.53
Address:	127.0.0.53#53

Non-authoritative answer:
*** Can't find herten.waloschek.me: No answer

Any other subdomain (literally any) can be found without issues.

The only thing I can find in the logs is this:

{"level":"error","ts":1641700604.2080686,"logger":"tls","msg":"tls-alpn challenge","server_name":"","error":"no information found to solve challenge for identifier: "}

Is this a problem with my domain provider or did I do something wrong?

EDIT:
It happened to another subdomain (mon.waloschek.me) which seems now “burned” as in I cannot use it anymore.

1 Like
6

Hmm, something’s weird here. There’s missing the server_name value, and there’s no value at the end of the error message, there should be something after the colon.

Your DNS records don’t seem properly propagated: https://dnschecker.org/#A/hello.waloschek.me

This is definitely a problem with DNS.

2 Likes
7

Thank you, especially for the helpful dns checker link. I moved from Namecheap DNS to Cloudflare and all my problems are gone. Someone once said “It’s always the DNS.” :wink:

2 Likes
8

This topic was automatically closed after 30 days. New replies are no longer allowed.