Caddy doesn't bind to ipv6

1. Caddy version (v2.4.0):

2. How I run Caddy:

I run caddy using a simple Dockerfile:

➜  fly-proxy git:(main) cat Dockerfile 
FROM caddy:2.4.0-alpine
COPY ./Caddyfile /etc/caddy/Caddyfile%

a. System environment:

On fly.io which takes Docker images and runs them as firecracker micro VMs.

b. Command:

The default entrypoint of the docker image is the cmd that is running caddy.`

c. Service/unit/compose file:

d. My complete Caddyfile or JSON config:

{   
    debug
    
    auto_https off
    
    admin 0.0.0.0:2019
}

:80 {
    log {
       level DEBUG
       output stderr
    }

    respond /healthz "Im healthy!" 200
}

http://{$DOMAIN}, http://www.{$DOMAIN} {
    @http header X-Forwarded-Proto http

    redir @http https://{http.request.host}{uri} permanent

    reverse_proxy paypack-landing.internal:80
}

http://payments.{$DOMAIN}, http://payment.{$DOMAIN} {
    @http header X-Forwarded-Proto http

    redir @http https://{http.request.host}{uri} permanent

    reverse_proxy payments.internal:8080
}

http://wages.{$DOMAIN} {
    @http header X-Forwarded-Proto http

    redir @http https://{http.request.host}{uri} permanent
    
    reverse_proxy wages.internal:8080
}

3. The problem I’m having:

My host advertises both IPV4 and IPV6 Caddy doesn’t seem to be binding to the IPV6 address:

 $ curl -vL http://paypack.rw/
*   Trying 213.188.209.3...
* TCP_NODELAY set
* Connected to paypack.rw (213.188.209.3) port 80 (#0)
> GET / HTTP/1.1
> Host: paypack.rw
> User-Agent: curl/7.58.0
> Accept: */*
> 
< HTTP/1.1 301 Moved Permanently
< location: https://paypack.rw/
< server: Fly/b365a3d (2021-05-13)
< date: Sun, 23 May 2021 09:31:38 GMT
< via: 1.1 fly.io
< fly-request-id: 01F6C91M50BSDZN9ZDYSKPA86M
< content-length: 0
< 
* Connection #0 to host paypack.rw left intact
* Issue another request to this URL: 'https://paypack.rw/'
*   Trying 213.188.209.3...
* TCP_NODELAY set
*   Trying 2a09:8280:1:de13:15:1d4c:cffe:d09b...
* TCP_NODELAY set
* Immediate connect fail for 2a09:8280:1:de13:15:1d4c:cffe:d09b: Network is unreachable
*   Trying 2a09:8280:1:de13:15:1d4c:cffe:d09b...
* TCP_NODELAY set
* Immediate connect fail for 2a09:8280:1:de13:15:1d4c:cffe:d09b: Network is unreachable
*   Trying 2a09:8280:1:de13:15:1d4c:cffe:d09b...
* TCP_NODELAY set
* Immediate connect fail for 2a09:8280:1:de13:15:1d4c:cffe:d09b: Network is unreachable
*   Trying 2a09:8280:1:de13:15:1d4c:cffe:d09b...
* TCP_NODELAY set
* Immediate connect fail for 2a09:8280:1:de13:15:1d4c:cffe:d09b: Network is unreachable
* Connected to paypack.rw (213.188.209.3) port 443 (#1)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:

Testing IPV4:

$ nc -vz 213.188.209.3 80                      
Connection to 213.188.209.3 80 port [tcp/http] succeeded!

Testing IPV6:

$ nc -vz 2a09:8280:1:de13:15:1d4c:cffe:d09b 80
nc: connect to 2a09:8280:1:de13:15:1d4c:cffe:d09b port 80 (tcp) failed: Network is unreachable

4. Error messages and/or full log output:

5. What I already tried:

I tried to explicitly bind both 0.0.0.0 and :: in each site block including :80 but the logs kept saying 0.0.0.0:80 was already in use:

2021-05-23T09:22:08Z [info] Running: `caddy run --config /etc/caddy/Caddyfile --adapter caddyfile` as root
2021-05-23T09:22:08Z [info] 2021/05/23 09:22:08 listening on [fdaa:0:1af2:a7b:a9a:de66:18ac:2]:22 (DNS: [fdaa::3]:53)
2021-05-23T09:22:09Z [info] {"level":"info","ts":1621761729.1096156,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":"caddyfile"}
2021-05-23T09:22:09Z [info] {"level":"warn","ts":1621761729.1157136,"msg":"input is not formatted with 'caddy fmt'","adapter":"caddyfile","file":"/etc/caddy/Caddyfile","line":1}
2021-05-23T09:22:09Z [info] {"level":"info","ts":1621761729.1194496,"logger":"admin","msg":"admin endpoint started","address":"tcp/0.0.0.0:2019","enforce_origin":false,"origins":["0.0.0.0:2019"]}
2021-05-23T09:22:09Z [info] {"level":"warn","ts":1621761729.1217167,"logger":"admin","msg":"admin endpoint on open interface; host checking disabled","address":"tcp/0.0.0.0:2019"}
2021-05-23T09:22:09Z [info] {"level":"debug","ts":1621761729.1244981,"logger":"http","msg":"starting server loop","address":"[::]:80","http3":false,"tls":false}
2021-05-23T09:22:09Z [info] {"level":"info","ts":1621761729.12673,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc00025e850"}
2021-05-23T09:22:09Z [info] {"level":"info","ts":1621761729.128089,"logger":"tls.cache.maintenance","msg":"stopped background certificate maintenance","cache":"0xc00025e850"}
2021-05-23T09:22:09Z [info] run: loading initial config: loading new config: http app module: start: tcp: listening on [::]:80: listen tcp 0.0.0.0:80: bind: address already in use
2021-05-23T09:22:09Z [info] Main child exited normally with code: 1
2021-05-23T09:22:09Z [info] Starting clean up.
2021-05-23T09:22:12Z [info] Health check status changed 'warning' => 'passing'

fly.io is kind of a Container as A Service so It’s impossible that something else is listening on that port. Unless I’m running two processes in my container which I’m not.

6. Links to relevant resources:

You should know that I prefer to bind to 0.0.0.0 and :: since:

Hmm, I’m not sure.

Are you sure their Docker stack has ipv6 support enabled?

Yes, it does. In fact, it works out of the box on other apps. Though it’s not a docker daemon they are running. They extract the rootfs of the docker image and their custom kernel and run that as a firecracker microVM orchestrated with Nomad.

Caddy binds to all interfaces by default, so the problem might be elsewhere :thinking:

Caddy just uses the Go stdlib for this, so if there’s an incompatibility, it might be there. You could try writing a simple Go program which serves a simple http response, and see if that works any different, to narrow it down.

1 Like

I’m going to reference this back to the fly community. Ask them to verify wether they have assigned my app the write IPV6 address.

This topic was automatically closed after 30 days. New replies are no longer allowed.