Caddy on a IPv6-only VPS

Help
1

1. The problem I’m experiencing:

I have installed the Caddy server on an IPv6-only VPS. I’ve noticed that the Caddy service is unable to connect to api.zerossl.com due to a missing AAAA DNS record.

2. Error messages and/or full log output:

Jan 31 16:13:07 eu-it.pavo.pw caddy[7026]: {"level":"error","ts":1706713987.8088112,"logger":"tls.obtain","msg":"will retry","error":"[cockpit.pavo.pw] Obtain: account pre-registration callback: performing EAB credentials request: Post \"https://api.zerossl.com/acme/eab-credentials-email\": dial tcp 34.196.131.225:443: connect: network is unreachable","attempt":25,"retrying_in":21600,"elapsed":64931.251797778,"max_duration":2592000}

If I attempt to curl using IPv6 only:

curl -6 -v https://api.zerossl.com/acme/eab-credentials-email
* Could not resolve host: api.zerossl.com
* Closing connection 0
curl: (6) Could not resolve host: api.zerossl.com

Curl using IPv4 is not working due to the VPS not having an IPv4 network:

curl -4 -v https://api.zerossl.com/acme/eab-credentials-email
*   Trying 34.237.95.29:443...
* Immediate connect fail for 34.237.95.29: Network is unreachable
*   Trying 34.196.131.225:443...
* Immediate connect fail for 34.196.131.225: Network is unreachable
*   Trying 52.22.147.226:443...
* Immediate connect fail for 52.22.147.226: Network is unreachable
* Closing connection 0
curl: (7) Couldn't connect to server

3. Caddy version:

v2.7.6 h1:w0NymbG2m9PcvKWsrXO6EEkY9Ru4FJK8uQbYcev1p3A=

4. How I installed and ran Caddy:

dnf install caddy

a. System environment:

  Virtualization: kvm
Operating System: AlmaLinux 9.3 (Shamrock Pampas Cat) 
     CPE OS Name: cpe:/o:almalinux:almalinux:9::baseos
          Kernel: Linux 5.14.0-362.18.1.el9_3.x86_64
    Architecture: x86-64
 Hardware Vendor: OpenStack Foundation
  Hardware Model: OpenStack Nova
Firmware Version: 1.13.0-1ubuntu1.1

d. My complete Caddy config:

cockpit.pavo.pw {
    reverse_proxy https://localhost:9090 {
        
        transport http {
            tls_insecure_skip_verify
        }
    }
}

5. Links to relevant resources:

I found a similar issue discussed in this topic: The struggle to get Caddy installed in an IPv6-only machine

2

You can probably ignore that, Caddy will be trying Let’s Encrypt and ZeroSSL. It’s not a big deal if ZeroSSL isn’t working, as long as Let’s Encrypt works.

Is that really all you have in your logs? Are you sure Let’s Encrypt issuance didn’t work?

3

Just restarted caddy service, here is a more detailed logs.

TLDR: Maybe the revelant log is:

Jan 31 20:56:21 eu-it.pavo.pw caddy[37904]: {"level":"error","ts":1706730981.027879,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"cockpit.pavo.pw","issuer":"acme-v02.api.letsencrypt.org-directory","error":"HTTP 400 urn:ietf:params:acme:error:malformed - Unable to contact \"cockpit.pavo.pw\" at \"2a00:6d42:1242:1c00::5f\", no IPv4 addresses to try as fallback"}

Full logs:

Jan 31 20:46:08 eu-it.pavo.pw systemd[1]: Started Caddy.
Jan 31 20:46:08 eu-it.pavo.pw caddy[37904]: {"level":"info","ts":1706730368.3330045,"msg":"serving initial configuration"}
Jan 31 20:46:08 eu-it.pavo.pw caddy[37904]: {"level":"warn","ts":1706730368.336733,"logger":"tls","msg":"storage cleaning happened too recently; skipping for now","storage":"FileStorage:/var/lib/caddy/.local/share/caddy","instance":"398ef0d5-3f64-4b2f-9295-46cbf1aefec7","try_again":1706816768.3367307,"try_again_in":86399.999999094}
Jan 31 20:46:08 eu-it.pavo.pw caddy[37904]: {"level":"info","ts":1706730368.3370643,"logger":"tls","msg":"finished cleaning storage units"}
Jan 31 20:46:08 eu-it.pavo.pw caddy[37904]: {"level":"info","ts":1706730368.3373895,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc00027bb80"}
Jan 31 20:46:08 eu-it.pavo.pw caddy[37904]: {"level":"info","ts":1706730368.3386922,"logger":"tls.obtain","msg":"acquiring lock","identifier":"cockpit.pavo.pw"}
Jan 31 20:46:08 eu-it.pavo.pw caddy[37904]: {"level":"info","ts":1706730368.3405008,"logger":"tls.obtain","msg":"lock acquired","identifier":"cockpit.pavo.pw"}
Jan 31 20:46:08 eu-it.pavo.pw caddy[37904]: {"level":"info","ts":1706730368.340875,"logger":"tls.obtain","msg":"obtaining certificate","identifier":"cockpit.pavo.pw"}
Jan 31 20:46:08 eu-it.pavo.pw caddy[37904]: {"level":"info","ts":1706730368.343171,"logger":"http","msg":"waiting on internal rate limiter","identifiers":["cockpit.pavo.pw"],"ca":"https://acme-v02.api.letsencrypt.org/directory","account":""}
Jan 31 20:46:08 eu-it.pavo.pw caddy[37904]: {"level":"info","ts":1706730368.3432972,"logger":"http","msg":"done waiting on internal rate limiter","identifiers":["cockpit.pavo.pw"],"ca":"https://acme-v02.api.letsencrypt.org/directory","account":""}
Jan 31 20:46:09 eu-it.pavo.pw caddy[37904]: {"level":"info","ts":1706730369.5571034,"logger":"http.acme_client","msg":"trying to solve challenge","identifier":"cockpit.pavo.pw","challenge_type":"http-01","ca":"https://acme-v02.api.letsencrypt.org/directory"}
Jan 31 20:46:10 eu-it.pavo.pw caddy[37904]: {"level":"error","ts":1706730370.1305127,"logger":"http.acme_client","msg":"challenge failed","identifier":"cockpit.pavo.pw","challenge_type":"http-01","problem":{"type":"urn:ietf:params:acme:error:connection","title":"","detail":"2a00:6d42:1242:1c00::5f: Fetching http://cockpit.pavo.pw/.well-known/acme-challenge/K8FFQnkEf-XzJpIGKd_6luJsioSYExq4APMf72RjOXY: Error getting validation data","instance":"","subproblems":[]}}
Jan 31 20:46:10 eu-it.pavo.pw caddy[37904]: {"level":"error","ts":1706730370.1309326,"logger":"http.acme_client","msg":"validating authorization","identifier":"cockpit.pavo.pw","problem":{"type":"urn:ietf:params:acme:error:connection","title":"","detail":"2a00:6d42:1242:1c00::5f: Fetching http://cockpit.pavo.pw/.well-known/acme-challenge/K8FFQnkEf-XzJpIGKd_6luJsioSYExq4APMf72RjOXY: Error getting validation data","instance":"","subproblems":[]},"order":"https://acme-v02.api.letsencrypt.org/acme/order/1377580036/240850078807","attempt":1,"max_attempts":3}
Jan 31 20:46:11 eu-it.pavo.pw caddy[37904]: {"level":"info","ts":1706730371.6191645,"logger":"http.acme_client","msg":"trying to solve challenge","identifier":"cockpit.pavo.pw","challenge_type":"tls-alpn-01","ca":"https://acme-v02.api.letsencrypt.org/directory"}
Jan 31 20:46:12 eu-it.pavo.pw caddy[37904]: {"level":"error","ts":1706730372.1790886,"logger":"http.acme_client","msg":"challenge failed","identifier":"cockpit.pavo.pw","challenge_type":"tls-alpn-01","problem":{"type":"urn:ietf:params:acme:error:malformed","title":"","detail":"Unable to contact \"cockpit.pavo.pw\" at \"2a00:6d42:1242:1c00::5f\", no IPv4 addresses to try as fallback","instance":"","subproblems":[]}}
Jan 31 20:46:12 eu-it.pavo.pw caddy[37904]: {"level":"error","ts":1706730372.179562,"logger":"http.acme_client","msg":"validating authorization","identifier":"cockpit.pavo.pw","problem":{"type":"urn:ietf:params:acme:error:malformed","title":"","detail":"Unable to contact \"cockpit.pavo.pw\" at \"2a00:6d42:1242:1c00::5f\", no IPv4 addresses to try as fallback","instance":"","subproblems":[]},"order":"https://acme-v02.api.letsencrypt.org/acme/order/1377580036/240850083897","attempt":2,"max_attempts":3}
Jan 31 20:46:12 eu-it.pavo.pw caddy[37904]: {"level":"error","ts":1706730372.1798322,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"cockpit.pavo.pw","issuer":"acme-v02.api.letsencrypt.org-directory","error":"HTTP 400 urn:ietf:params:acme:error:malformed - Unable to contact \"cockpit.pavo.pw\" at \"2a00:6d42:1242:1c00::5f\", no IPv4 addresses to try as fallback"}
Jan 31 20:46:12 eu-it.pavo.pw caddy[37904]: {"level":"warn","ts":1706730372.1806428,"logger":"http","msg":"missing email address for ZeroSSL; it is strongly recommended to set one for next time"}
Jan 31 20:46:12 eu-it.pavo.pw caddy[37904]: {"level":"error","ts":1706730372.2244542,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"cockpit.pavo.pw","issuer":"acme.zerossl.com-v2-DV90","error":"account pre-registration callback: performing EAB credentials request: Post \"https://api.zerossl.com/acme/eab-credentials-email\": dial tcp 34.196.131.225:443: connect: network is unreachable"}
Jan 31 20:46:12 eu-it.pavo.pw caddy[37904]: {"level":"error","ts":1706730372.2248123,"logger":"tls.obtain","msg":"will retry","error":"[cockpit.pavo.pw] Obtain: account pre-registration callback: performing EAB credentials request: Post \"https://api.zerossl.com/acme/eab-credentials-email\": dial tcp 34.196.131.225:443: connect: network is unreachable","attempt":1,"retrying_in":60,"elapsed":3.884139548,"max_duration":2592000}
Jan 31 20:47:12 eu-it.pavo.pw caddy[37904]: {"level":"info","ts":1706730432.226078,"logger":"tls.obtain","msg":"obtaining certificate","identifier":"cockpit.pavo.pw"}
Jan 31 20:47:13 eu-it.pavo.pw caddy[37904]: {"level":"info","ts":1706730433.3537796,"logger":"http.acme_client","msg":"trying to solve challenge","identifier":"cockpit.pavo.pw","challenge_type":"tls-alpn-01","ca":"https://acme-staging-v02.api.letsencrypt.org/directory"}
Jan 31 20:47:13 eu-it.pavo.pw caddy[37904]: {"level":"error","ts":1706730433.9573002,"logger":"http.acme_client","msg":"challenge failed","identifier":"cockpit.pavo.pw","challenge_type":"tls-alpn-01","problem":{"type":"urn:ietf:params:acme:error:malformed","title":"","detail":"Unable to contact \"cockpit.pavo.pw\" at \"2a00:6d42:1242:1c00::5f\", no IPv4 addresses to try as fallback","instance":"","subproblems":[]}}
Jan 31 20:47:13 eu-it.pavo.pw caddy[37904]: {"level":"error","ts":1706730433.9577062,"logger":"http.acme_client","msg":"validating authorization","identifier":"cockpit.pavo.pw","problem":{"type":"urn:ietf:params:acme:error:malformed","title":"","detail":"Unable to contact \"cockpit.pavo.pw\" at \"2a00:6d42:1242:1c00::5f\", no IPv4 addresses to try as fallback","instance":"","subproblems":[]},"order":"https://acme-staging-v02.api.letsencrypt.org/acme/order/122997564/14090202514","attempt":1,"max_attempts":3}
Jan 31 20:47:13 eu-it.pavo.pw caddy[37904]: {"level":"error","ts":1706730433.95798,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"cockpit.pavo.pw","issuer":"acme-v02.api.letsencrypt.org-directory","error":"HTTP 400 urn:ietf:params:acme:error:malformed - Unable to contact \"cockpit.pavo.pw\" at \"2a00:6d42:1242:1c00::5f\", no IPv4 addresses to try as fallback"}
Jan 31 20:47:13 eu-it.pavo.pw caddy[37904]: {"level":"warn","ts":1706730433.9584997,"logger":"http","msg":"missing email address for ZeroSSL; it is strongly recommended to set one for next time"}
Jan 31 20:47:13 eu-it.pavo.pw caddy[37904]: {"level":"error","ts":1706730433.9807427,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"cockpit.pavo.pw","issuer":"acme.zerossl.com-v2-DV90","error":"account pre-registration callback: performing EAB credentials request: Post \"https://api.zerossl.com/acme/eab-credentials-email\": dial tcp 34.237.95.29:443: connect: network is unreachable"}
Jan 31 20:47:13 eu-it.pavo.pw caddy[37904]: {"level":"error","ts":1706730433.9810572,"logger":"tls.obtain","msg":"will retry","error":"[cockpit.pavo.pw] Obtain: account pre-registration callback: performing EAB credentials request: Post \"https://api.zerossl.com/acme/eab-credentials-email\": dial tcp 34.237.95.29:443: connect: network is unreachable","attempt":2,"retrying_in":120,"elapsed":65.640384345,"max_duration":2592000}
4

So this means LE isn’t able to reach your IPv6 address, and has nothing to fall back to because of that.

Make sure that IP is valid, make sure your firewall isn’t blocking ports 80 and 443 and that you have those ports forwarded to your Caddy server.

5

My mistake.

Both the HTTP and HTTPS ports are closed on my VPS for external connections -.-

I appreciate the quick feedback!

Apologize :frowning:

1 Like
6

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.