Could not get certificate from issuer due to DNS timeout

1. The problem I’m having:

The requests from Caddy to get certificates stopped working suddenly. I do not recollect making any changes to my setup recently except updating Ubuntu and rebooting my Raspberry Pi 4.

I connected to the docker container and I verified using curl that the domains are reachable and I’m able to get a response. I don’t quite understand why caddy is having issues connecting to the DNS server. Appreciate any help to fix this problem.

/etc/caddy # curl -v https://acme-staging-v02.api.letsencrypt.org/directory
* Host acme-staging-v02.api.letsencrypt.org:443 was resolved.
* IPv6: 2606:4700:60:0:f41b:d4fe:4325:6026
* IPv4: 172.65.46.172
*   Trying [2606:4700:60:0:f41b:d4fe:4325:6026]:443...
* Connected to acme-staging-v02.api.letsencrypt.org (2606:4700:60:0:f41b:d4fe:4325:6026) port 443
* ALPN: curl offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384 / X25519 / RSASSA-PSS
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=acme-staging-v02.api.letsencrypt.org
*  start date: Dec 27 10:10:47 2023 GMT
*  expire date: Mar 26 10:10:46 2024 GMT
*  subjectAltName: host "acme-staging-v02.api.letsencrypt.org" matched cert's "acme-staging-v02.api.letsencrypt.org"
*  issuer: C=US; O=Let's Encrypt; CN=R3
*  SSL certificate verify ok.
*   Certificate level 0: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 1: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 2: Public key type RSA (4096/152 Bits/secBits), signed using sha256WithRSAEncryption
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://acme-staging-v02.api.letsencrypt.org/directory
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: acme-staging-v02.api.letsencrypt.org]
* [HTTP/2] [1] [:path: /directory]
* [HTTP/2] [1] [user-agent: curl/8.5.0]
* [HTTP/2] [1] [accept: */*]
> GET /directory HTTP/2
> Host: acme-staging-v02.api.letsencrypt.org
> User-Agent: curl/8.5.0
> Accept: */*
> 
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
< HTTP/2 200 
< server: nginx
< date: Sat, 17 Feb 2024 21:05:17 GMT
< content-type: application/json
< content-length: 826
< cache-control: public, max-age=0, no-cache
< x-frame-options: DENY
< strict-transport-security: max-age=604800
< 
{
  "G4wLOYJydaQ": "https://community.letsencrypt.org/t/adding-random-entries-to-the-directory/33417",
  "keyChange": "https://acme-staging-v02.api.letsencrypt.org/acme/key-change",
  "meta": {
    "caaIdentities": [
      "letsencrypt.org"
    ],
    "termsOfService": "https://letsencrypt.org/documents/LE-SA-v1.3-September-21-2022.pdf",
    "website": "https://letsencrypt.org/docs/staging-environment/"
  },
  "newAccount": "https://acme-staging-v02.api.letsencrypt.org/acme/new-acct",
  "newNonce": "https://acme-staging-v02.api.letsencrypt.org/acme/new-nonce",
  "newOrder": "https://acme-staging-v02.api.letsencrypt.org/acme/new-order",
  "renewalInfo": "https://acme-staging-v02.api.letsencrypt.org/draft-ietf-acme-ari-02/renewalInfo/",
  "revokeCert": "https://acme-staging-v02.api.letsencrypt.org/acme/revoke-cert"
* Connection #0 to host acme-staging-v02.api.letsencrypt.org left intact
}
/etc/caddy # curl -v https://acme.zerossl.com/v2/DV90
* Host acme.zerossl.com:443 was resolved.
* IPv6: (none)
* IPv4: 91.199.212.80
*   Trying 91.199.212.80:443...
* Connected to acme.zerossl.com (91.199.212.80) port 443
* ALPN: curl offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-CHACHA20-POLY1305 / X25519 / RSASSA-PSS
* ALPN: server accepted h2
* Server certificate:
*  subject: C=GB; ST=Manchester; O=Sectigo Limited; CN=acme.zerossl.com
*  start date: Jan 26 00:00:00 2024 GMT
*  expire date: Apr 25 23:59:59 2024 GMT
*  subjectAltName: host "acme.zerossl.com" matched cert's "acme.zerossl.com"
*  issuer: C=GB; ST=Greater Manchester; L=Salford; O=Sectigo Limited; CN=Sectigo RSA Organization Validation Secure Server CA
*  SSL certificate verify ok.
*   Certificate level 0: Public key type RSA (4096/152 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 1: Public key type RSA (2048/112 Bits/secBits), signed using sha384WithRSAEncryption
*   Certificate level 2: Public key type RSA (4096/152 Bits/secBits), signed using sha384WithRSAEncryption
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://acme.zerossl.com/v2/DV90
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: acme.zerossl.com]
* [HTTP/2] [1] [:path: /v2/DV90]
* [HTTP/2] [1] [user-agent: curl/8.5.0]
* [HTTP/2] [1] [accept: */*]
> GET /v2/DV90 HTTP/2
> Host: acme.zerossl.com
> User-Agent: curl/8.5.0
> Accept: */*
> 
< HTTP/2 200 
< server: nginx
< date: Sat, 17 Feb 2024 21:06:57 GMT
< content-type: application/json
< content-length: 645
< access-control-allow-origin: *
< strict-transport-security: max-age=15724800; includeSubDomains
< 
{
  "newNonce": "https://acme.zerossl.com/v2/DV90/newNonce",
  "newAccount": "https://acme.zerossl.com/v2/DV90/newAccount",
  "newOrder": "https://acme.zerossl.com/v2/DV90/newOrder",
  "revokeCert": "https://acme.zerossl.com/v2/DV90/revokeCert",
  "keyChange": "https://acme.zerossl.com/v2/DV90/keyChange",
  "meta": {
    "termsOfService": "https://secure.trust-provider.com/repository/docs/Legacy/20230516_Certificate_Subscriber_Agreement_v_2_6_click.pdf",
    "website": "https://zerossl.com",
    "caaIdentities": ["sectigo.com", "trust-provider.com", "usertrust.com", "comodoca.com", "comodo.com"],
    "externalAccountRequired": true
  }
* Connection #0 to host acme.zerossl.com left intact

2. Error messages and/or full log output:

INF ts=1708201167.5861866 logger=tls.obtain msg=obtaining certificate identifier=bwtechhideout.duckdns.org

WRN ts=1708201177.607031 logger=http.acme_client msg=HTTP request failed; retrying url=https://acme-staging-v02.api.letsencrypt.org/directory error=performing request: Get "https://acme-staging-v02.api.letsencrypt.org/directory": dial tcp: lookup acme-staging-v02.api.letsencrypt.org on 1.1.1.1:53: read udp 192.168.1.22:47631->1.1.1.1:53: i/o timeout

WRN ts=1708201187.8615015 logger=http.acme_client msg=HTTP request failed; retrying url=https://acme-staging-v02.api.letsencrypt.org/directory error=performing request: Get "https://acme-staging-v02.api.letsencrypt.org/directory": dial tcp: lookup acme-staging-v02.api.letsencrypt.org on 1.1.1.1:53: read udp 192.168.1.22:56622->1.1.1.1:53: i/o timeout

WRN ts=1708201198.1158926 logger=http.acme_client msg=HTTP request failed; retrying url=https://acme-staging-v02.api.letsencrypt.org/directory error=performing request: Get "https://acme-staging-v02.api.letsencrypt.org/directory": dial tcp: lookup acme-staging-v02.api.letsencrypt.org on 1.1.1.1:53: read udp 192.168.1.22:36649->1.1.1.1:53: i/o timeout

ERR ts=1708201198.116072 logger=tls.obtain msg=could not get certificate from issuer identifier=bwtechhideout.duckdns.org issuer=acme-v02.api.letsencrypt.org-directory error=[bwtechhideout.duckdns.org] creating new order: provisioning client: performing request: Get "https://acme-staging-v02.api.letsencrypt.org/directory": dial tcp: lookup acme-staging-v02.api.letsencrypt.org on 1.1.1.1:53: read udp 192.168.1.22:36649->1.1.1.1:53: i/o timeout (ca=https://acme-staging-v02.api.letsencrypt.org/directory)

WRN ts=1708201208.121339 logger=http.acme_client msg=HTTP request failed; retrying url=https://acme.zerossl.com/v2/DV90 error=performing request: Get "https://acme.zerossl.com/v2/DV90": dial tcp: lookup acme.zerossl.com on 1.1.1.1:53: read udp 192.168.1.22:37054->1.1.1.1:53: i/o timeout

WRN ts=1708201218.375631 logger=http.acme_client msg=HTTP request failed; retrying url=https://acme.zerossl.com/v2/DV90 error=performing request: Get "https://acme.zerossl.com/v2/DV90": dial tcp: lookup acme.zerossl.com on 1.1.1.1:53: read udp 192.168.1.22:55081->1.1.1.1:53: i/o timeout

WRN ts=1708201228.6300025 logger=http.acme_client msg=HTTP request failed; retrying url=https://acme.zerossl.com/v2/DV90 error=performing request: Get "https://acme.zerossl.com/v2/DV90": dial tcp: lookup acme.zerossl.com on 1.1.1.1:53: read udp 192.168.1.22:43947->1.1.1.1:53: i/o timeout

ERR ts=1708201228.6302085 logger=tls.obtain msg=could not get certificate from issuer identifier=bwtechhideout.duckdns.org issuer=acme.zerossl.com-v2-DV90 error=[bwtechhideout.duckdns.org] creating new order: provisioning client: performing request: Get "https://acme.zerossl.com/v2/DV90": dial tcp: lookup acme.zerossl.com on 1.1.1.1:53: read udp 192.168.1.22:43947->1.1.1.1:53: i/o timeout (ca=https://acme.zerossl.com/v2/DV90)

ERR ts=1708201228.6303654 logger=tls.obtain msg=will retry error=[bwtechhideout.duckdns.org] Obtain: [bwtechhideout.duckdns.org] creating new order: provisioning client: performing request: Get "https://acme.zerossl.com/v2/DV90": dial tcp: lookup acme.zerossl.com on 1.1.1.1:53: read udp 192.168.1.22:43947->1.1.1.1:53: i/o timeout (ca=https://acme.zerossl.com/v2/DV90) attempt=66 retrying_in=21600 elapsed=954520.23355187 max_duration=2592000

3. Caddy version:

caddy version
v2.7.6 h1:w0NymbG2m9PcvKWsrXO6EEkY9Ru4FJK8uQbYcev1p3A=

4. How I installed and ran Caddy:

a. System environment:

Docker on Ubuntu 22.04

Distributor ID: Ubuntu
Description:    Ubuntu 22.04.3 LTS
Release:        22.04
Codename:       jammy

b. Command:

docker compose up

c. Service/unit/compose file:

  caddy:
    image: caddy:alpine
    restart: unless-stopped
    container_name: caddy
    volumes:
      - /media/kiran/Kiran/docker-config/caddy/Caddyfile:/etc/caddy/Caddyfile
      - /media/kiran/Kiran/docker-config/caddy/data:/data
      - /media/kiran/Kiran/docker-config/caddy/config:/config
      - /media/kiran/Kiran/docker-config/caddy/certs:/certs
      - /media/kiran/Kiran/docker-config/caddy//sites:/srv
    network_mode: "host"

d. My complete Caddy config:

https://nxttechhideout.duckdns.org:443 {
    header Strict-Transport-Security max-age=31536000;
    reverse_proxy localhost:11000
}

https://bwtechhideout.duckdns.org:443 {
#    header Strict-Transport-Security max-age=31536000;
    reverse_proxy localhost:7080
}

5. Links to relevant resources:

I have a temporary workaround now by copying the IP address of these domains to /etc/hosts file and I’m able to get certificates now. But, I still don’t understand why Caddy is having issues connecting to the DNS server when curl does not have a problem.

:man_shrugging:

It must be something weird with your Docker setup. Caddy just uses the system’s DNS resolver.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.