Unable to get certificate from Let's Encrypt

1. Caddy version (caddy version): v2.4.5 h1:P1mRs6V2cMcagSPn+NWpD+OEYUYLIf6ecOa48cFGeUg=

2. How I run Caddy:

Using systemctl (caddy enabled and started successfully)

a. System environment:

Raspbian latest with systemd

b. Command:

systemctl start

c. Service/unit/compose file:

# caddy.service
#
# For using Caddy with a config file.
#
# Make sure the ExecStart and ExecReload commands are correct
# for your installation.
#
# See https://caddyserver.com/docs/install for instructions.
#
# WARNING: This service does not use the --resume flag, so if you
# use the API to make changes, they will be overwritten by the
# Caddyfile next time the service is restarted. If you intend to
# use Caddy's API to configure it, add the --resume flag to the
# `caddy run` command or use the caddy-api.service file instead.

[Unit]
Description=Caddy
Documentation=https://caddyserver.com/docs/
After=network.target network-online.target
Requires=network-online.target

[Service]
Type=notify
User=caddy
Group=caddy
ExecStart=/usr/bin/caddy run --environ --config /etc/caddy/Caddyfile
ExecReload=/usr/bin/caddy reload --config /etc/caddy/Caddyfile
TimeoutStopSec=5s
LimitNOFILE=1048576
LimitNPROC=512
PrivateTmp=true
ProtectSystem=full
AmbientCapabilities=CAP_NET_BIND_SERVICE

d. My complete Caddyfile or JSON config:

*.xyzat.xyz {

    root * /var/www/public_html
    encode gzip
    file_server

    tls myname.herel@gmail.com

    header / {
        Content-Security-Policy = "upgrade-insecure-requests; default-src 'self'; style-src 'self'; script-src 'self'; img-src 'self'; object-src 'self'; worker-src 'self'; manifest-src 'self';"
        Strict-Transport-Security = "max-age=63072000; includeSubDomains; preload"
        X-Xss-Protection = "1; mode=block"
        X-Frame-Options = "DENY"
        X-Content-Type-Options = "nosniff"
        Referrer-Policy = "strict-origin-when-cross-origin"
        Permissions-Policy = "fullscreen=(self)"
        cache-control = "max-age=0,no-cache,no-store,must-revalidate"
    }

}

3. The problem I’m having:

Unable to get a certificate on startup.

4. Error messages and/or full log output:

{“level”:“error”,“ts”:1634001876.7426176,“logger”:“tls.obtain”,“msg”:“could not get certificate from issuer”,“identifier”:"*.xyzat.xyz",“issuer”:“acme-v02.api.letsencrypt.org-directory”,“error”:“registering account [mailto:my.name.here@gmail.com] with server: provisioning client: performing request: Get “https://acme-v02.api.letsencrypt.org/directory”: dial tcp: lookup acme-v02.api.letsencrypt.org on 192.168.1.1:53: read udp 192.168.1.55:57115->192.168.1.1:53: i/o timeout”}

The above keeps repeating

5. What I already tried:

I have spent more than two days trying to fix the issue changing many things, on firewall, Caddyfile, re-generating static site, etc… All to no avail.

6. Links to relevant resources:

What happens if you curl that directory endpoint from your server?

As an aside, these lines all have problems:

Matching in Caddy is exact. If you use a / matcher, then only requests to exactly / will have headers set. Remove the / to match all requests (which is the same as having specified *). See the request matching docs.

Also, the syntax for setting headers does not include =. Using = will make Caddy have the wrong behaviour, i.e. doing a header replacement, searching for = and replacing it with the value. This’ll never have the right result. See the docs for the header directive for the correct syntax (just remove all of the =).

Keep in mind that configuring the DNS challenge is necessary to get a wildcard certificate from ACME CAs.

Hello Matt,

Thanks for your attention to this, but I am not sure what you mean by curl that directory endpoint. What curl command should I use and which directory exactly?

Hello Francis,
Thanks for you attention to this. I did delete the “/” and all the “=” signs, restarted the service, but the result was identical. Exactly the same error.

Maybe it is important to know that I am using Dynamic DNS. But at the provider and the router level it is working.

I am suspecting a firewall issue but I read in the documentation that only TCP was required which is obviously allowed on my DMZ subnet.

Referring to the GET request being performed to Let’s Encrypt that has the error:

curl -v https://acme-v02.api.letsencrypt.org/directory

Run that on the same server that Caddy is on.

I’m guessing your local DNS is misconfigured.

Hello Matt,

The result is near endless of:

Blockquote

  • Expire in 0 ms for 6 (transfer 0x58d8b0)
  • Expire in 1 ms for 1 (transfer 0x58d8b0)
  • Expire in 0 ms for 1 (transfer 0x58d8b0)
  • Expire in 1 ms for 1 (transfer 0x58d8b0)
  • Expire in 0 ms for 1 (transfer 0x58d8b0)
  • Expire in 0 ms for 1 (transfer 0x58d8b0)
  • Expire in 2 ms for 1 (transfer 0x58d8b0)

That looks like a bug in your version of curl: CURL error "Expire in 1 ms for 1 (transfer 0x...)" - Stack Overflow

What is the full output?

That is the full output. I stopped it after a few hundred lines. Only the number of ms goes up.

I have however noticed something else. There is an attempt to contact mule.caddyserver.com which I noticed get blocked at the moment by my firewall. What is its purpose and could it be related to this issue?

Completely unrelated. That’s the Caddy website’s use of https://usefathom.com/ for site analytics. The JS for the analytics is served from that subdomain.

Make sure your server can reach Let’s Encrypt and/or ZeroSSL servers. If your firewall is too aggressive, then it won’t be possible to have certificates issued.

1 Like

OK, I opened my firewall even more and I made progress but still have an issue.

Here is the copy of my log:

-- Logs begin at Tue 2021-10-12 07:56:39 AWST. --
Oct 12 14:36:51 xyz systemd[1]: Started Caddy.
Oct 12 14:36:51 xyz caddy[2986]: {"level":"info","ts":1634020611.1427953,"msg":"serving initial configuration"}
Oct 12 14:36:51 xyz caddy[2986]: {"level":"info","ts":1634020611.1446311,"logger":"tls.obtain","msg":"acquiring lock","identifier":"*.xyzat.xyz"}
Oct 12 14:36:51 xyz sudo[2983]: pam_unix(sudo:session): session closed for user root
Oct 12 14:36:51 xyz caddy[2986]: {"level":"info","ts":1634020611.1606796,"logger":"tls.obtain","msg":"lock acquired","identifier":"*.xyzat.xyz"}
Oct 12 14:36:51 xyz caddy[2986]: {"level":"info","ts":1634020611.1879816,"logger":"tls.issuance.acme","msg":"waiting on internal rate limiter","identifiers":["*.xyzat.xyz"],"ca":"https://acme-v02.api.letsencrypt.org/directory","account":"myname.here@gmail.com"}
Oct 12 14:36:51 xyz caddy[2986]: {"level":"info","ts":1634020611.1881683,"logger":"tls.issuance.acme","msg":"done waiting on internal rate limiter","identifiers":["*.xyzat.xyz"],"ca":"https://acme-v02.api.letsencrypt.org/directory","account":"myname.here@gmail.com"}
Oct 12 14:36:53 xyz caddy[2986]: {"level":"error","ts":1634020613.5643933,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"*.xyzat.xyz","issuer":"acme-v02.api.letsencrypt.org-directory","error":"[*.xyzat.xyz] solving challenges: *.xyzat.xyz: no solvers available for remaining challenges (configured=[http-01 tls-alpn-01] offered=[dns-01] remaining=[dns-01]) (order=https://acme-v02.api.letsencrypt.org/acme/order/235707010/31273085370) (ca=https://acme-v02.api.letsencrypt.org/directory)"}
Oct 12 14:36:53 xyz caddy[2986]: {"level":"info","ts":1634020613.5746617,"logger":"tls.issuance.acme","msg":"waiting on internal rate limiter","identifiers":["*.xyzat.xyz"],"ca":"https://acme.zerossl.com/v2/DV90","account":"myname.here@gmail.com"}
Oct 12 14:36:53 xyz caddy[2986]: {"level":"info","ts":1634020613.5748856,"logger":"tls.issuance.acme","msg":"done waiting on internal rate limiter","identifiers":["*.xyzat.xyz"],"ca":"https://acme.zerossl.com/v2/DV90","account":"myname.here@gmail.com"}
Oct 12 14:36:56 xyz caddy[2986]: {"level":"error","ts":1634020616.0055635,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"*.xyzat.xyz","issuer":"acme.zerossl.com-v2-DV90","error":"[*.xyzat.xyz] solving challenges: *.xyzat.xyz: no solvers available for remaining challenges (configured=[tls-alpn-01 http-01] offered=[dns-01] remaining=[dns-01]) (order=https://acme.zerossl.com/v2/DV90/order/bM8izAyyNZ6K3OZX4FFV0g) (ca=https://acme.zerossl.com/v2/DV90)"}
Oct 12 14:36:56 xyz caddy[2986]: {"level":"error","ts":1634020616.005901,"logger":"tls.obtain","msg":"will retry","error":"[*.xyzat.xyz] Obtain: [*.xyzat.xyz] solving challenges: *.xyzat.xyz: no solvers available for remaining challenges (configured=[tls-alpn-01 http-01] offered=[dns-01] remaining=[dns-01]) (order=https://acme.zerossl.com/v2/DV90/order/bM8izAyyNZ6K3OZX4FFV0g) (ca=https://acme.zerossl.com/v2/DV90)","attempt":1,"retrying_in":60,"elapsed":4.844895502,"max_duration":2592000}

Is that the result of having too many tries earlier on?

No, it’s just the DNS challenge is required for wildcard domain names, so you’ll have to configure that.

To clarify, if you use * in your hostname, then Caddy will try to manage a wildcard certificate with that name. But Caddy can only get wildcard certificates if the DNS challenge is installed.

If you don’t actually need wildcard certificates and only serve one or a few domains, list those domains instead of using a wildcard *.

2 Likes

Just wanted to let everyone know that I finally did mange to get it all worked out. As suggested I did not use a wildcard, made a small fix to my dns records which did contain an ambiguity.

Thanks again for all your help which really did get me going in the right direction.

3 Likes