"no solvers available for remaining challenges" but Caddyfile _used to_ work

sclarkxyz · September 10, 2021, 2:44am

1. Caddy version (`caddy version`):

Docker - 2.4.5-alpine built with the Cloudflare DNS module using xcaddy See Dockerfile below
I don’t know how to get it to print the version at startup, but I can provide that if someone lets me know how.

2. How I run Caddy:

I am trying to host sclark.xyz and www.sclark.xyz on my internal network at home. However, I want them to have real certificates and I don’t want to have to forward any ports to do so.

a. System environment:

Ubuntu 20.04.3 LTS
Docker version 20.10.8, build 3967b7d
docker-compose version 1.27.4, build 40524192

b. Command:

sudo docker-compose up

c. Service/unit/compose file:

#./docker-compose.yml
version: "3.8"
services:
  caddyserver:
    container_name: "caddyserver"
    build:
      context: "./caddy"
      dockerfile: "Dockerfile"
      args:
        caddybuilderversion: "2.4.5-builder-alpine"
        caddyversion: "2.4.5-alpine"
    restart: "unless-stopped"
    ports:
      - target: 80
        published: 80
        protocol: "tcp"
        mode: "host"
      - target: 443
        published: 443
        protocol: tcp
        mode: "host"
    environment:
      TZ: "America/Chicago"
    volumes:
      - "/home/sam/xyz/sclark/caddy/config/Caddyfile:/etc/caddy/Caddyfile"
      - "/home/sam/xyz/sclark/caddy/data:/data"
      - "/home/sam/xyz/sclark/www:/var/www"
      - "Certfiles:/root/.caddy"
volumes:
  Certfiles:

# ./caddy/Dockerfile
ARG caddybuilderversion
ARG caddyversion

FROM caddy:${caddybuilderversion} AS builder

RUN xcaddy build \
    --with github.com/caddy-dns/cloudflare

FROM caddy:${caddyversion}

COPY --from=builder /usr/bin/caddy /usr/bin/caddy

d. My complete Caddyfile or JSON config:

# ./caddy/config/Caddyfile
{
  debug # per the instructions
  email personal-email@example.com # Redacted my email address
  acme_ca https://acme-staging-v02.api.letsencrypt.org/directory
}

(encoding) {
  encode zstd gzip
}

(dnschallenge) {
  tls personal-email@example.com { # Redacted my email address
    dns cloudflare {Redacted API key}
  }
}

(common) {
  import encoding
  import dnschallenge
}

http://sclark.xyz, https://sclark.xyz {
  redir https://www.sclark.xyz{uri}
}

www.sclark.xyz {
  import common

  root * /var/www
  file_server
}

3. The problem I’m having:

Caddy is not able to start with the above configuration and I don’t know why.

I used to have caddy working and had used it for six months or so. I really thought I got the hang of it. I moved and the server has been turned off since May. Just tried to turn it back on the other day and it hasn’t worked.

I have a bunch of services and manage them with the docker-compose.yml. I go through and update the version and run docker-compose up -d every once in a while to keep them updated. This is a stripped down version to demonstrate the problem.

4. Error messages and/or full log output:

Creating network "sclark_default" with the default driver
Creating caddyserver ... done
Attaching to caddyserver
caddyserver    | {"level":"info","ts":1631240253.378472,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":"caddyfile"}
caddyserver    | {"level":"info","ts":1631240253.3801055,"logger":"admin","msg":"admin endpoint started","address":"tcp/localhost:2019","enforce_origin":false,"origins":["localhost:2019","[::1]:2019","127.0.0.1:2019"]}
caddyserver    | {"level":"info","ts":1631240253.3803442,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc00037b110"}
caddyserver    | {"level":"info","ts":1631240253.3803976,"logger":"http","msg":"server is listening only on the HTTP port, so no automatic HTTPS will be applied to this server","server_name":"srv0","http_port":80}
caddyserver    | {"level":"info","ts":1631240253.380441,"logger":"http","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv1","https_port":443}
caddyserver    | {"level":"info","ts":1631240253.3804607,"logger":"http","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv1"}
caddyserver    | {"level":"warn","ts":1631240253.3804817,"logger":"http","msg":"user server is listening on same interface as automatic HTTP->HTTPS redirects; user-configured routes might override these redirects","server_name":"srv0","interface":"tcp/:80"}
caddyserver    | {"level":"info","ts":1631240253.380939,"logger":"http","msg":"enabling automatic TLS certificate management","domains":["www.sclark.xyz","sclark.xyz"]}
caddyserver    | {"level":"info","ts":1631240253.38198,"logger":"tls","msg":"cleaned up storage units"}
caddyserver    | {"level":"info","ts":1631240253.389078,"msg":"autosaved config","file":"/config/caddy/autosave.json"}
caddyserver    | {"level":"info","ts":1631240253.3890939,"msg":"serving initial configuration"}
caddyserver    | {"level":"info","ts":1631240253.3893764,"logger":"tls.obtain","msg":"acquiring lock","identifier":"sclark.xyz"}
caddyserver    | {"level":"info","ts":1631240253.3895674,"logger":"tls.obtain","msg":"lock acquired","identifier":"sclark.xyz"}
caddyserver    | {"level":"info","ts":1631240253.3899245,"logger":"tls.issuance.acme","msg":"waiting on internal rate limiter","identifiers":["sclark.xyz"]}
caddyserver    | {"level":"info","ts":1631240253.3899317,"logger":"tls.issuance.acme","msg":"done waiting on internal rate limiter","identifiers":["sclark.xyz"]}
caddyserver    | {"level":"info","ts":1631240253.7842014,"logger":"tls.issuance.acme.acme_client","msg":"trying to solve challenge","identifier":"sclark.xyz","challenge_type":"tls-alpn-01","ca":"https://acme-staging-v02.api.letsencrypt.org/directory"}
caddyserver    | {"level":"error","ts":1631240254.4169948,"logger":"tls.issuance.acme.acme_client","msg":"challenge failed","identifier":"sclark.xyz","challenge_type":"tls-alpn-01","status_code":400,"problem_type":"urn:ietf:params:acme:error:dns","error":"No valid IP addresses found for sclark.xyz"}
caddyserver    | {"level":"error","ts":1631240254.4170365,"logger":"tls.issuance.acme.acme_client","msg":"validating authorization","identifier":"sclark.xyz","error":"authorization failed: HTTP 400 urn:ietf:params:acme:error:dns - No valid IP addresses found for sclark.xyz","order":"https://acme-staging-v02.api.letsencrypt.org/acme/order/16429146/499875248","attempt":1,"max_attempts":3}
caddyserver    | {"level":"info","ts":1631240255.5188823,"logger":"tls.issuance.acme.acme_client","msg":"trying to solve challenge","identifier":"sclark.xyz","challenge_type":"http-01","ca":"https://acme-staging-v02.api.letsencrypt.org/directory"}
caddyserver    | {"level":"error","ts":1631240257.0420775,"logger":"tls.issuance.acme.acme_client","msg":"challenge failed","identifier":"sclark.xyz","challenge_type":"http-01","status_code":400,"problem_type":"urn:ietf:params:acme:error:dns","error":"No valid IP addresses found for sclark.xyz"}
caddyserver    | {"level":"error","ts":1631240257.0421429,"logger":"tls.issuance.acme.acme_client","msg":"validating authorization","identifier":"sclark.xyz","error":"authorization failed: HTTP 400 urn:ietf:params:acme:error:dns - No valid IP addresses found for sclark.xyz","order":"https://acme-staging-v02.api.letsencrypt.org/acme/order/16429146/499875358","attempt":2,"max_attempts":3}
caddyserver    | {"level":"error","ts":1631240258.2028787,"logger":"tls.obtain","msg":"will retry","error":"[sclark.xyz] Obtain: [sclark.xyz] solving challenges: sclark.xyz: no solvers available for remaining challenges (configured=[http-01 tls-alpn-01] offered=[http-01 dns-01 tls-alpn-01] remaining=[dns-01]) (order=https://acme-staging-v02.api.letsencrypt.org/acme/order/16429146/499875458) (ca=https://acme-staging-v02.api.letsencrypt.org/directory)","attempt":1,"retrying_in":60,"elapsed":4.813302395,"max_duration":2592000}

Those last logs just kinda loop

# ...
caddyserver    | {"level":"info","ts":1631240318.495699,"logger":"tls.issuance.acme.acme_client","msg":"trying to solve challenge","identifier":"sclark.xyz","challenge_type":"http-01","ca":"https://acme-staging-v02.api.letsencrypt.org/directory"}
caddyserver    | {"level":"error","ts":1631240319.434231,"logger":"tls.issuance.acme.acme_client","msg":"challenge failed","identifier":"sclark.xyz","challenge_type":"http-01","status_code":400,"problem_type":"urn:ietf:params:acme:error:dns","error":"No valid IP addresses found for sclark.xyz"}
caddyserver    | {"level":"error","ts":1631240319.4342926,"logger":"tls.issuance.acme.acme_client","msg":"validating authorization","identifier":"sclark.xyz","error":"authorization failed: HTTP 400 urn:ietf:params:acme:error:dns - No valid IP addresses found for sclark.xyz","order":"https://acme-staging-v02.api.letsencrypt.org/acme/order/16429146/499879568","attempt":1,"max_attempts":3}
caddyserver    | {"level":"info","ts":1631240320.5380478,"logger":"tls.issuance.acme.acme_client","msg":"trying to solve challenge","identifier":"sclark.xyz","challenge_type":"tls-alpn-01","ca":"https://acme-staging-v02.api.letsencrypt.org/directory"}
caddyserver    | {"level":"error","ts":1631240320.882537,"logger":"tls.issuance.acme.acme_client","msg":"challenge failed","identifier":"sclark.xyz","challenge_type":"tls-alpn-01","status_code":400,"problem_type":"urn:ietf:params:acme:error:dns","error":"No valid IP addresses found for sclark.xyz"}
caddyserver    | {"level":"error","ts":1631240320.8825984,"logger":"tls.issuance.acme.acme_client","msg":"validating authorization","identifier":"sclark.xyz","error":"authorization failed: HTTP 400 urn:ietf:params:acme:error:dns - No valid IP addresses found for sclark.xyz","order":"https://acme-staging-v02.api.letsencrypt.org/acme/order/16429146/499879668","attempt":2,"max_attempts":3}
caddyserver    | {"level":"error","ts":1631240322.05715,"logger":"tls.obtain","msg":"will retry","error":"[sclark.xyz] Obtain: [sclark.xyz] solving challenges: sclark.xyz: no solvers available for remaining challenges (configured=[http-01 tls-alpn-01] offered=[http-01 dns-01 tls-alpn-01] remaining=[dns-01]) (order=https://acme-staging-v02.api.letsencrypt.org/acme/order/16429146/499879858) (ca=https://acme-staging-v02.api.letsencrypt.org/directory)","attempt":2,"retrying_in":120,"elapsed":68.667573384,"max_duration":2592000}

5. What I already tried:

Well, stripping things down. Like cutting out the other services I used to run (Gitea, Bookstack, Jellyfin, Wireguard, and another static site on blog.). Updating versions of Caddy and the OS packages. I recreated my API key with Cloudflare to make sure it didn’t expire. I already got blocked from the prod Let’s Encrypt endpoint so I switched to stage for the time being.

Since it says “No valid IP addresses found for sclark.xyz” I tried adding the resolvers subdirective to the tls directive I have but I get another error in that case:

To be clear, I mean that making it looks like

(dnschallenge) {
  tls personal-email@example.com { # Redacted my email address
    dns cloudflare {Redacted API key}
    resolvers 1.1.1.1
  }
}

yields this error

caddyserver    | run: adapting config using caddyfile: parsing caddyfile tokens for 'tls': /etc/caddy/Caddyfile:13 - Error during parsing: unknown subdirective: resolvers

Which just doesn’t make any sense to me since it seems like a valid subdirective according to the docs and is recommended on the cloudflare dns module GitHub page for another issue.

I don’t really know what else to try and it’s been like this for two days.

6. Links to relevant resources:

The last weird thing I can think of is that this documentation page doesn’t tell me enough to use it. It doesn’t look like a directive I can use in a Caddyfile and doesn’t match what is provided in the README on GitHub for that module.

francislavoie · September 10, 2021, 3:38am

Caddy is still trying to issue a certificate for sclark.xyz but you didn’t tell it to use the DNS challenge for that domain, only for www.sclark.xyz.

Let’s Encrypt is saying it couldn’t solve the HTTP challenge because there’s no DNS A record for that domain, i.e. no IP address to route to.

So I think all you need to do is add import dnschallenge to your other site block and Caddy should issue a cert for sclark.xyz using the DNS challenge.

sclarkxyz · September 10, 2021, 1:34pm

Altering the sclark.xyz site block to look like this:

http://sclark.xyz, https://sclark.xyz {
  import common
  redir https://www.sclark.xyz{uri}
}

fixed it for me. Thanks for that, I don’t know why that started causing issues now.

In any case, part of my problems seem to be that the DNS on my own computer is all messed up. I’ll have to dig in to that later, and I’ll probably get to restore my previous config.

Thanks so much for the help, I feel ridiculous.

system · October 10, 2021, 2:44am

This topic was automatically closed after 30 days. New replies are no longer allowed.