"404 - The given domain is not registered" with GoDaddy and Caddy 2

Help
1

1. Caddy version (caddy version): 2.2.1

2. How I run Caddy:

a. System environment:

Docker container on CentOS 7

b. Command:

Spun up via ansible-playbook

c. Service/unit/compose file: This is just a test setup.

version: "3.7"

networks:
  wordpress:
    external: yes

services:
  caddy2:
    build:
      context: /tmp/docker-caddy2
      dockerfile: /tmp/docker-caddy2/Dockerfile-builder
    restart: unless-stopped
    container_name: staging-caddy2
    hostname: staging-caddy2
    networks:
      wordpress:
    volumes:
      - /opt/docker-caddy2/Caddyfile:/etc/caddy/Caddyfile:ro
      - /opt/docker-caddy2/acme:/root/.caddy/acme:z
      - /etc/ssl/priv/fullchain.crt:/root/.caddy/cert.crt:ro
      - /etc/ssl/priv/fullchain.key:/root/.caddy/cert.key:ro
    ports:
      - 80:80
      - 443:443
    environment:
      ENABLE_TELEMETRY: "false"
      ACME_AGREE: "true"
      GODADDY_API_SECRET: "redacted"
      GODADDY_API_KEY: "redacted"
    healthcheck:
      test: wget --spider -q http://127.0.0.1/status || exit 1
      interval: 30s

Dockerfile:

FROM caddy:2.2.1-builder AS builder

RUN xcaddy build \
    --with github.com/caddy-dns/lego-deprecated
FROM caddy:2.2.1-alpine

COPY --from=builder /usr/bin/caddy /usr/bin/caddy

d. My complete Caddyfile or JSON config:

I’m migrating from Caddy 1 so this may have leftovers from that. (E. g. haven’t looked up the ACME directory yet.)

{
  acme_ca https://acme-staging-v02.api.letsencrypt.org/directory
  email admins@bw-labor.de
}

127.0.0.1:80 {
  respond /status 200
}

www.staging.bw-labor.de {
  header / {
    Strict-Transport-Security "max-age=31536000;"
    X-XSS-Protection "1; mode=block"
    X-Frame-Options allow-from false
  }
  @blacklist_1 {
    not remote_ip 192.168.0.0/17 10.253.0.0/16
  }
  respond /wp-admin @blacklist_1 403
  reverse_proxy / staging-wordpress:80
  tls {
    dns lego_deprecated godaddy
  }
}

3. The problem I’m having:

I’m trying to use the lego_deprecated DNS plugin for Caddy 2.2.1 to set up certificate generation for internal-only servers. Our domain is bw-labor.de, referenced as @ because otherwise your very intelligent forum software won’t let me post this because it has too many links. The subdomain in question is *.staging.@. @ is hosted with GoDaddy, but also on an internal PowerDNS server. (I know you shouldn’t have two separate “authoritative” DNS servers but can’t change that.)

Internally, *.staging.@ is a CNAME for staging.@. staging.@ is an A record for the server Caddy is running on (in a Docker container). Externally, staging.@ shouldn’t exist, but I tried setting an A record for it without any change. The Caddy server can not be reached from the outside.

The goal is to be able to just throw any virtualhost on the Caddy server as necessary, e. g. www.staging.@, shop.staging.@ etc without having to set up individual RRs for them, but still being able to generate valid certificates.

I’m not sure where the problem is since the documentation on all involved components is extremely sparse (or maybe I’m just too daft). I’m not sure what exactly lego does in the process of submitting the correct request, what GoDaddy expects of it or what LE expects of the resulting RRs.

I’d be willing to part with the wildcard idea even though it’d be neat, but since it doesn’t even work with staging.example.com itself (which is a normal A record internally and externally), I don’t know how to proceed. For now, all I need are valid certificates for anything under the staging subdomain.

4. Error messages and/or full log output:

{
  "level": "info",
  "ts": 1603437519.5953393,
  "logger": "tls.issuance.acme.acme_client",
  "msg": "trying to solve challenge",
  "identifier": "www.staging.bw-labor.de",
  "challenge_type": "dns-01",
  "ca": "https://acme-staging-v02.api.letsencrypt.org/directory"
}
{
  "level": "error",
  "ts": 1603437521.0568206,
  "logger": "tls.issuance.acme.acme_client",
  "msg": "cleaning up solver",
  "identifier": "www.staging.bw-labor.de",
  "challenge_type": "dns-01",
  "error": "godaddy: failed to get TXT records: could not get records: Domain: staging.bw-labor.de; Record: _acme-challenge.www, Status: 404; Body: {\"code\":\"UNKNOWN_DOMAIN\",\"message\":\"The given domain is not registered, or does not have a zone file\"}\n"
}
{
  "level": "error",
  "ts": 1603437521.2265606,
  "logger": "tls.obtain",
  "msg": "will retry",
  "error": "[www.staging.bw-labor.de] Obtain: [www.staging.bw-labor.de] solving challenges: presenting for challenge: godaddy: failed to get TXT records: could not get records: Domain: staging.bw-labor.de; Record: _acme-challenge.www, Status: 404; Body: {\"code\":\"UNKNOWN_DOMAIN\",\"message\":\"The given domain is not registered, or does not have a zone file\"}\n (order=https://acme-staging-v02.api.letsencrypt.org/acme/order/16256985/171395567) (ca=https://acme-staging-v02.api.letsencrypt.org/directory)",
  "attempt": 5,
  "retrying_in": 600,
  "elapsed": 614.639372671,
  "max_duration": 2592000
}

5. What I already tried:

As described above, setting the A record externally. Also tried using staging.@ directly instead of www.staging.@, but same result.

{
  "level": "info",
  "ts": 1603438264.6010396,
  "logger": "tls.issuance.acme.acme_client",
  "msg": "trying to solve challenge",
  "identifier": "staging.bw-labor.de",
  "challenge_type": "dns-01",
  "ca": "https://acme-staging-v02.api.letsencrypt.org/directory"
}
{
  "level": "error",
  "ts": 1603438266.0431566,
  "logger": "tls.issuance.acme.acme_client",
  "msg": "cleaning up solver",
  "identifier": "staging.bw-labor.de",
  "challenge_type": "dns-01",
  "error": "godaddy: failed to get TXT records: could not get records: Domain: staging.bw-labor.de; Record: _acme-challenge, Status: 404; Body: {\"code\":\"UNKNOWN_DOMAIN\",\"message\":\"The given domain is not registered, or does not have a zone file\"}\n"
}
{
  "level": "error",
  "ts": 1603438266.2152994,
  "logger": "tls.obtain",
  "msg": "will retry",
  "error": "[staging.bw-labor.de] Obtain: [staging.bw-labor.de] solving challenges: presenting for challenge: godaddy: failed to get TXT records: could not get records: Domain: staging.bw-labor.de; Record: _acme-challenge, Status: 404; Body: {\"code\":\"UNKNOWN_DOMAIN\",\"message\":\"The given domain is not registered, or does not have a zone file\"}\n (order=https://acme-staging-v02.api.letsencrypt.org/acme/order/16257239/171399908) (ca=https://acme-staging-v02.api.letsencrypt.org/directory)",
  "attempt": 1,
  "retrying_in": 60,
  "elapsed": 3.412579701,
  "max_duration": 2592000
}

6. Links to relevant resources:

Uh… Github issue? "404 - The given domain is not registered" with GoDaddy and Caddy 2 · Issue #1274 · go-acme/lego · GitHub

2

I’m not sure I can say much to help with the DNS plugin, but your Docker volumes aren’t correct for Caddy v2. The paths you should persist are /data and /config. I think the ones you have now are from v1. Please take another look at the Docker docs :+1:

Also those two first env vars do nothing in v2 (there’s no telemetry, and agree is now implicit)

Only hunch I have, from looking at GoDaddy DDNS Updater - Page 1 | TeaNazaR.comTeaNazaR.com from a quick Google, there’s a comment below that mentions about a mail subdomain that GoDaddy expects just the subdomain and not the whole domain for that API call. I have to delegate to @matt for that though, I don’t know enough about what can be done about that.

The other option if you want to give it a shot, you could port the GoDaddy DNS provider to libdns, i.e. the new interfaces Caddy supports.

3

Thanks for the response! Yes, I suspected that some things are incorrect there, the Let’s Encrypt functionality is just the first thing I’m testing due to a request from higher up. The Ansible role generating them is almost unchanged from v1, but it shouldn’t matter in context. Thanks for pointing out the mistakes, though!

And I just tested it on the command line with somewhat surprising results. This is what I used:

curl -X GET -H"Authorization: sso-key $GDKEY:$GDSEC"
    "https://api.godaddy.com/v1/domains/staging.bw-labor.de/records/TXT/_acme-challenge.www"

This 404s like the plugin. If I use the root domain and _acme-challenge.www.staging instead, the correct RR is returned. However, on update:

curl -X PUT -H"Content-Type: application/json"
    -d '[{"data":"test"}]' -H"Authorization: sso-key $GDKEY:$GDSEC"
    "https://api.godaddy.com/v1/domains/staging.bw-labor.de/records/TXT/_acme-challenge.www"

The record does get created, but as _acme-challenge.www.bw-labor.de. No 404. If I use the root domain instead, it does get created correctly.

Seems to be a bit of an inconsistent API… but the first case is likely the issue. However, question is why the name including staging is recognized as being the domain, that makes little sense to me.

As for the libdns provider - I unfortunately don’t speak Go (and I’m not a programmer) and I assume that’d be a bit of a larger project. Or is that assumption wrong?

4

As for the libdns provider - I unfortunately don’t speak Go (and I’m not a programmer) and I assume that’d be a bit of a larger project. Or is that assumption wrong?

It would largely be copy-paste of the existing GoDaddy DNS provider code, plus some extra glue code to make it work with libdns instead. But yeah, if you’re not a programmer, likely not a task for you.

If you need support on this for business reasons, I recommend you reach out to https://caddyserver.com/business and see if they can organize something for you (e.g. pay to have someone set up the new plugin and help you get it working)

5

I figured it out - but it’s still failing - with a combination of tcpdump and looking at the source code, there goes my Friday evening… the way it arrives at the domain value is just looping through the domain name and cutting it off level by level, then looking for a SOA record there. What I didn’t mention - sorry for that - is that internally, staging.bw-labor.de has its own SOA record and externally it doesn’t. All I had to do was add an external DNS server to the docker-compose.yml and everything started to work up to a point.

Now, it unfortunately just fails to set the record silently. That is, I never get any log messages from the GoDaddy API, just from the LE API that tells me the TXT record doesn’t exist. I can see that it finds the correct SOA record and then gets busy with an Akamai server which appears to be the GoDaddy API. Unfortunately, I can’t look into these packets, of course, but the record is never created according to my management portal.

I don’t suppose there’s any way of adding debugging output without forking at least lego-deprecated and lego, right?

Edit: Just found out what an absolute horror forking a Go module is… especially if you have zero clue about Go.

  {
    "level": "info",
    "ts": 1603482074.220456,
    "logger": "tls.issuance.acme.acme_client",
    "msg": "trying to solve challenge",
    "identifier": "www.staging.bw-labor.de",
    "challenge_type": "dns-01",
    "ca": "https://acme-staging-v02.api.letsencrypt.org/directory"
  },
  {
    "level": "error",
    "ts": 1603482082.8506875,
    "logger": "tls.issuance.acme.acme_client",
    "msg": "challenge failed",
    "identifier": "www.staging.bw-labor.de",
    "challenge_type": "dns-01",
    "status_code": 400,
    "problem_type": "urn:ietf:params:acme:error:dns",
    "error": "DNS problem: NXDOMAIN looking up TXT for _acme-challenge.www.staging.bw-labor.de - check that a DNS record exists for this domain"
  },
  {
    "level": "error",
    "ts": 1603482082.8513207,
    "logger": "tls.issuance.acme.acme_client",
    "msg": "validating authorization",
    "identifier": "www.staging.bw-labor.de",
    "error": "authorization failed: HTTP 400 urn:ietf:params:acme:error:dns - DNS problem: NXDOMAIN looking up TXT for _acme-challenge.www.staging.bw-labor.de - check that a DNS record exists for this domain",
    "order": "https://acme-staging-v02.api.letsencrypt.org/acme/order/16266445/171647668",
    "attempt": 1,
    "max_attempts": 3
  },
  {
    "level": "error",
    "ts": 1603482084.391076,
    "logger": "tls.obtain",
    "msg": "will retry",
    "error": "[www.staging.bw-labor.de] Obtain: [www.staging.bw-labor.de] solving challenges: www.staging.bw-labor.de: no solvers available for remaining challenges (configured=[dns-01] offered=[http-01 dns-01 tls-alpn-01] remaining=[http-01 tls-alpn-01]) (order=https://acme-staging-v02.api.letsencrypt.org/acme/order/16266445/171647757) (ca=https://acme-staging-v02.api.letsencrypt.org/directory)",
    "attempt": 1,
    "retrying_in": 60,
    "elapsed": 11.990071837,
    "max_duration": 2592000
  }
6

Finally. I was able to intercept the traffic with mitmproxy and the plugin appears to do everything correctly. GoDaddy’s API even responds with the entry after it has been created - but unfortunately, it doesn’t appear to propagate to their DNS servers in time.

After the API response, the plugin doesn’t check that (not sure if it should) but instead immediately makes a request with Let’sEncrypt. Then it gets the entire list of TXT records, deletes the entry from it and uploads the list again.

Unfortunately, that means I can’t share the dump but it’s very clear now. Letting nslookup run in a loop during the process using the authoritative NS confirms that it’s never being returned. I’m not sure whose fault that is but all things point to GoDaddy…

2 Likes
7

That wouldn’t surprise me. GoDaddy aren’t well known for their quality of APIs :stuck_out_tongue:

8

Awesome, thanks for drilling down.

Caddy (actually CertMagic) should be checking propagation before it initiates the challenge with Let’s Encrypt:

If that isn’t working, I’d like to find out why…

9

To debug that, I unfortunately lack the skills. Here’s a screenshot of mitmdump and tcpdump src <Caddy-IP> and port 53 side-by-side, though:

(I was first about to submit it as text but realize that it would be even less informative. This way, you can see there is no DNS request when there should be one when comparing the output of mitmdump.)

10

This topic was automatically closed after 30 days. New replies are no longer allowed.