Cloudflare DNS: Propagation fails due NXDOMAIN

Help
1

Hello People of the Forums, I’m new to Caddy and I thinks it’s an awesome Solution to replace my current Reverse Prox setup for my Smarthome (HomeAssistant) which is makes use of Traefik. (As it’s much easier to configure). But currently I won’t get the DNS Challenge working and I hope you can help me. More information below, thanks in advance, any kind of help is appreciated!

1. Caddy version (2.11):

with Cloudflare DNS as written here.

2. How I run Caddy:

I’m running Caddy through an Home-Assistant Addon which makes use of Docker.
I’m using a binary with Cloudflare DNS

a. System environment:

Ubuntu 18.04.4, Docker - HomeAssistant

b. Command:

none

c. Service/unit/compose file:

Only the Enviroment Variable for the API Token
log-level: debug

d. My complete Caddyfile or JSON config:

my.domain.de {
	tls {
		dns cloudflare {env.CLOUDFLARE_API_TOKEN}
	}
	header {
		Strict-Transport-Security "max-age=31536000; includeSubdomains"
		X-XSS-Protection "1; mode=block"
		X-Content-Type-Options "nosniff"
		X-Frame-Options "SAMEORIGIN"
		Referrer-Policy "same-origin"
		-Server
	}
	encode gzip zstd
	reverse_proxy localhost:8123
}

3. The problem I’m having:

Dns Propagation fails and I get an NXDOMAIN error. I’m not sure if Cloudflare even receives the TXT entry. The certificate is only for local use and this is why I use the DNS Challenge, I don’t want to expose any ports.

4. Error messages and/or full log output:

2020/08/03 18:53:50 [INFO] [my.domain.de] acme: Waiting for DNS record propagation.
2020/08/03 18:53:59 [INFO] [my.domain.de] acme: Cleaning DNS-01 challenge
2020/08/03 18:54:00 [INFO] Deactivating auth: https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/someauth
2020/08/03 18:54:00 [INFO] Unable to deactivate the authorization: https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/someauth
2020/08/03 18:54:00 [ERROR] error: one or more domains had a problem:
[my.domain.de] acme: error: 400 :: urn:ietf:params:acme:error:dns :: DNS problem: NXDOMAIN looking up TXT for _acme-challenge.my.domain.de - check that a DNS record exists for this domain, url: 
 (challenge=dns-01 remaining=[])

5. What I already tried:

  • Checked the Cloudflare API Permissions, should be allright.
  • Created a DNS entry (A-Record) for the subdomain as I thought the DNS Validation needed an entry
  • Tried it after removing the fancy schmancy header and encode stuff
  • Read all threads that are related to Cloudflare DNS Challenge (that google gave me, I hope lol :slight_smile: )

6. Links to relevant resources:

Hope this everything relevant you guys need to help me out :slight_smile:

tenor

2

Hi Henrik, welcome –

I think this is fixed with the latest version on master as of… like, Friday, so try building from source (or download the latest build artifacts) and see if that helps!

3

Unfortunately still not working rebuilt the whole thing with xcaddy, anything else I could do?
I have a local dns rewrite could this maybe cause the problem? Or is there anything i could do wrong on the Cloudflare side?
Thanks for your fast answer @matt :slight_smile:

4

It should be working in the latest. (Cloudflare works for me, I use it myself.) Can you verify your Caddy version for me and be sure that you are running the version you think you are?

5

Output when i run the build command:

grafik
grafik860×241 3.53 KB

Command:
sudo xcaddy build --with github.com/caddy-dns/cloudflare
I also tried to download the newest version + dns module through your download section (which gives me the same file size), same error.

Or is there a command I can run through Docker which gives you more information?

EDIT: ok tried caddy version through docker
v2.1.1 h1:X9k1+ehZPYYrSqBvf/ocUgdLSRIuiNiMo7CvyGUQKeA=

6

Ah right, so you need to build from the latest commit; by default, the go tooling (wrapped by xcaddy) will only use the latest release. So you want xcaddy build 68529e2f9ecb6b5cb9552482d73b8337252c7f59 --with github.com/caddy-dns/cloudflare (as of this morning, although any commit from the weekend should work too).

Right now you’re still using v2.1.1 which does not have the fix.

7

Thanks again Matt!
Probably not one of your issues but still confusing.
So as written above I’m using Caddy through an HomeAssistant Addon.
What’s strange for me is that as you can see in the log files, it says version 2.1.2-0, but when i run caddy version through docker it says again 2.11. So now I don’t know if this due caddy or the Docker “HomeAssistant Addon” (it provides an option to use your own binary). Also I’m still not seeing any changes on the Cloudflare records.

Would be awesome if you could exclude that it’s caddy’s fault for the wrong version number.

INFO: Found custom Caddy
v2.1.2-0.20200803164242-68529e2f9ecb h1:KzUG0R7zyfK2DLb5h4ZCmpUcHG5A6MHz3fN4AExZEuE=
INFO: Caddyfile found
{"level":"info","ts":1596479845.2290487,"msg":"using provided configuration","config_file":"/share/caddy/Caddyfile","config_adapter":""}
{"level":"info","ts":1596479845.2353516,"logger":"admin","msg":"admin endpoint started","address":"tcp/localhost:2019","enforce_origin":false,"origins":["localhost:2019","[::1]:2019","127.0.0.1:2019"]}
{"level":"info","ts":1596479845.2357266,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc000512000"}
{"level":"info","ts":1596479845.2359543,"logger":"http","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443}
{"level":"info","ts":1596479845.2359815,"logger":"http","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
{"level":"info","ts":1596479845.2364848,"logger":"http","msg":"enabling automatic TLS certificate management","domains":["my.domain.de"]}
{"level":"info","ts":1596479845.2367368,"logger":"tls","msg":"cleaned up storage units"}
{"level":"info","ts":1596479845.2376146,"msg":"autosaved config","file":"/data/caddy/autosave.json"}
{"level":"info","ts":1596479845.237637,"msg":"serving initial configuration"}
{"level":"info","ts":1596479845.2589157,"logger":"tls.obtain","msg":"acquiring lock","identifier":"my.domain.de"}
2020/08/03 20:37:25 [INFO][FileStorage:/ssl/caddy] Lock for 'cert_acme_my.domain.de_acme-v02.api.letsencrypt.org-directory' is stale (created: 2020-08-03 20:30:21.589727073 +0200 CEST, last update: 2020-08-03 20:33:21.615617489 +0200 CEST); removing then retrying: /ssl/caddy/locks/cert_acme_my.domain.de_acme-v02.api.letsencrypt.org-directory.lock
{"level":"info","ts":1596479845.2602267,"logger":"tls.obtain","msg":"lock acquired","identifier":"my.domain.de"}
{"level":"info","ts":1596479845.296871,"logger":"tls.issuance.acme","msg":"waiting on internal rate limiter","identifiers":["my.domain.de"]}
{"level":"info","ts":1596479845.2969065,"logger":"tls.issuance.acme","msg":"done waiting on internal rate limiter","identifiers":["my.domain.de"]}
{"level":"info","ts":1596479846.6952558,"logger":"tls.issuance.acme.acme_client","msg":"trying to solve challenge","identifier":"my.domain.de","challenge_type":"dns-01"}
{"level":"error","ts":1596479855.6996667,"logger":"tls.issuance.acme.acme_client","msg":"challenge failed","identifier":"my.domain.de","challenge_type":"dns-01","status_code":400,"problem_type":"urn:ietf:params:acme:error:dns","error":"DNS problem: NXDOMAIN looking up TXT for _acme-challenge.my.domain.de - check that a DNS record exists for this domain"}
{"level":"error","ts":1596479855.6997533,"logger":"tls.issuance.acme.acme_client","msg":"validating authorization","identifier":"my.domain.de","error":"authorization failed: HTTP 400 urn:ietf:params:acme:error:dns - DNS problem: NXDOMAIN looking up TXT for _acme-challenge.my.domain.de - check that a DNS record exists for this domain","attempt":1,"max_attempts":3}
{"level":"error","ts":1596479857.3161683,"logger":"tls.obtain","msg":"will retry","error":"[my.domain.de] Obtain: [my.domain.de] solving challenges: my.domain.de: no solvers available for remaining challenges (configured=[dns-01] offered=[http-01 dns-01 tls-alpn-01] remaining=[http-01 tls-alpn-01])","attempt":1,"retrying_in":60,"elapsed":12.055809915,"max_duration":2592000}
8

v2.1.2 basically means that you did correctly build from source (latest tag +0.0.1). You can see following that the timestamp of the build, and 68529e2f9ecb which is the commit hash it was built from.

1 Like
9

We generally ask that users not obfuscate their domains because it just makes it harder for us to help you debug. Domains are public information due to OCSP so it only complicates things to omit it.

10

That’s a tell-tale sign that you’re not running the same binary. (What is 2.11 though? We haven’t released 2.11 or 2.10 or anything even CLOSE to that.)

As for 2.1.2-* that’s just how the Go module versions work, it’s related to Minimal Version Selection: Modules · golang/go Wiki · GitHub - since you’re building on a commit newer than v2.1.1, it uses the pseudo-version 2.1.2 with a timestamp and commit suffix. Edit: better link! Go Modules Reference - The Go Programming Language

Pseudo-versions

A pseudo-version is a specially formatted pre-release version that encodes information about a specific revision in a version control repository. For example, v0.0.0-20191109021931-daa7c04131f5 is a pseudo-version.

Pseudo-versions may refer to revisions for which no semantic version tags are available. They may be used to test commits before creating version tags, for example, on a development branch.

Each pseudo-version has three parts:

  • A base version prefix ( vX.0.0 or vX.Y.Z-0 ), which is either derived from a semantic version tag that precedes the revision or vX.0.0 if there is no such tag.
  • A timestamp ( yymmddhhmmss ), which is the UTC time the revision was created. In Git, this is the commit time, not the author time.
  • A revision identifier ( abcdefabcdef ), which is a 12-character prefix of the commit hash, or in Subversion, a zero-padded revision number.

Each pseudo-version may be in one of three forms, depending on the base version. These forms ensure that a pseudo-version compares higher than its base version, but lower than the next tagged version.

  • vX.0.0-yyyymmddhhmmss-abcdefabcdef is used when there is no known base version. As with all versions, the major version X must match the module’s major version suffix.
  • vX.Y.Z-pre.0.yyyymmddhhmmss-abcdefabcdef is used when the base version is a pre-release version like vX.Y.Z-pre .
  • vX.Y.(Z+1)-0.yyyymmddhhmmss-abcdefabcdef is used when the base version is a release version like vX.Y.Z . For example, if the base version is v1.2.3 , a pseudo-version might be v1.2.4-0.20191109021931-daa7c04131f5 .

More than one pseudo-version may refer to the same commit by using different base versions. This happens naturally when a lower version is tagged after a pseudo-version is written.

These forms give pseudo-versions two useful properties:

  • Pseudo-versions with known base versions sort higher than those versions but lower than other pre-release for later versions.
  • Pseudo-versions with the same base version prefix sort chronologically.

The go command performs several checks to ensure that module authors have control over how pseudo-versions are compared with other versions and that pseudo-versions refer to revisions that are actually part of a module’s commit history.

  • If a base version is specified, there must be a corresponding semantic version tag that is an ancestor of the revision described by the pseudo-version. This prevents developers from bypassing minimal version selection using a pseudo-version that compares higher than all tagged versions like v1.999.999-99999999999999-daa7c04131f5 .
  • The timestamp must match the revision’s timestamp. This prevents attackers from flooding module proxies with an unbounded number of otherwise identical pseudo-versions. This also prevents module consumers from changing the relative ordering of versions.
  • The revision must be an ancestor of one of the module repository’s branches or tags. This prevents attackers from referring to unapproved changes or pull requests.

Anyway, make sure all your Caddy binaries are using the latest version; v2.1.1 does not have the fix.

11

My bad, typo. Meant to be 2.1.1. Anyway thanks for your time, I will check why I’m not running the desired version. (Even though I swapped the binary) I understand that you can’t give me support for this third party usage case. (HomeAssistant)

12

I can’t debug home assistant, true, I can only help you make sure you’ve got the Caddy stuff right. :+1:

So this is resolved then?

13

Yes, thanks for your time! I’ll send you some money for a coffee😃

1 Like
15

Yep, all publicly-validated ACME certificates you can currently solve challenges for need to have publicly accessible DNS records for verification. For DNS challenges this is a TXT record.

That said, assuming you’ve got Caddy using the Cloudflare DNS provider, it should be updating their public name server anyway when carrying out this challenge, so it seems like that’s where something’s going wrong.

1 Like
16

Hmm, yeah, so Caddy should only continue with the DNS challenge once it has performed its own authoritative lookup to verify that the record is visible – but this is only a guess, and it’s ultimately up to the ACME server to decide whether it sees it from its vantage point. In those logs, the ACME server is responding that it can’t find the TXT record required to validate the domain.

I’m not sure how it’s possible for Caddy to see it and for Let’s Encrypt to not see it as in this case, but maybe something else funky is going on. (Make sure you test with the staging endpoint, not production.)

17

This topic was automatically closed after 30 days. New replies are no longer allowed.