DNS challenge with Namecheap and split horizon DNS

Help
1

1. Caddy version (caddy version):

v2.2.0-rc.1

2. How I run Caddy:

Docker, based on caddy:2 image from Docker Hub.

Caddy binary has been custom built with lego-deprecated in order to use Namecheap with DNS challenge.

My goal is to run split horizon DNS so intranet services are not exposed to the internet at large. Internally, *.internal.chris-cartwright.com resolves to the computer running the Caddy container. Externally, it resolves to a VM hosted on LunaNode.

Dockerfile 

# Go 1.15 does not work with one or more of the dependencies.
# Link is only tangentally related, but fixed the issue.
# https://github.com/syncthing/syncthing/issues/6889#issuecomment-672791645
FROM golang:1.14-buster AS builder

RUN go get -u github.com/caddyserver/xcaddy/cmd/xcaddy

# Need to use v2.2.0-rc.1 to work around a pinning issue with a dependency.
# The tag can be removed once Caddy v2.2.0 is released.
# https://github.com/caddy-dns/cloudflare/issues/11
RUN xcaddy build v2.2.0-rc.1 \
    --with github.com/caddy-dns/lego-deprecated

FROM caddy:2 AS runner

COPY --from=builder /go/caddy /usr/bin/caddy

docker-compose.yml 

version: "3"

services:
  caddy:
    image: ccartwright/caddy
    container_name: caddy
    networks:
      - web
      - default
    volumes:
      - ./data/:/etc/caddy/
      - ./files/:/usr/share/caddy/
    environment:
      - "NAMECHEAP_API_USER=CCartwright"
      - "NAMECHEAP_API_KEY=*****"
    ports:
      - 80:80
      - 443:443

networks:
  web:
    external: true

The web network was created with docker network create web. Services I wish to proxy are added to this network.

Caddyfile 

# The Caddyfile is an easy way to configure your Caddy web server.
#
# Unless the file starts with a global options block, the first
# uncommented line is always the address of your site.
#
# To use your own domain name (with automatic HTTPS), first make
# sure your domain's A/AAAA DNS records are properly pointed to
# this machine's public IP, then replace the line below with your
# domain name.
# :80

{
    debug
    acme_ca https://acme-staging-v02.api.letsencrypt.org/directory
    email cartwright.christopher@gmail.com
    admin :2019
}

# Set this path to your site's directory.
#root * /usr/share/caddy

# Enable the static file server.
# file_server

whoami1.internal.chris-cartwright.com {
    tls {
        dns lego_deprecated namecheap
    }

    reverse_proxy whoami-1_whoami_1:80
}

# Another common task is to set up a reverse proxy:
#reverse_proxy whoami1.internal.chris-cartwright.com {
#    to whoami-1_whoami_1:80
#}

#reverse_proxy whoami2.internal.chris-cartwright.com {
#    to whoami-2_whoami_1:9090
#}

# Or serve a PHP site through php-fpm:
# php_fastcgi localhost:9000

#tls {
#    dns lego_deprecated namecheap
#}

# Refer to the Caddy docs for more information:
# https://caddyserver.com/docs/caddyfile

3. The problem I’m having:

The TXT record for the domain is not created, which causes the Let’s Encrypt validation to fail. Credentials are confirmed working via Traefik. It could create the TXT record, but didn’t work for me because of other reasons.

4. Error messages and/or full log output:

{"level":"info","ts":1598659581.950414,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":"caddyfile"}
{"level":"info","ts":1598659581.9514906,"logger":"admin","msg":"admin endpoint started","address":"tcp/:2019","enforce_origin":false,"origins":[":2019"]}
{"level":"warn","ts":1598659581.9515183,"logger":"admin","msg":"admin endpoint on open interface; host checking disabled","address":"tcp/:2019"}
{"level":"info","ts":1598659581.9527543,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc0002fb9d0"}
{"level":"info","ts":1598659582.457994,"logger":"http","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443}
{"level":"info","ts":1598659582.458037,"logger":"http","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
{"level":"debug","ts":1598659582.45882,"logger":"http","msg":"starting server loop","address":"[::]:443","http3":false,"tls":true}
{"level":"debug","ts":1598659582.4588857,"logger":"http","msg":"starting server loop","address":"[::]:80","http3":false,"tls":false}
{"level":"info","ts":1598659582.458891,"logger":"http","msg":"enabling automatic TLS certificate management","domains":["whoami1.internal.chris-cartwright.com"]}
{"level":"info","ts":1598659582.4589417,"logger":"tls","msg":"cleaned up storage units"}
{"level":"info","ts":1598659582.4589846,"msg":"autosaved config","file":"/config/caddy/autosave.json"}
{"level":"info","ts":1598659582.4589891,"msg":"serving initial configuration"}
{"level":"info","ts":1598659582.4591625,"logger":"tls.obtain","msg":"acquiring lock","identifier":"whoami1.internal.chris-cartwright.com"}
{"level":"info","ts":1598659582.4594145,"logger":"tls.obtain","msg":"lock acquired","identifier":"whoami1.internal.chris-cartwright.com"}
{"level":"debug","ts":1598659582.8578024,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"GET","url":"https://acme-staging-v02.api.letsencrypt.org/directory","headers":{"User-Agent":["Caddy/2.2.0-rc.1 CertMagic acmez (linux; amd64)"]},"status_code":200,"response_headers":{"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["724"],"Content-Type":["application/json"],"Date":["Sat, 29 Aug 2020 00:06:22 GMT"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]}}
{"level":"debug","ts":1598659582.9345744,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"HEAD","url":"https://acme-staging-v02.api.letsencrypt.org/acme/new-nonce","headers":{"User-Agent":["Caddy/2.2.0-rc.1 CertMagic acmez (linux; amd64)"]},"status_code":200,"response_headers":{"Cache-Control":["public, max-age=0, no-cache"],"Date":["Sat, 29 Aug 2020 00:06:22 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["0001TmtJ7Lka2f0-SaDBH6uH5v1Nxb33qmPfKciYR1Soufk"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]}}
{"level":"debug","ts":1598659583.0256824,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-staging-v02.api.letsencrypt.org/acme/new-acct","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.2.0-rc.1 CertMagic acmez (linux; amd64)"]},"status_code":201,"response_headers":{"Boulder-Requester":["15376743"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["334"],"Content-Type":["application/json"],"Date":["Sat, 29 Aug 2020 00:06:22 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\"","<https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf>;rel=\"terms-of-service\""],"Location":["https://acme-staging-v02.api.letsencrypt.org/acme/acct/15376743"],"Replay-Nonce":["0002nOoaQ2nxsDOTKUbnUZ6uD0JMc-eQq-CfIqQsUFna48o"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]}}
{"level":"info","ts":1598659583.0283735,"logger":"tls.issuance.acme","msg":"waiting on internal rate limiter","identifiers":["whoami1.internal.chris-cartwright.com"]}
{"level":"info","ts":1598659583.028401,"logger":"tls.issuance.acme","msg":"done waiting on internal rate limiter","identifiers":["whoami1.internal.chris-cartwright.com"]}
{"level":"debug","ts":1598659583.1193461,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-staging-v02.api.letsencrypt.org/acme/new-order","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.2.0-rc.1 CertMagic acmez (linux; amd64)"]},"status_code":201,"response_headers":{"Boulder-Requester":["15376743"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["381"],"Content-Type":["application/json"],"Date":["Sat, 29 Aug 2020 00:06:23 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Location":["https://acme-staging-v02.api.letsencrypt.org/acme/order/15376743/138836523"],"Replay-Nonce":["0002ZkoQybwmCpYAO96sMWtkJwPUWd3RhJKCkde_enYYHJY"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]}}
{"level":"debug","ts":1598659583.1988206,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/101510630","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.2.0-rc.1 CertMagic acmez (linux; amd64)"]},"status_code":200,"response_headers":{"Boulder-Requester":["15376743"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["836"],"Content-Type":["application/json"],"Date":["Sat, 29 Aug 2020 00:06:23 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["0002lM6qTkpmkRxypjtaWOJVvl1s-3_joKLtytB2-55HJtQ"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]}}
{"level":"debug","ts":1598659583.1989594,"logger":"tls.issuance.acme.acme_client","msg":"no solver configured","challenge_type":"tls-alpn-01"}
{"level":"debug","ts":1598659583.198965,"logger":"tls.issuance.acme.acme_client","msg":"no solver configured","challenge_type":"http-01"}
{"level":"info","ts":1598659583.1989684,"logger":"tls.issuance.acme.acme_client","msg":"trying to solve challenge","identifier":"whoami1.internal.chris-cartwright.com","challenge_type":"dns-01","ca":"https://acme-staging-v02.api.letsencrypt.org/directory"}
{"level":"debug","ts":1598659589.4328656,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-staging-v02.api.letsencrypt.org/acme/chall-v3/101510630/YxrjIA","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.2.0-rc.1 CertMagic acmez (linux; amd64)"]},"status_code":200,"response_headers":{"Boulder-Requester":["15376743"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["191"],"Content-Type":["application/json"],"Date":["Sat, 29 Aug 2020 00:06:29 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\"","<https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/101510630>;rel=\"up\""],"Location":["https://acme-staging-v02.api.letsencrypt.org/acme/chall-v3/101510630/YxrjIA"],"Replay-Nonce":["0002wheQBRMTCrTmZjzS1JVbw47ckYdfuXZWqPzZrhKlsAU"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]}}
{"level":"debug","ts":1598659589.4329436,"logger":"tls.issuance.acme.acme_client","msg":"challenge accepted","identifier":"whoami1.internal.chris-cartwright.com","challenge_type":"dns-01"}
{"level":"debug","ts":1598659589.7647457,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/101510630","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.2.0-rc.1 CertMagic acmez (linux; amd64)"]},"status_code":200,"response_headers":{"Boulder-Requester":["15376743"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["659"],"Content-Type":["application/json"],"Date":["Sat, 29 Aug 2020 00:06:29 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["00027kQ_WHBiR5ldi_0piVtJJ22-C7HGyaISfoBnkS9Kr4s"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]}}
{"level":"error","ts":1598659590.6208777,"logger":"tls.issuance.acme.acme_client","msg":"challenge failed","identifier":"whoami1.internal.chris-cartwright.com","challenge_type":"dns-01","status_code":400,"problem_type":"urn:ietf:params:acme:error:dns","error":"DNS problem: NXDOMAIN looking up TXT for _acme-challenge.whoami1.internal.chris-cartwright.com - check that a DNS record exists for this domain"}
{"level":"error","ts":1598659590.6209447,"logger":"tls.issuance.acme.acme_client","msg":"validating authorization","identifier":"whoami1.internal.chris-cartwright.com","error":"authorization failed: HTTP 400 urn:ietf:params:acme:error:dns - DNS problem: NXDOMAIN looking up TXT for _acme-challenge.whoami1.internal.chris-cartwright.com - check that a DNS record exists for this domain","order":"https://acme-staging-v02.api.letsencrypt.org/acme/order/15376743/138836523","attempt":1,"max_attempts":3}
{"level":"debug","ts":1598659591.717692,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-staging-v02.api.letsencrypt.org/acme/new-order","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.2.0-rc.1 CertMagic acmez (linux; amd64)"]},"status_code":201,"response_headers":{"Boulder-Requester":["15376743"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["380"],"Content-Type":["application/json"],"Date":["Sat, 29 Aug 2020 00:06:31 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Location":["https://acme-staging-v02.api.letsencrypt.org/acme/order/15376743/138836604"],"Replay-Nonce":["0002dGxgN5rzeRbrAgu_ejoGTkfTwJq5wGA_FsWkkmG-cMI"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]}}
{"level":"debug","ts":1598659591.7980237,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/101510704","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.2.0-rc.1 CertMagic acmez (linux; amd64)"]},"status_code":200,"response_headers":{"Boulder-Requester":["15376743"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["836"],"Content-Type":["application/json"],"Date":["Sat, 29 Aug 2020 00:06:31 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["0001IZCTZTEW2ED-nqWEFTnSBYzJWjoYU1lVVXNoKGZHCsk"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]}}
{"level":"debug","ts":1598659591.7981493,"logger":"tls.issuance.acme.acme_client","msg":"no solver configured","challenge_type":"tls-alpn-01"}
{"level":"debug","ts":1598659591.7981727,"logger":"tls.issuance.acme.acme_client","msg":"no solver configured","challenge_type":"http-01"}
{"level":"debug","ts":1598659591.8842742,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/101510704","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.2.0-rc.1 CertMagic acmez (linux; amd64)"]},"status_code":200,"response_headers":{"Boulder-Requester":["15376743"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["840"],"Content-Type":["application/json"],"Date":["Sat, 29 Aug 2020 00:06:31 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["0002o-bezinnSIQIqN6VGNg0HEVk-1rEjI9cul2MmToxNOs"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]}}
{"level":"error","ts":1598659591.884386,"logger":"tls.obtain","msg":"will retry","error":"[whoami1.internal.chris-cartwright.com] Obtain: [whoami1.internal.chris-cartwright.com] solving challenges: whoami1.internal.chris-cartwright.com: no solvers available for remaining challenges (configured=[dns-01] offered=[http-01 dns-01 tls-alpn-01] remaining=[http-01 tls-alpn-01]) (order=https://acme-staging-v02.api.letsencrypt.org/acme/order/15376743/138836604) (ca=https://acme-staging-v02.api.letsencrypt.org/directory)","attempt":1,"retrying_in":60,"elapsed":9.4249488,"max_duration":2592000}

5. What I already tried:

Not sure what else I can try. It just doesn’t work and doesn’t provide useful output. I don’t see any log entries regarding Namecheap or the TXT record. Caddy seems to jump right to asking Let’s Encrypt perform their verifications.

I could not find alternatives to lego-deprecated that support Namecheap.

6. Links to relevant resources:

Caddy 2 can support all 75+ DNS providers - basic info
GitHub - caddy-dns/lego-deprecated: (DEPRECATED) DNS modules so Caddy can solve the ACME DNS challenge with over 75 providers - basic info
Caddy2 with DNS challenge and Namecheap in Docker - other issues, unresolved

1 Like
2

This is fixed on the master branch. If you build from master instead of v2.2.0-rc.1, then you should no longer have that issue.

FYI you’ll want to persist /data in the container, i.e. caddy_data:/data for example, which is where the certificates and keys are stored. If you don’t persist that directory, then you’d be throwing those away whenever you restart the container.

Caddy since v2.2 no longer uses lego behind the scenes, it now uses GitHub - mholt/acmez: Premier ACME client library for Go which is a “sort-of” fork of lego to better fit Caddy’s needs. The lego-deprecated plugin should still work for now though.

I don’t know what the issue is here, I’ll need to defer to @matt when he finds the time to look into it further.

3

To clarify, acmez is a complete rewrite with a novel code base and design. I only borrowed some DNS utility code from lego.

Split horizon DNS is supported on the latest commits to master if you configure the right DNS resolver to use for the DNS challenge. Lego configures this from an env variable. The new code bases don’t rely on env vars so you’d configure it in your config file.

4

Thanks for the note about /data! I would have been very confused when my certificates disappeared.

5

Image has been updated to use Go 1.15 and master. Ran it again for fun and have the same result as my original post. Where/how would I configure which resolver to use? I don’t see anything in the documentation for Caddy or lego-deprecated.

6

The docs haven’t been updated yet, but it was just added in this commit:

https://github.com/caddyserver/caddy/commit/24f34780b61fc82df6507cf95535c3dfc9f78058

It would look something like this:

tls {
	issuer acme {
		resolvers <dns_server>
	}
}

And this new issuer config is added to the tls directive:

1 Like
7

I am also interested in testing split DNS, in my case with Cloudflare handling the external zone. I’m building master caddy with go 1.15 and xcaddy like this:

xcaddy build master --with github.com/caddy-dns/cloudflare

I get this version string from caddy version: v2.2.0-rc.1.0.20200831192326-24f34780b61f, so it appears the build includes the correct commit.
I’m using caddy adapt to make sure I get the syntax right in my test Caddyfile. When I try this Caddyfile:

test1.carloscarrasco.com {
    tls {
        ca https://acme-staging-v02.api.letsencrypt.org/directory
        dns cloudflare XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
        issuer acme {
            resolvers 1.0.0.1
        }
    }
    respond "testing"
}

I get this error:

> ./caddy adapt
2020/09/01 18:41:17.356 INFO    using adjacent Caddyfile
adapt: parsing caddyfile tokens for 'tls': Caddyfile:13 - Error during parsing: when defining an issuer, all its config must be in its block, rather than from separate tls subdirectives

I only have one tls directive in the Caddyfile so I am at a loss at what to do. I tried removing everything in the tls except for the issuer acme block, and it complains with the same error.

1 Like
8

Ah, this is a new error, because if you use issuer at the same time as the tls's ca option, they are in conflict. Instead, do this:

    tls {
        dns cloudflare XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
        issuer acme {
            dir https://acme-staging-v02.api.letsencrypt.org/directory
            resolvers 1.0.0.1
        }
    }

The syntax is documented in the code here, for now:

https://github.com/caddyserver/caddy/commit/24f34780b61fc82df6507cf95535c3dfc9f78058#diff-5bfc542cc13143ff34aa90c370f81a49L212-L223

By the way, thanks for trying this out before it hits stable, it’s very helpful to get early feedback on this stuff!

Edit: Wait a sec, I think this might still not work because of a conflict between dns and issuer as well. I’ll come back with another edit after checking.

1 Like
9

Oh, sorry, this is still an undocumented feature that is coming out with Caddy 2.2.

If you use the issuer subdirective you cannot use the other “shortcut” subdirectives of tls like ca (and dns too if I recall correctly) because it becomes ambiguous how to merge the configs and which one to use.

What @francislavoie suggested is on the right track - just move all the issuer config into the issuers subdirective. Sorry for the confusion. And he’s right, it’ll need a patch to support dns within the issuers block.

10

Alright yeah, confirmed that it doesn’t work yet cause dns also conflicts, and dns wasn’t yet configurable inside of issuer acme so I wrote a quick PR to fix:

https://github.com/caddyserver/caddy/pull/3701

11

I would like to build caddy with your branch to test this change with my config. Is there a way to tell xcaddy to use your branch?

12

I’m not sure, actually. I was hoping @matt could merge it asap :sweat_smile:

1 Like
13

Sorry, will get on that shortly! Now that I’ve finished up work with a client for the past few weeks I’m hit with a holiday weekend, but this week should start getting back to normal.

1 Like
14

Logs from a trial run:

{"level":"info","ts":1599599280.591688,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":"caddyfile"}
{"level":"info","ts":1599599280.592885,"logger":"admin","msg":"admin endpoint started","address":"tcp/:2019","enforce_origin":false,"origins":[":2019"]}
{"level":"warn","ts":1599599280.592921,"logger":"admin","msg":"admin endpoint on open interface; host checking disabled","address":"tcp/:2019"}
{"level":"info","ts":1599599280.5931556,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc000c0d5e0"}
{"level":"info","ts":1599599281.007829,"logger":"http","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443}     
{"level":"info","ts":1599599281.007873,"logger":"http","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
{"level":"debug","ts":1599599281.008058,"logger":"http","msg":"starting server loop","address":"[::]:443","http3":false,"tls":true}
{"level":"debug","ts":1599599281.0081055,"logger":"http","msg":"starting server loop","address":"[::]:80","http3":false,"tls":false}
{"level":"info","ts":1599599281.0081127,"logger":"http","msg":"enabling automatic TLS certificate management","domains":["whoami1.internal.chris-cartwright.com"]}
{"level":"info","ts":1599599281.009753,"msg":"autosaved config","file":"/config/caddy/autosave.json"}
{"level":"info","ts":1599599281.009777,"msg":"serving initial configuration"}
{"level":"info","ts":1599599281.0175982,"logger":"tls","msg":"cleaned up storage units"}
{"level":"info","ts":1599599281.047593,"logger":"tls.obtain","msg":"acquiring lock","identifier":"whoami1.internal.chris-cartwright.com"}
2020/09/08 21:08:01 [INFO][FileStorage:/data/caddy] Lock for 'cert_acme_whoami1.internal.chris-cartwright.com_acme-staging-v02.api.letsencrypt.org-directory' is stale (created: 2020-09-01 04:18:36.2737634 +0000 UTC, last update: 2020-09-01 04:52:15.514304 +0000 UTC); removing then retrying: /data/caddy/locks/cert_acme_whoami1.internal.chris-cartwright.com_acme-staging-v02.api.letsencrypt.org-directory.lock
{"level":"info","ts":1599599281.0725226,"logger":"tls.obtain","msg":"lock acquired","identifier":"whoami1.internal.chris-cartwright.com"}
{"level":"info","ts":1599599281.0816188,"logger":"tls.issuance.acme","msg":"waiting on internal rate limiter","identifiers":["whoami1.internal.chris-cartwright.com"]}
{"level":"info","ts":1599599281.081658,"logger":"tls.issuance.acme","msg":"done waiting on internal rate limiter","identifiers":["whoami1.internal.chris-cartwright.com"]}
{"level":"debug","ts":1599599281.5027473,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"GET","url":"https://acme-staging-v02.api.letsencrypt.org/directory","headers":{"User-Agent":["Caddy/2.2.0-rc.1.0.20200908173646-00e6b77fe4eb CertMagic acmez (linux; amd64)"]},"status_code":200,"response_headers":{"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["724"],"Content-Type":["application/json"],"Date":["Tue, 08 Sep 2020 21:07:59 GMT"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]}}
{"level":"debug","ts":1599599281.5829754,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"HEAD","url":"https://acme-staging-v02.api.letsencrypt.org/acme/new-nonce","headers":{"User-Agent":["Caddy/2.2.0-rc.1.0.20200908173646-00e6b77fe4eb CertMagic acmez (linux; amd64)"]},"status_code":200,"response_headers":{"Cache-Control":["public, max-age=0, no-cache"],"Date":["Tue, 08 Sep 2020 21:07:59 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["0001VfaZf1xgOwdZIrf_vv2HLS-LfBsTRE3NRU9XbWimg7Q"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]}}
{"level":"debug","ts":1599599281.6851752,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-staging-v02.api.letsencrypt.org/acme/new-order","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.2.0-rc.1.0.20200908173646-00e6b77fe4eb CertMagic acmez (linux; amd64)"]},"status_code":201,"response_headers":{"Boulder-Requester":["15420394"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["381"],"Content-Type":["application/json"],"Date":["Tue, 08 Sep 2020 21:07:59 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Location":["https://acme-staging-v02.api.letsencrypt.org/acme/order/15420394/145517759"],"Replay-Nonce":["0001Bcf7OMm4xuKOaM33eDxw8nLWJKgR7XKz97hP7QAYT1U"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]}}
{"level":"debug","ts":1599599281.7679696,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/108026596","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.2.0-rc.1.0.20200908173646-00e6b77fe4eb CertMagic acmez (linux; amd64)"]},"status_code":200,"response_headers":{"Boulder-Requester":["15420394"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["836"],"Content-Type":["application/json"],"Date":["Tue, 08 Sep 2020 21:07:59 GMT"],"Link":["<https://acme-staging-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["00026WtzyLbq0eMnP7lMj3zrkxr63MLmliMbb6i9VVHdFJ8"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]}}
{"level":"debug","ts":1599599281.7682176,"logger":"tls.issuance.acme.acme_client","msg":"no solver configured","challenge_type":"http-01"}
{"level":"debug","ts":1599599281.768244,"logger":"tls.issuance.acme.acme_client","msg":"no solver configured","challenge_type":"tls-alpn-01"}
{"level":"info","ts":1599599281.7682495,"logger":"tls.issuance.acme.acme_client","msg":"trying to solve challenge","identifier":"whoami1.internal.chris-cartwright.com","challenge_type":"dns-01","ca":"https://acme-staging-v02.api.letsencrypt.org/directory"}

Side note: Line 14 (“Lock for 'cert_acme_whoami1”…) is not formatted correctly. It may be using a different logger.

Updated Caddyfile (comments removed):

{
    debug
    acme_ca https://acme-staging-v02.api.letsencrypt.org/directory
    email cartwright.christopher@gmail.com
    admin :2019
}

whoami1.internal.chris-cartwright.com {
    tls {
        issuer acme {
            dir https://acme-staging-v02.api.letsencrypt.org/directory
            dns lego_deprecated namecheap
            resolvers 1.1.1.1
        }
    }

    reverse_proxy whoami-1_whoami_1:80
}

It appears something is still not working correctly. Version string looks correct to me: 00e6b77fe4eb. I don’t see the TXT record on the Namecheap side, but I’m not sure if lego_deprecated cleans up after itself.

15

My guess is this is coming from the namecheap DNS provider, which isn’t under Caddy’s control. It’s a downside of needing to use the lego_deprecated plugin.

That said, I don’t understand what the issue is. I don’t see any errors there. What are you saying is the problem exactly?

16

@francislavoie

Caddy doesn’t seem to be creating the TXT record for Let’s Encrypt. I don’t see anything in the logs and I don’t see the record in the Namecheap dashboard.

17

I’ve compiled caddy master again with:

> xcaddy build master --with github.com/caddy-dns/cloudflare

Version commit appears to be correct:

> ./caddy version
v2.2.0-rc.1.0.20200908173646-00e6b77fe4ebh1:l5u8lY5VzBmu0cBjZ7dTXbCg3bAAapjQxzhJs+AjlVs=

But given this Caddyfile as suggested:

test1.carloscarrasco.com {
    tls {
        dns cloudflare XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
        issuer acme {
            dir https://acme-staging-v02.api.letsencrypt.org/directory
            resolvers 1.0.0.1
        }
    }
    respond "testing"
}

I still get the same error:

> ./caddy adapt
2020/09/09 11:15:54.867 INFO    using adjacent Caddyfile
adapt: parsing caddyfile tokens for 'tls': Caddyfile:8 - Error during parsing: when defining an issuer, all its config must be in its block, rather than from separate tls subdirectives

I then moved the dns directive into the issuer block, as this:

test1.carloscarrasco.com {
    tls {
        issuer acme {
            dir https://acme-staging-v02.api.letsencrypt.org/directory
            dns cloudflare XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
            resolvers 1.0.0.1
        }
    }
    respond "testing"
}

And this time the Caddyfile is accepted, but the resulting JSON does not mention any custom resolver:

./caddy adapt
2020/09/09 11:25:16.581 INFO    using adjacent Caddyfile
{"apps":{"http":{"servers":{"srv0":{"listen":[":443"],"routes":[{"match":[{"host":["test1.carloscarrasco.com"]}],"handle":[{"handler":"subroute","routes":[{"handle":[{"body":"testing","handler":"static_response"}]}]}],"terminal":true}]}}},"tls":{"automation":{"policies":[{"subjects":["test1.carloscarrasco.com"],"issuer":{"ca":"https://acme-staging-v02.api.letsencrypt.org/directory","challenges":{"dns":{"provider":{"api_token":"XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX","name":"cloudflare"}}},"module":"acme"}}]}}}}

So I guess that’s why the resolver is still not working.

18

Ah, interesting… I’ll look into this.

19

@carloscm Thanks for testing. I’ve pushed a simple fix for that in caddytls: Fix resolvers option of acme issuer (Caddyfile) · caddyserver/caddy@9859ab8 · GitHub. Can you confirm with that commit? I was able to see the resolvers be added to the JSON after that.

1 Like
20

Yes, it works!caddy adapt now outputs a JSON when the resolvers is directive inside the issuer block, with the corresponding resolvers array in the JSON string.
Outside of the block it gives a Error during parsing: unknown subdirective: resolvers. I guess it will be just an issue of documentation from now on.

After this issue with the configuration, the problem now appears to be that the resolver is not being used. I’ve fired wireshark to confirm this, and I can see the successive DNS SOA queries being sent to the local LAN resolver (which is doing the split DNS internally), rather than to 1.0.0.1.

2 Likes