Cloudflare refused

1. The problem I’m having:

Hi All,

Long story short, my caddy server crapped the bed last night. (Completely my fault.) I have tried several things including completely blowing out the server and starting from scratch as well as deleting and recreating my auth token. I keep getting a refused message from Cloudflare. I can’t even reach the default web page for caddy. It keeps forcing the page to https. (Oddly enough I can reach it in Brave which isn’t forcing it to https.)

2. Error messages and/or full log output:

Feb 25 08:08:18 caddy caddy[377]: {"level":"info","ts":1772028498.0802274,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0x26b0bbc72800"}
Feb 25 08:08:18 caddy caddy[377]: {"level":"info","ts":1772028498.083107,"logger":"tls.obtain","msg":"acquiring lock","identifier":"hdhr.7263377.xyz"}
Feb 25 08:08:18 caddy caddy[377]: {"level":"info","ts":1772028498.0856736,"logger":"tls.obtain","msg":"lock acquired","identifier":"hdhr.7263377.xyz"}
Feb 25 08:08:18 caddy caddy[377]: {"level":"info","ts":1772028498.0858102,"logger":"tls.obtain","msg":"obtaining certificate","identifier":"hdhr.7263377.xyz"}
Feb 25 08:08:18 caddy caddy[377]: {"level":"info","ts":1772028498.0864487,"logger":"http","msg":"waiting on internal rate limiter","identifiers":["hdhr.7263377.xyz"],"ca":"https://acme-v02.api.letsencrypt.org/directory","account":""}
Feb 25 08:08:18 caddy caddy[377]: {"level":"info","ts":1772028498.0865183,"logger":"http","msg":"done waiting on internal rate limiter","identifiers":["hdhr.7263377.xyz"],"ca":"https://acme-v02.api.letsencrypt.org/directory","account":""}
Feb 25 08:08:18 caddy caddy[377]: {"level":"info","ts":1772028498.0865674,"logger":"http","msg":"using ACME account","account_id":"https://acme-v02.api.letsencrypt.org/acme/acct/3095846256","account_contact":[]}
Feb 25 08:08:18 caddy caddy[377]: {"level":"info","ts":1772028498.5830033,"msg":"trying to solve challenge","identifier":"hdhr.7263377.xyz","challenge_type":"dns-01","ca":"https://acme-v02.api.letsencrypt.org/directory"}
Feb 25 08:08:21 caddy caddy[377]: {"level":"error","ts":1772028501.9734297,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"hdhr.7263377.xyz","issuer":"acme-v02.api.letsencrypt.org-directory","error":"[hdhr.7263377.xyz] solving challenges: waiting for solver certmagic.solverWrapper to be ready: checking DNS propagation of \"_acme-challenge.hdhr.7263377.xyz.\" (relative=_acme-challenge.hdhr zone=7263377.xyz. resolvers=[192.168.40.2:53 192.168.40.3:53 192.168.40.4:53]): NS carioca.ns.cloudflare.com.:53 returned REFUSED for _acme-challenge.hdhr.7263377.xyz. (order=https://acme-v02.api.letsencrypt.org/acme/order/3095846256/484276946916) (ca=https://acme-v02.api.letsencrypt.org/directory)"}
Feb 25 08:08:21 caddy caddy[377]: {"level":"error","ts":1772028501.9734697,"logger":"tls.obtain","msg":"will retry","error":"[hdhr.7263377.xyz] Obtain: [hdhr.7263377.xyz] solving challenges: waiting for solver certmagic.solverWrapper to be ready: checking DNS propagation of \"_acme-challenge.hdhr.7263377.xyz.\" (relative=_acme-challenge.hdhr zone=7263377.xyz. resolvers=[192.168.40.2:53 192.168.40.3:53 192.168.40.4:53]): NS carioca.ns.cloudflare.com.:53 returned REFUSED for _acme-challenge.hdhr.7263377.xyz. (order=https://acme-v02.api.letsencrypt.org/acme/order/3095846256/484276946916) (ca=https://acme-v02.api.letsencrypt.org/directory)","attempt":1,"retrying_in":60,"elapsed":3.887710134,"max_duration":2592000}
Feb 25 08:09:21 caddy caddy[377]: {"level":"info","ts":1772028561.9739292,"logger":"tls.obtain","msg":"obtaining certificate","identifier":"hdhr.7263377.xyz"}
Feb 25 08:09:21 caddy caddy[377]: {"level":"info","ts":1772028561.9745142,"logger":"http","msg":"using ACME account","account_id":"https://acme-staging-v02.api.letsencrypt.org/acme/acct/269075213","account_contact":[]}
Feb 25 08:09:22 caddy caddy[377]: {"level":"info","ts":1772028562.2821252,"msg":"trying to solve challenge","identifier":"hdhr.7263377.xyz","challenge_type":"dns-01","ca":"https://acme-staging-v02.api.letsencrypt.org/directory"}

3. Caddy version:

v2.11.1 - Cloudflare build downloaded from caddy.

4. How I installed and ran Caddy:

a. System environment:

Debian 13 lxc

b. Command:

apt install -y debian-keyring debian-archive-keyring apt-transport-https curl
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | tee /etc/apt/sources.list.d/caddy-stable.list
chmod o+r /usr/share/keyrings/caddy-stable-archive-keyring.gpg
chmod o+r /etc/apt/sources.list.d/caddy-stable.list
apt update
apt install caddy
cd /tmp
curl -o caddy 'https://caddyserver.com/api/download?os=linux&arch=amd64&p=github.com%2Fcaddy-dns%2Fcloudflare'
systemctl stop caddy
chmod +x caddy
mv caddy /usr/bin/caddy
systemctl start caddy
nano /etc/caddy/.env
    CF_API_TOKEN=REDACTED_TOKEN
chmod 600 /etc/caddy/.env
chown caddy:caddy /etc/caddy/.env
systemctl edit caddy
    [Service]
    EnvironmentFile=/etc/caddy/.env
nano /etc/caddy/Caddyfile
systemctl daemon-reload
systemctl restart caddy

d. My complete Caddy config:

{
        acme_dns cloudflare {env.CF_API_TOKEN}
}
hdhr.7263377.xyz {
        reverse_proxy 192.168.25.4:80
}

Do you have stale records left at Cloudflare?

Should your acme_dns be tls { dns cloudflare ... } now?

By the way, you shouldn’t overwrite the caddy binary in /usr/bin, as it will get replaced when the caddy package is updated. Put it in /usr/local/bin instead, and add that new path to your systemd overrides. eg

ExecStart=
ExecStart=/usr/local/bin/caddy run --environ --config /etc/caddy/Caddyfile

You also don’t need the systemctl daemon-reload, it’s implied after systemctl edit.

1 Like

Thanks for the tips @hmoffatt. I purged cache in Cloudflare and moved the caddy binary to the /usr/local/bin folder but still get the refused error.

I did try changing acme_dns to acme_tls, but that causes caddy to fail to run.

Our recommended steps for diverting the custom binary are here: Build from source — Caddy Documentation

acme_dns is correct @hmoffatt.

Try settings resolvers 1.1.1.1, I think your local machine’s DNS resolvers aren’t seeing your domain.

1 Like

Sorry, yes, I had trouble finding it in the documentation, but I see now. I have apparently not used the global option in any of my configs yet.

I seemed to have fixed my issue by adding the following to each entry in Caddyfile. I tried some lower time outs, but they were inconsistent.

        tls {
                propagation_delay 1h
        }

I’m still having an issue with one device not fully loading once logged in, but I’ll open a separate thread for that.

1 Like