1. Caddy version (caddy version
):
2.4.6
2. How I run Caddy:
a. System environment:
Docker
b. Command:
caddy run --config /dockerapp/caddy/Caddyfile
c. Service/unit/compose file:
d. My complete Caddyfile or JSON config:
{
storage redis {
host {$REDIS_HOST}
}
on_demand_tls {
ask https://www.my-domain.org/custom-domain-check
interval 2m
burst 5
}
debug
}
(SecurityHeaders) {
header_up X-Real-IP {remote_host}
header_up X-Forwarded-Proto {scheme}
}
my-domain.org, *.my-domain.org {
@notStatic {
not file
}
reverse_proxy @notStatic web:3000
request_body {
max_size 100MB
}
log {
output stdout
}
tls me@my-domain.org {
dns route53
}
}
:443, :80 {
@notStatic {
not file
}
reverse_proxy @notStatic web:3000
request_body {
max_size 100MB
}
tls {
on_demand
}
}
3. The problem I’m having:
I’m using Caddy on instances behind a load balancer, to generate on demand tls for a website we host on a client’s custom domain, my-client.org. I tried to write my Caddyfile so that on demand certs are stored in redis, but it’s unclear to me if that’s working since it’s repeatedly “asking” caddy to generate on demand tls for my-client.org.
Visiting my-client.org in the browser, everything appears to work fine - Caddy issues a valid certificate and the page loads over https. But when I look at the Caddy logs, it’s full of errors about reaching rate limits, could not get certificate from issuer, etc (log below). Furthermore, in my web app’s production logs, I see tons of 404 errors for GET requests to https://my-client.org/.well-known/acme-challenge/****
So https://my-client.org appears to be working fine, but the Caddy logs and acme-challenge 404s are telling me that something must be misconfigured. Does anyone have any insights into what’s going wrong and how to fix it?
4. Error messages and/or full log output:
caddy_1 | {"level":"info","ts":1641181016.9673479,"logger":"tls.issuance.acme","msg":"done waiting on internal rate limiter","identifiers":["my-client.org"],"ca":"https://acme-v02.api.letsencrypt.org/directory","account":""}
caddy_1 | {"level":"error","ts":1641181017.2001925,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"my-client.org","issuer":"acme-v02.api.letsencrypt.org-directory","error":"HTTP 429 urn:ietf:params:acme:error:rateLimited - Error creating new order :: too many failed authorizations recently: see https://letsencrypt.org/docs/rate-limits/"}
caddy_1 | {"level":"warn","ts":1641181017.2008576,"logger":"tls.issuance.zerossl","msg":"missing email address for ZeroSSL; it is strongly recommended to set one for next time"}
caddy_1 | {"level":"info","ts":1641181017.86142,"logger":"tls.issuance.zerossl","msg":"generated EAB credentials","key_id":"Wu_IeQsno34jbOSPxDPWwg"}
caddy_1 | {"level":"info","ts":1641181027.7515316,"logger":"tls.issuance.acme","msg":"waiting on internal rate limiter","identifiers":["my-client.org"],"ca":"https://acme.zerossl.com/v2/DV90","account":""}
caddy_1 | {"level":"info","ts":1641181027.7515597,"logger":"tls.issuance.acme","msg":"done waiting on internal rate limiter","identifiers":["my-client.org"],"ca":"https://acme.zerossl.com/v2/DV90","account":""}
03:37:12.238
caddy_1 | {"level":"info","ts":1641181032.2379267,"logger":"tls.issuance.acme.acme_client","msg":"trying to solve challenge","identifier":"my-client.org","challenge_type":"http-01","ca":"https://acme.zerossl.com/v2/DV90"}
caddy_1 | {"level":"info","ts":1641181090.0311844,"logger":"tls.on_demand","msg":"obtaining new certificate","server_name":"my-client.org"}
caddy_1 | {"level":"info","ts":1641181090.0329618,"logger":"tls.obtain","msg":"acquiring lock","identifier":"my-client.org"}
caddy_1 | {"level":"warn","ts":1641181194.9624903,"logger":"tls.issuance.acme.acme_client","msg":"HTTP request failed; retrying","url":"https://acme.zerossl.com/v2/DV90/authz/OUSc4KaAYokoSmloWA9keA","error":"performing request: Post \"https://acme.zerossl.com/v2/DV90/authz/OUSc4KaAYokoSmloWA9keA\": context deadline exceeded"}
caddy_1 | {"level":"error","ts":1641181194.9625244,"logger":"tls.issuance.acme.acme_client","msg":"deactivating authorization","identifier":"my-client.org","authz":"https://acme.zerossl.com/v2/DV90/authz/OUSc4KaAYokoSmloWA9keA","error":"attempt 1: https://acme.zerossl.com/v2/DV90/authz/OUSc4KaAYokoSmloWA9keA: context deadline exceeded"}
caddy_1 | {"level":"error","ts":1641181194.9625866,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"my-client.org","issuer":"acme.zerossl.com-v2-DV90","error":"[my-client.org] solving challenges: [my-client.org] context deadline exceeded (order=https://acme.zerossl.com/v2/DV90/order/CZXYnA-1DDPTqGkw3i4ZnQ) (ca=https://acme.zerossl.com/v2/DV90)"}
caddy_1 | {"level":"error","ts":1641181194.9625986,"logger":"tls.obtain","msg":"will retry","error":"[my-client.org] Obtain: [my-client.org] solving challenges: [my-client.org] context deadline exceeded (order=https://acme.zerossl.com/v2/DV90/order/CZXYnA-1DDPTqGkw3i4ZnQ) (ca=https://acme.zerossl.com/v2/DV90)","attempt":1,"retrying_in":60,"elapsed":177.99758203,"max_duration":2592000}
5. What I already tried:
Reading through documentation and threads in the Caddy forum.