Too many failed authorizations recently

[Anwar_Hussain]

1. The problem I’m having:

I have a domain hosting platform that has hundreds of domains. When renewing SSL certificates sometimes I get “too many failed authorizations recently” error message from let’s encrypt.

2. Error messages and/or full log output:

{
    "level": "error",
    "ts": 1690368347.5156703,
    "logger": "tls.obtain",
    "msg": "could not get certificate from issuer",
    "identifier": "www.#####.###",
    "issuer": "acme-v02.api.letsencrypt.org-directory",
    "error": "HTTP 429 urn:ietf:params:acme:error:rateLimited - Error creating new order :: too many failed authorizations recently: see https://letsencrypt.org/docs/failed-validation-limit/"
}

3. Caddy version:

2.4.5

4. How I installed and ran Caddy:

a. System environment:

AWS Fargate, Linux, ECS

b. Command:

It's a Fargate container so I don't need to run any command. I just have provided my docker image made out of caddy:2.4.5

c. Service/unit/compose file:

local Dockerfile, using that I build my dcoker image

FROM caddy:2.4.5

COPY Caddyfile /etc/caddy/Caddyfile

ENV SslValidation ${SslValidation}
ENV ViewerEndpoint ${ViewerEndpoint}
ENV SitemapEndpoint ${SitemapEndpoint}
ENV DashboardEndpoint ${DashboardEndpoint}

EXPOSE 80
EXPOSE 443

d. My complete Caddy config:

{
  on_demand_tls {
    ask {env.SslValidation}
  }
}

:443 {
  header Server "Server_name"
  header -x-powered-by
  
  @trailing_slash {
    path_regexp no_slash (.+)\/$
  }
  @domain {
    header_regexp domain host ^www\.(.+)$
  }

  redir @domain https://{http.regexp.domain.1}{uri}
  redir @trailing_slash {re.no_slash.1} 308

  tls {
    on_demand
  }

  handle_path /dashboard {
    reverse_proxy {env.DashboardEndpoint}
  }

  handle_path /dashboard/* {
    reverse_proxy {env.DashboardEndpoint}
  }

  reverse_proxy {env.ViewerEndpoint}
}

5. Links to relevant resources:

[matt (Matt Holt)]

Are you persisting your certificate storage so Caddy can reuse the certs it obtains?

Let’s Encrypt has rate limits for how many certs they’ll issue repeatedly. Make sure to reuse certs you obtain the first time.

[Anwar_Hussain]

Our initial setup did not have persistence enabled from the container. Is the error happening because of this?

[matt (Matt Holt)]

Yep, absolutely.

[Anwar_Hussain]

Correct me if I am wrong. Docker container will contain all the downloaded certs until the next restart, I haven’t restarted the container for quite a while. In that case, why is it a problem with persistency?

[francislavoie]

@Anwar_Hussain

3. Caddy version:

2.4.5

That’s a very old version. Please upgrade to the latest, v2.6.4.

@Anwar_Hussain

reverse_proxy {env.DashboardEndpoint}

I recommend using reverse_proxy {$DashboardEndpoint} instead which would make certain aspects of the proxy work better since the value won’t need to be replaced at runtime every time it’s read.

@Anwar_Hussain

  handle_path /dashboard {
    reverse_proxy {env.DashboardEndpoint}
  }

  handle_path /dashboard/* {
    reverse_proxy {env.DashboardEndpoint}
  }

You don’t need both of these. Just do handle_path /dashboard* which covers both cases.

@Anwar_Hussain

reverse_proxy {env.ViewerEndpoint}

I recommend wrapping this in a handle (with no matcher) so it’s properly mutually exclusive from the other routes.

@Anwar_Hussain

  @trailing_slash {
    path_regexp no_slash (.+)\/$
  }
  @domain {
    header_regexp domain host ^www\.(.+)$
  }

Both of these can be one-liners:

  @trailing_slash path_regexp no_slash (.+)\/$
  @domain header_regexp domain host ^www\.(.+)$

@matt

Are you persisting your certificate storage so Caddy can reuse the certs it obtains?

Make sure you persist the /data volume.

[Anwar_Hussain]

Thank you @matt and @francislavoie for your help.