Cleaning up on-demand certificates

After a long time of running Caddy (v1.0.4 now), I notice the acme/acme-v02.api.letsencrypt.org/sites contains a lot of directories for domains that are no longer hosted on the platform.

Judging from the logs, Caddy is trying to renew the domains that are pointing to other A records and hence fail validation.

Should I just make a script that periodically runs through the sites folder and purges the inactive domains?

We use an “ask” endpoint to validate whether a domain should be allowed SSL, but it seems it’s not used for renewals.

I’m also thinking we’ll run into problems when “sites” contains more than 64000 directories due to limits on ext4. But then it’s probably time to switch to a redis/consul backed store anyway.

1 Like

You’re probably right. That does sound like a good idea to do, so that Caddy can clean up old certs like you say.

Any change in behaviour here is extremely unlikely to land in v1, all development has shifted to v2. The docs for the on_demand/ask functionality in v2 is here: https://caddyserver.com/docs/automatic-https#on-demand-tls

@matt will definitely be more able to shed some light on the specifics than I can, but this definitely sounds like a reasonable feature request if you want to open an issue on github for this.

Certificates that have been expired for more than 14 days should be removed from storage. Caddy will eventually give up trying to renew on-demand certificates as well, and remove them from the cache, when it stops getting requests for that name and when it continues to have errors. You shouldn’t need to run a script.

Can you provide more information – how many certificates are still present, which of them are configured for “on-demand” management, and when they expired? Is your server still seeing requests (technically, ClientHellos with those ServerName indicators) for those domain names?

Aw man, is that really a limit in ext4? I thought we were passed arbitrary restrictions like that :frowning_face:

That would probably be a good idea; I don’t see why we shouldn’t consult the ask endpoint for on-demand renewals too.

Almost everything has arbitrary restrictions.

Some systems (like ZFS) just have restrictions in the billions, so you’d never practically encounter them.