How to force renewal of Let's Encrypt certificates

Help
21

i upgraded to 2.4.3 and the server i took out of the load balancer is down, it doesn’t serve request anymore. After i replaced caddy with the new version and reloaded the service i saw this error:

6 [ERROR] While deleting old OCSP staples, unable to load staple file: unable to decrypt data for ocsp/
many of them for each domain.

After that, i couldn’t access any domain on the server. the logs don’t show any error. I even tried to put back the old 2.3 version, not working. I restearted caddy, not working

btw, we use proxy-protocol and redis storage if that helps

22

Just wondering… why not upgrade to the latest version?

Sounds like a problem from the redis plugin. You’ll have to ask the author about that, I have no clue. GitHub - gamalan/caddy-tlsredis: Redis Storage using for Caddy TLS Data

Looks like caddy is working though

23

yes, that seems to be the issue. will use 2.3 until i find a fix. i’ll set renewal_window_ratio=1 on the old caddy version.

Beginner question: how do I set this value using Caddyfile not json? Or this is not available on 2.3.0? Unfortunately i can’t use other version because of redis issue

24

It’s not configurable via Caddyfile at this time (neither on 2.3.0 or latest) because it has never been relevant for anyone to change.

Did you try to delete the OCSP data from Redis? If they’re deleted from storage, then Caddy will refresh them next time it runs that process.

25

i deleted 2 of them 30 minutes ago, will check 1 hour later.

Later edit: more than 1 hour passed and still nothing in the logs. Can be more than 1 hour on 2.30 version?

Btw, looking at journal log I’m seeing this for each domain every 2 days “advancing OCSP staple”. I hope that’s not when the OCSP is checked.

26

Isn’t possible to push that configuration via API?

27

Yep, absolutely. Even if you’re using a Caddyfile.

:warning: I haven’t tested these instructions, I probably got these slightly wrong, be sure to test in a staging or dev environment first.

First, do a GET /config/ to get Caddy’s current config:

curl "http://localhost:2019/config/"

Then identify the path to the relevant TLS automation policy. It should be at apps/tls/automation/policies/N, where N is an index into the array (0-based as usual).

If you find it (look at subjects for your affected domain name) then you can do:

curl -X PUT \
	-H "Content-Type: application/json" \
	-d '1' \
	"http://localhost:2019/config/apps/tls/automation/policies/N/renewal_window_ratio"

If you don’t see any policies, it’s probably implicit, just create one:

curl -X POST \
	-H "Content-Type: application/json" \
	-d '{"subjects": ["example.com"], "renewal_window_ratio": 1}' \
	"http://localhost:2019/config/apps/tls/automation/policies"

Then be sure to clear the setting when you are done:

curl -X DELETE \
	-H "Content-Type: application/json" \
	"http://localhost:2019/config/apps/tls/automation/policies/N/renewal_window_ratio"
2 Likes
28

Thanks a lot @matt Once this is over I’ll try to convince my company to donate. Right now the only thing that worked is to delete the certificate completely, restart caddy and follow the link to issue a new one.

All other tries with deleting the OCSP and waiting failed (waited 3 hours).

29

The certificates haven’t even been revoked yet. Deleting the OCSP staple will not have any effect.

30

Hi Matt,
This is my output:

Untitled2
Untitled21536×864 10.5 KB

My caddy server version: v2.4.6

i got email from Letsencrypt about expire.
Any help how to make it become auto renewal for letsencrypt?
Or caddy auto make it for me?
Thank you

31

Caddy will do it automatically, you don’t have to do anything.

1 Like
32

Thank you. I tried this but even if it start to renew some certificates, about 10 in 10 minutes, it starts to renew again the same certificates, for ex. one certificate was renewed 3 times. Is that normal?

Also, i tried to set it as 0.995 and after 100 or so slows to 1 per 15 minutes.

33

We were previously using a version of Caddy earlier than v2.4.2 and upgraded to v2.4.6 today. Do you know if Caddy will automatically handle the certificate revocations, even if the certs were issued with the earlier version of Caddy?

34

Yes, there’s nothing inherently different about the certificate data itself, it was just an implementation bug prior to v2.4.2 that caused it to not properly react when it found out about revocation. The relevant commit is here:

1 Like
35

Yes; like I said, turn off the setting immediately after it gets the certificate, otherwise you’ll hit rate limits. It will always renew certificates at every scan with that setting. Not suitable for most deployments tbh.

Just let Caddy replace them for you.

36

it kept renewing the same certificates, i let it 30 minutes and same 10 certificates kept renewing… and we have 1000+

37

Probably because it would always look at those first 10, attempt renewing all of them, then not do any of the rest because Caddy’s internal rate limiter prevented doing more than 10, rinse, repeat.

See more details on how this works here: Automatic HTTPS — Caddy Documentation

38

makes sense. but with 0.995 why does it renew it so slow after renewing about 120 certificates? right now it renews like 5 every hour lately, for the past 4 hours.

39

I don’t think that field supports floating point values, only integers. It probably got converted to an int as 0.

Either way, easiest way to force a renewal is to just remove the certificates from storage and reload Caddy. It’s clear the renewal_window_ratio option isn’t sufficient to change to make all certs get renewed.

40

the doc says it’s a ratio… should we use it like this: 1/3 ?