Let's Encrypt Certificate revoked but not renewed

1. Caddy version (caddy version):

v2.4.6 h1

2. How I run Caddy:

Running Nextcloud on Caddy Webserver

a. System environment:

Debian 11

3. The problem I’m having:

The certificate was automatically renewed at the beginning of january. In a web browser like Microsoft Edge or Google Chrome on Windows 10 the certificate is secure. Some days ago i read that Let’s Encrypt was revoking certifiactes.
On an iPhone 13 (iOS 15.2) the certificate is insecure. I have found nothing about renewing (or revoking) certificates with caddy. SSLLABS also says that the certificate

The certificate is valid - but insecure and not trusted.
image

Let’s Encrypt is revoking lots of SSL certificates in two days (bleepingcomputer.com)

How to renew a Let’s Encrypt certificate the right way to solve my problem?

1 Like

The same here - Caddy v2.4.6. At least two servers affected. On one server, however, one certificate was correctly identified as revoked and renewed accordingly, another was not.

The way Caddy finds out about the revocation is via the OCSP responder on the certificate. Caddy caches the in response from the CA, which is signed, and shows that response to the client when it connects. As long as Caddy thinks the certificate is Good due to OCSP, it will continue to use it.

If the client ignores Caddy’s OCSP stapling message (in which Caddy says it’s still good) and does its own revocation checks, then this can happen. I know that Chrome and Firefox should be respecting OCSP stapling, but it’s possible that Safari doesn’t.

OCSP responses are valid for 7 days, and Caddy refreshes it every 3-4 days (halfway through). So it’s possible that Caddy hasn’t found out about the revocation yet, which would trigger a renewal.

If you need to force a renewal, delete the certificate from Caddy’s storage, then reload Caddy.

A post was split to a new topic: Force renewal of certificates

What’s in your Caddy logs? Please fill out the help template completely, you’ve omitted half of it, and we can’t do anything about it until we have enough information. All we can do is guess and that will only waste everyone’s time.

Does this have issue have to do with this? Let's Encrypt is revoking lots of SSL certificates in two days
Nevermind, saw the link posted in OP.

I am having the same issue on one of my Caddy served sites.

Can you indicate where I might find the certificate in Caddy’s storage? Is there a default folder I can look in?

Depends how you’re running Caddy. But in general: Conventions — Caddy Documentation

1 Like

Thank you!
image

Do I delete all 4 files?

And if I can put in a request for Caddy devs, having a specific command to do this would be pretty nifty!

You can delete the whole directory for that domain.

We don’t really want to make it easy, on purpose. This is an extraneous circumstance, and Caddy should automatically recover from this.

But testing for OCSP revocation events is very difficult in code, so bugs happened. And it happens very infrequently because generally CAs do their job well.

1 Like

I wanted to force a renewal by deleting a certificate, but I can’t find where they are - they’ve moved!

I am on Windows, and the certs have always been in %AppData%\Caddy. That location still has certificates, but they are all out of date - they appear to date back to before I installed Caddy v2.4.5 last November.

But Caddy is serving current and up-to-date certificates, which must be stored somewhere else; however, I can’t find them on the disk!

Another reason I want them is because I copy the certificate for my mail server from Caddy (where the webmail is proxied through) to the mail server (where it is used for SMTP and IMAP), and this process has now failed because of the change.

Paul

@pwhodges when you run Caddy with caddy run, use the --environ flag, which will have Caddy print its environment to the logs before starting up. You’ll find the path to the storage location in there. The storage location is entirely dependent on how you’re running Caddy.

Thank you for all your help!

“You can delete the whole directory for that domain.”

image
I’m confused about which directory is the directory for the domain. That is the directory where those 4 files are.

That’s the dir for the local CA that Caddy provisions when you use tls internal. Those are your publicly trusted certs/keys. Those would be in caddy/certificates/...

After some time I have a solution for this problem.

If you are under linux:

 find / -type d -name "*caddy*"

My output was (and many more):

/root/.local/share/caddy
/var/lib/caddy/.local/share/caddy

Deleteting the certificates in /root/.local/share/caddy/pki... did not solve my problem.

So I went through all of those folders and searched for other certificates until i found them in /var/lib/caddy/.local/share/caddy/certificates/acme-v02.api.letsencrypt.org-directory/.
I deleted all of them and restarted the caddy service. Caddy created new certificates for all my domains.

Feature Request
A new function called --renew to force renewing all of the certificates would be really nice.

1 Like

Well, I have Caddy running as a service, and I presume that simply running “Caddy environ” would give the appropriate answer in this instance; that showed that I am looking in the correct place.

However, all my certificate files in that location are either expired, or will expire in about a week - but all my web sites are being served with valid and up-to-date certificates.

I have also searched the whole disk for certificates, and can only find these old ones; I even deleted one (for an unimportant site), and this had no effect. This is so weird I can only think I’m doing something unutterably stupid.

The date of the newest certificate I can see is the date I finished moving my web sites from an old server running Caddy v1 to a new one with Caddy v2. But the old one is now switched off and disconnected, so that can’t be anything to do with it.

Paul

No, running caddy environ will show the environment for your current user, not the user under which Caddy runs.

The caddy run command has an --environ flag which makes Caddy print out the environment in the logs when it starts up. You might need to scroll up a bunch in your logs to find when Caddy started up. The storage location will be written there.

I couldn’t persuade Caddy to tell me anything when running as a Windows service. But that reminded me that the service is running under the “Local System” account, not a user at all. Application to Microsoft documentation showed me that Caddy’s app data in these circumstances is stored in:

 C:\Windows\System32\config\systemprofile\AppData\Roaming\Caddy

As access there is restricted, my search didn’t penetrate there for me to find. As I said, I was being stupid - but yet, this information might be useful to someone!

It may be that my scheduled batch file for copying the certificate might not have access there - but it happens, by a big coincidence, that today there was a new release of my mail server that can for the first time get and renew its own certificate from Let’s Encrypt. It even appears that the way my programs are arranged, Caddy and the mail server can each get and renew a certificate for the same URL without any conflict, so my batch file is retired!

Paul

How did you set up the service? You can follow these instructions (and add --environ to the <arguments> there):