When you have a certificate issued, this is the general process:
You generate a key pair (a private key, and its derived public key)
You make a CSR (Certificate Signing Request) from the key pair, which basically says “hey signing authority, here’s my public key, along with some information about me and the domain I want a certificate for”
The authority takes the CSR, extracts the public key and the information you provided, puts that information into a certificate, then signs that certificate with THEIR private key.
They then give you back the certificate chain with your signed certificate
So here, Caddy is checking that the public key inside the certificate matches the public component of your key (public keys can be derived from a private key, by doing some fancy math, depends on the type of key how this is done).
I don’t know what exactly is wrong in your files. It might be that you’re not using the same key that was used when issuing the certificates. You can’t mix and match certificates and keys.
I like using ASN.1 JavaScript decoder when trying to debug issues with certificates and keys. You can paste your key and certificate, and you can compare whether they public keys inside match. ASN.1 is the underlying encoding format inside of PEM files (which are just base64 encoded DER… and DER is just the binary representation of an ASN.1 tree)
Thank you for your clear explanations. I might have a hint on what I am doing wrong.
You said that the CA signs the certificate with their private key. Inside Caddy, I am using the key the CSR was created with. Do I need the key the CA emits?
The problem was that I was using the CA bundles for certificate, instead of the certificate issued to the domain. This was because the CA was sending me a corrupted .zip where this one file was missing, but other two not, and got me confused.