Can't get certificate to work. OCSP stapling error given

Jacen_Martin · May 3, 2024, 7:41pm

I have vaultwarden running on port 9000 and am running Caddy for forward web traffic to it. It works but I can’t figure the certificate out. I have been through the documentation and still can’t find a solution. I am able to get to the site locally (only want it to be locally) but when I try other types of code and/or solutions, none of them have worked for me. All ports used have been opened.

What is the most basic way to make your Caddy file have a Lets encrypt cert? This is what I have in my Caddyfile and I run ‘caddy run’

I also see that in the logs while it is running, it says localhost:2019, which is confusing to me.

2. Error messages and/or full log output:

caddy run
2024/05/03 19:25:52.441 INFO    using adjacent Caddyfile
2024/05/03 19:25:52.443 WARN    Caddyfile input is not formatted; run 'caddy fmt --overwrite' to fix inconsistencies{"adapter": "caddyfile", "file": "Caddyfile", "line": 2}
2024/05/03 19:25:52.444 INFO    admin   admin endpoint started  {"address": "localhost:2019", "enforce_origin": false, "origins": ["//localhost:2019", "//[::1]:2019", "//127.0.0.1:2019"]}
2024/05/03 19:25:52.445 INFO    tls.cache.maintenance   started background certificate maintenance      {"cache": "0xc000528700"}
2024/05/03 19:25:52.445 INFO    http.auto_https server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS      {"server_name": "srv0", "https_port": 443}
2024/05/03 19:25:52.445 INFO    http.auto_https enabling automatic HTTP->HTTPS redirects        {"server_name": "srv0"}
2024/05/03 19:25:52.445 INFO    http    enabling HTTP/3 listener        {"addr": ":443"}
2024/05/03 19:25:52.446 INFO    http.log        server running  {"name": "srv0", "protocols": ["h1", "h2", "h3"]}
2024/05/03 19:25:52.446 INFO    http.log        server running  {"name": "remaining_auto_https_redirects", "protocols": ["h1", "h2", "h3"]}
2024/05/03 19:25:52.446 INFO    http    enabling automatic TLS certificate management   {"domains": ["fl000sbwd001.network.root"]}
2024/05/03 19:25:52.446 WARN    tls     stapling OCSP   {"error": "no OCSP stapling for [fl000sbwd001.network.root]: no OCSP server specified in certificate", "identifiers": ["fl000sbwd001.network.root"]}
2024/05/03 19:25:52.448 WARN    tls     storage cleaning happened too recently; skipping for now        {"storage": "FileStorage:/root/.local/share/caddy", "instance": "099a3dc3-49b0-4e88-9d6f-2da38c9bb8bb", "try_again": "2024/05/04 19:25:52.448", "try_again_in": 86399.999999484}
2024/05/03 19:25:52.448 INFO    tls     finished cleaning storage units
2024/05/03 19:25:52.464 INFO    pki.ca.local    root certificate is already trusted by system   {"path": "storage:pki/authorities/local/root.crt"}
2024/05/03 19:25:52.464 INFO    autosaved config (load with --resume flag)      {"file": "/root/.config/caddy/autosave.json"}
2024/05/03 19:25:52.465 INFO    serving initial configuration

Caddy version: v2.7.6

How I installed and ran Caddy:

Installed and ran it on RockyOS

caddy run
2024/05/03 19:30:25.605 INFO using adjacent Caddyfile
2024/05/03 19:30:25.606 WARN Caddyfile input is not formatted; run ‘caddy fmt --overwrite’ to fix inconsistencies{“adapter”: “caddyfile”, “file”: “Caddyfile”, “line”: 2}
2024/05/03 19:30:25.607 INFO admin admin endpoint started {“address”: “localhost:2019”, “enforce_origin”: false, “origins”: [“//localhost:2019”, “//[::1]:2019”, “//127.0.0.1:2019”]}
2024/05/03 19:30:25.608 INFO tls.cache.maintenance started background certificate maintenance {“cache”: “0xc0003b2500”}
2024/05/03 19:30:25.608 INFO http.auto_https server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS {“server_name”: “srv0”, “https_port”: 443}
2024/05/03 19:30:25.608 INFO http.auto_https enabling automatic HTTP->HTTPS redirects {“server_name”: “srv0”}
2024/05/03 19:30:25.608 INFO http enabling HTTP/3 listener {“addr”: “:443”}
2024/05/03 19:30:25.609 INFO http.log server running {“name”: “srv0”, “protocols”: [“h1”, “h2”, “h3”]}
2024/05/03 19:30:25.609 INFO http.log server running {“name”: “remaining_auto_https_redirects”, “protocols”: [“h1”, “h2”, “h3”]}
2024/05/03 19:30:25.609 INFO http enabling automatic TLS certificate management {“domains”: [“fl000sbwd001.network.root”]}
2024/05/03 19:30:25.609 WARN tls stapling OCSP {“error”: “no OCSP stapling for [fl000sbwd001.network.root]: no OCSP server specified in certificate”, “identifiers”: [“fl000sbwd001.network.root”]}
2024/05/03 19:30:25.611 WARN tls storage cleaning happened too recently; skipping for now {“storage”: “FileStorage:/root/.local/share/caddy”, “instance”: “099a3dc3-49b0-4e88-9d6f-2da38c9bb8bb”, “try_again”: “2024/05/04 19:30:25.611”, “try_again_in”: 86399.999999258}
2024/05/03 19:30:25.612 INFO tls finished cleaning storage units
2024/05/03 19:30:25.629 INFO pki.ca.local root certificate is already trusted by system {“path”: “storage:pki/authorities/local/root.crt”}
2024/05/03 19:30:25.630 INFO autosaved config (load with --resume flag) {“file”: “/root/.config/caddy/autosave.json”}
2024/05/0

a. System environment:

b. Command:

[root@FL000SBWD001 caddy]# ls
Caddyfile  caddy.json
[root@FL000SBWD001 caddy]# caddy run

c. Service/unit/compose file:

d. My complete Caddy config:

caddy fmt
fl000sbwd001.network.root {
        tls internal

        reverse_proxy localhost:9000
}

I have also tried

:80 

reverse_proxy localhost:9000

As well as

:80
reverse_proxy localhost:9000
tls internal

There are other things I have tried but I have been troubleshooting for several hours and forget them all.

5. Links to relevant resources:

francislavoie · May 3, 2024, 10:42pm

That’s not an error, it’s a warning. It’s harmless. When you use tls internal, OCSP stapling isn’t possible because there’s no public authority involved.

Jacen_Martin · May 3, 2024, 11:30pm

Ok but what would you change about the caddy config to get the certificate working? I am misunderstanding or missing something about that.

matt · May 4, 2024, 2:06am

Our guess is as good as yours, since all we know is “doesn’t work” – do you have some evidence of that? What’s a curl -v command that exhibits it not working?

Jacen_Martin · May 6, 2024, 3:49pm

Ok, sorry about that. This is what I get when running curl -v.


[root@FL000SBWD001 roo]# curl -v fl000sbwd001.network.root
*   Trying 172.28.5.58:80...
* Connected to fl000sbwd001.network.root (172.28.5.58) port 80 (#0)
> GET / HTTP/1.1
> Host: fl000sbwd001.network.root
> User-Agent: curl/7.76.1
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 308 Permanent Redirect
< Connection: close
< Location: https://fl000sbwd001.network.root/
< Server: Caddy
< Date: Mon, 06 May 2024 15:47:18 GMT
< Content-Length: 0
<
* Closing connection 0

Just to reiterate, this is my caddyfile and am running caddy run

fl000sbwd001.network.root {
        tls internal

        reverse_proxy localhost:9000
}

francislavoie · May 6, 2024, 4:04pm

You just made an HTTP request to Caddy, which it responded to with an HTTP->HTTPS redirect. Make sure to use https:// when you run that command.

Jacen_Martin · May 6, 2024, 5:37pm

Ah, I see. I have run into this problem now, however. Because my TLD is a .root, I cannot use ACME. So, what I am trying to do is use my domain’s CA rather than the built in ACME.

I cannot get Caddy to use the .pem and .key files though, it keeps trying to obtain ACME and that will not work because of my .root. What am I doing incorrectly in my caddyfile / caddy.json that is not making it use my domain’s CA?

caddy.json:

    "apps": {
        "http": {
            "servers": {
                "srv0": {
                    "listen": [":443"],
                    "routes": [
                        {
                            "match": [{"host": ["fl000sbwd.network.root"]}],
                            "handle": [
                                {
                                    "handler": "reverse_proxy",
                                    "upstreams": [{"dial": "localhost:9000"}]
                                }
                            ],
                            "terminal": true
                        }
                    ]
                }
            }
        }
    },
    "tls": {
        "automation": {
            "policies": [
                {
                    "subjects": ["fl000sbwd.network.root"],
                    "issuer": {
                        "module": "internal"
                    },
                    "protocols": ["tls1.2", "tls1.3"],
                    "on_demand": false,
                    "renewal_window": "24h",
                    "disable_stapling": true
                }
            ]
        }
    }
}

This is my caddyfile

fl000sbwd001.network.root {
        tls wildcard2024.pem priv-decrypt.key

        reverse_proxy localhost:9000
}

These files are in this directory.

 caddy run
2024/05/06 17:32:23.962 INFO    using adjacent Caddyfile
2024/05/06 17:32:23.965 INFO    admin   admin endpoint started  {"address": "localhost:2019", "enforce_origin": false, "origins": ["//localhost:2019", "//[::1]:2019", "//127.0.0.1:2019"]}
2024/05/06 17:32:23.965 INFO    tls.cache.maintenance   started background certificate maintenance      {"cache": "0xc000118f80"}
2024/05/06 17:32:23.968 INFO    http.auto_https enabling automatic HTTP->HTTPS redirects        {"server_name": "srv0"}
2024/05/06 17:32:23.969 INFO    http    enabling HTTP/3 listener        {"addr": ":443"}
2024/05/06 17:32:23.969 INFO    http.log        server running  {"name": "srv0", "protocols": ["h1", "h2", "h3"]}
2024/05/06 17:32:23.970 INFO    http.log        server running  {"name": "remaining_auto_https_redirects", "protocols": ["h1", "h2", "h3"]}
2024/05/06 17:32:23.970 INFO    http    enabling automatic TLS certificate management   {"domains": ["fl000sbwd001.network.root"]}
2024/05/06 17:32:23.970 INFO    autosaved config (load with --resume flag)      {"file": "/root/.config/caddy/autosave.json"}
2024/05/06 17:32:23.970 INFO    serving initial configuration
2024/05/06 17:32:23.971 INFO    tls.obtain      acquiring lock  {"identifier": "fl000sbwd001.network.root"}
2024/05/06 17:32:23.973 WARN    tls     storage cleaning happened too recently; skipping for now        {"storage": "FileStorage:/root/.local/share/caddy", "instance": "91ea7e18-e575-48c4-ae0c-a59cd9ceeaba", "try_again": "2024/05/07 17:32:23.973", "try_again_in": 86399.999999014}
2024/05/06 17:32:23.974 INFO    tls     finished cleaning storage units
2024/05/06 17:34:37.433 INFO    tls.obtain      obtaining certificate   {"identifier": "fl000sbwd001.network.root"}
2024/05/06 17:34:37.803 ERROR   tls.obtain      could not get certificate from issuer   {"identifier": "fl000sbwd001.network.root", "issuer": "acme-v02.api.letsencrypt.org-directory", "error": "HTTP 400 urn:ietf:params:acme:error:rejectedIdentifier - Error creating new order :: Cannot issue for \"fl000sbwd001.network.root\": Domain name does not end with a valid public suffix (TLD)"}
2024/05/06 17:34:38.447 ERROR   tls.obtain      could not get certificate from issuer   {"identifier": "fl000sbwd001.network.root", "issuer": "acme.zerossl.com-v2-DV90", "error": "HTTP 400 urn:ietf:params:acme:error:rejectedIdentifier - DNS identifier is invalid [fl000sbwd001.network.root]"}
2024/05/06 17:34:38.447 ERROR   tls.obtain      will retry      {"error": "[fl000sbwd001.network.root] Obtain: [fl000sbwd001.network.root] creating new order: attempt 1: https://acme.zerossl.com/v2/DV90/newOrder: HTTP 400 urn:ietf:params:acme:error:rejectedIdentifier - DNS identifier is invalid [fl000sbwd001.network.root] (ca=https://acme.zerossl.com/v2/DV90)", "attempt": 5, "retrying_in": 600, "elapsed": 603.705819121, "max_duration": 2592000}

Here is the curl -v

 curl -v https://fl000sbwd001.network.root
*   Trying 172.28.5.58:443...
* Connected to fl000sbwd001.network.root (172.28.5.58) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
*  CAfile: /etc/pki/tls/certs/ca-bundle.crt
* TLSv1.0 (OUT), TLS header, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS header, Finished (20):
* TLSv1.2 (IN), TLS header, Unknown (23):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.2 (IN), TLS header, Unknown (23):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS header, Unknown (21):
* TLSv1.3 (OUT), TLS alert, unknown CA (560):
* SSL certificate problem: unable to get local issuer certificate
* Closing connection 0
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.se/docs/sslcerts.html

francislavoie · May 7, 2024, 5:10pm

What do you mean by your “domain’s CA”?

Are you using JSON config or Caddyfile config? Those two configs don’t match.

Your logs show that Caddy is still trying to hit Let’s Encrypt. It shouldn’t, if you actually loaded that Caddyfile config using the tls directive passing a cert + key.