Localhost reverse proxy - certificate error

1. Caddy version (caddy version):

v2.3.0

2. How I run Caddy:

via systemd service in archlinux

# /etc/hosts/
127.0.0.1	  localhost
127.0.0.1   code.localhost
127.0.0.1   cockpit.localhost

# /etc/caddy/Caddyfile
import /etc/caddy/conf.d/*

## code-server (vscode on server, listens on port 8080)
# /etc/caddy/conf.d/code-server
code.localhost {
    reverse_proxy localhost:8080
}

## cockpit
/etc/caddy/conf.d/cockpit
cockpit.localhost {
    reverse_proxy localhost:9090 
}

a. System environment:

Archlinux, systemd-248.2

b. Command:

I run this as root

# Paste command here.
systemctl start caddy

c. Service/unit/compose file:

Paste full file contents here.
Make sure backticks stay on their own lines,
and the post looks nice in the preview pane.

systemctl enable --now caddy

d. My complete Caddyfile or JSON config:

# caddy.service
#
# For using Caddy with a config file.
#
# Make sure the ExecStart and ExecReload commands are correct
# for your installation.
#
# See https://caddyserver.com/docs/install for instructions.
#
# WARNING: This service does not use the --resume flag, so if you
# use the API to make changes, they will be overwritten by the
# Caddyfile next time the service is restarted. If you intend to
# use Caddy's API to configure it, add the --resume flag to the
# `caddy run` command or use the caddy-api.service file instead.

[Unit]
Description=Caddy web server
Documentation=https://caddyserver.com/docs/
After=network-online.target
Wants=network-online.target systemd-networkd-wait-online.service
StartLimitIntervalSec=14400
StartLimitBurst=10

[Service]
User=caddy
Group=caddy
Environment=XDG_DATA_HOME=/var/lib
Environment=XDG_CONFIG_HOME=/etc
ExecStartPre=/usr/bin/caddy validate --config /etc/caddy/Caddyfile
ExecStart=/usr/bin/caddy run --environ --config /etc/caddy/Caddyfile
ExecReload=/usr/bin/caddy reload --config /etc/caddy/Caddyfile
ExecStopPost=/usr/bin/rm -f /run/caddy/admin.socket

# Do not allow the process to be restarted in a tight loop. If the
# process fails to start, something critical needs to be fixed.
Restart=on-abnormal

# Use graceful shutdown with a reasonable timeout
TimeoutStopSec=5s

LimitNOFILE=1048576
LimitNPROC=512

# Hardening options
AmbientCapabilities=CAP_NET_BIND_SERVICE
CapabilityBoundingSet=CAP_NET_BIND_SERVICE
DevicePolicy=closed
LockPersonality=true
MemoryAccounting=true
MemoryDenyWriteExecute=true
NoNewPrivileges=true
PrivateDevices=true
PrivateTmp=true
ProcSubset=pid
ProtectClock=true
ProtectControlGroups=true
ProtectHome=true
ProtectHostname=true
ProtectKernelLogs=true
ProtectKernelModules=true
ProtectKernelTunables=true
ProtectProc=invisible
ProtectSystem=strict
RemoveIPC=true
ReadWritePaths=/var/lib/caddy /var/log/caddy /run/caddy
RestrictNamespaces=true
RestrictRealtime=true
RestrictSUIDSGID=true

[Install]
WantedBy=multi-user.target

3. The problem I’m having:

I can’t connect to the given localhost sites using https.

4. Error messages and/or full log output:

Brave (chrome) gives the error NET::ERR_CERT_AUTHORITY_INVALID.

$ curl -v cockpit.localhost
*   Trying 127.0.0.1:80...
* Connected to cockpit.localhost (127.0.0.1) port 80 (#0)
> GET / HTTP/1.1
> Host: cockpit.localhost
> User-Agent: curl/7.76.1
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 308 Permanent Redirect
< Connection: close
< Location: https://cockpit.localhost/
< Server: Caddy
< Date: Sun, 23 May 2021 08:37:03 GMT
< Content-Length: 0
<
* Closing connection 0

$ curl -v https://cockpit.localhost
*   Trying 127.0.0.1:443...
* Connected to cockpit.localhost (127.0.0.1) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (OUT), TLS alert, unknown CA (560):
* SSL certificate problem: unable to get local issuer certificate
* Closing connection 0
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

$ curl -v code.localhost
*   Trying 127.0.0.1:80...
* Connected to code.localhost (127.0.0.1) port 80 (#0)
> GET / HTTP/1.1
> Host: code.localhost
> User-Agent: curl/7.76.1
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 308 Permanent Redirect
< Connection: close
< Location: https://code.localhost/
< Server: Caddy
< Date: Sun, 23 May 2021 08:42:41 GMT
< Content-Length: 0
<
* Closing connection 0

$ curl -v https://code.localhost
*   Trying 127.0.0.1:443...
* Connected to code.localhost (127.0.0.1) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (OUT), TLS alert, unknown CA (560):
* SSL certificate problem: unable to get local issuer certificate
* Closing connection 0
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

5. What I already tried:

I was wondering if the certificate was considered self signed, but then I’m not sure if curl would give any error in that case.

Went through suggested answers in similar posts (everything seems to be already done / similar to what the docs said).

6. Links to relevant resources:

I just consulted caddy docs and past posts. Will post the specific ones in a bit.

So if I understand correctly, caddy uses letsencrypt for publicly available sites and its own CA (which is not trusted by default) for the non-public ones. So localhost uses an ssl certificate from a non-trusted CA and hence the error. Bye.

That’s correct. You can find the root CA certificate in Caddy’s storage, which you can install on whatever systems/browsers you’ll be using.

It’s probably located in /var/lib/caddy/pki/authorities/local/root.crt considering your systemd config.

1 Like

Thank you.

There’s another problem. The systemd unit doesn’t seem to make the system trust caddy’s certificate. But if I start it manually, it starts working as expected.

# All commands are run as root.

# This doesn't work
systemctl start caddy

# But this works
cd /etc/caddy
caddy start

Is something wrong with the systemd config?

The problem is that the caddy user doesn’t have sudo so it can’t install the root cert to your system.

You can run Caddy like this to make it trust the CA cert from the storage pointed to by your systemd config:

sudo XDG_DATA_HOME=/var/lib caddy trust

You may need to run this beforehand to untrust the CA cert from your current user though.

caddy untrust

If my username is user, are you talking about user or caddy?

Also, any advisable way to automate this trust? Like putting XDG_DATA_HOME in the systemd unit file or running the trust command from the unit file as a pre-hook.

You only need to run caddy trust once because the root CA certificate has a very long lifetime.

I do mean the user caddy because that’s the user in the systemd config.

One more thing, if caddy doesn’t have the permissions to install the certificate, how does it have the permissions to listen on port 80 and 443?

Here are the package files for reference.

These lines give it permission

1 Like

Alright, thank you for your time.

This topic was automatically closed after 30 days. New replies are no longer allowed.