Reverse-proxy, upgrade to https, custom/non-root ports

1. Caddy version (caddy version):

v2.3.0

2. How I run Caddy:

As a non-root reverse-proxy

a. System environment:

Archlinux

b. Command:

cd /path/to/Caddyfile
caddy start

c. Service/unit/compose file:

-

d. My complete Caddyfile or JSON config:

{
    http_port 8081
    https_port 8082
}

:8081, :8082 {
    reverse_proxy localhost:8080
}

3. The problem I’m having:

I have a code-server program listening to http://localhost:8080 requests.
I’m trying to run a non-root reverse proxy on custom http(s) ports as shown in the Caddyfile to relay requests to this server through https. The problem is http/8081 works, but https/8082 doesn’t, even though it is not a root port.

Non-root caddy’s certificate is already trusted. Https works as desired when I run the following command instead. I have to insert a https://localhost though for it to work.

sudo caddy reverse-proxy --from https://localhost:8082 --to http://localhost:8080

4. Error messages and/or full log output:

No errors.
Log -

$ caddy start
2021/05/26 14:29:19.470	INFO	using adjacent Caddyfile
2021/05/26 14:29:19.472	INFO	admin	admin endpoint started	{"address": "tcp/localhost:2019", "enforce_origin": false, "origins": ["localhost:2019", "[::1]:2019", "127.0.0.1:2019"]}
2021/05/26 14:29:19.472	INFO	http	server is listening only on the HTTP port, so no automatic HTTPS will be applied to this server	{"server_name": "srv0", "http_port": 8081}
2021/05/26 14:29:19.472	INFO	http	server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS	{"server_name": "srv1", "https_port": 8082}
2021/05/26 14:29:19.472	INFO	http	enabling automatic HTTP->HTTPS redirects	{"server_name": "srv1"}
2021/05/26 14:29:19.472	WARN	http	user server is listening on same interface as automatic HTTP->HTTPS redirects; user-configured routes might override these redirects	{"server_name": "srv0", "interface": "tcp/:8081"}
2021/05/26 14:29:19.472	INFO	tls.cache.maintenance	started background certificate maintenance	{"cache": "0xc00035f2d0"}
2021/05/26 14:29:19.472	INFO	tls	cleaned up storage units
2021/05/26 14:29:19.472	INFO	autosaved config	{"file": "/home/user/.local/share/caddy/autosave.json"}
2021/05/26 14:29:19.472	INFO	serving initial configuration
Successfully started Caddy (pid=76436) - Caddy is running in the background

5. What I already tried:

Running it as superuser with command line options works (as shown above), but that is far from what I need.
I’ve followed the following resources, no fruit.

6. Links to relevant resources:

No :443 in my config :frowning:

Not a reverse proxy :frowning: :frowning:

You didn’t tell Caddy to manage any certificates, so the TLS handshake cannot be completed.

When you ran it with localhost, Caddy issued a certificate with the name localhost, using its internal CA, which is automatically enabled if Caddy sees the domain to look like something that can’t be publicly trusted. Read the docs here:

Caddy can issue certificates on demand for connections without a domain (i.e. an IP address), but you need to enable the on_demand option.

You’d be better off running Caddy as a systemd service, which will run as the caddy user typically (non-root) but with the CAP_NET_BIND_SERVICE capability enabled (which allows binding to low ports for non-root users). If you installed Caddy using pacman then you should have the systemd service already wired up:

If not then you can set up the service yourself:

1 Like

Thank you.
Just read up on tls and on_demand_tls directives.
So is this config good?

{
    http_port 8081
    https_port 8082
    on_demand_tls {
        interval 5s
        burst 1
    }
}

:8081, :8082 {
    tls {
        on_demand
    }
    reverse_proxy localhost:8080
}

Also, does this require a new caddy trust for every new website?

Edit -
On entering https://localhost:8082 in the browser, this is what I’m getting.


I added the interval and the burst directives because of this output (too many "obtaining new certificate" messages). I’m not sure why this is happening.

That allows 1 new certificate every 5 seconds. If that’s suitable for you, then sure. Only you will really be able to decide what limits are safe.

Nope, just a single root is trusted and that root lasts for 10 years.

It’s possible the browser is retrying every second. Try using curl -v instead. Are those the only log messages? That’s a bit unusual.

1 Like

You’re right. It’s four attempts per request, twice a second on average.
It’s not the only output though.

Here’s the complete log after one curl request (I put the on_demand_tls directive to avoid duplicate certificate requests).

2021/05/27 05:31:10.477	INFO	using adjacent Caddyfile
2021/05/27 05:31:10.479	INFO	admin	admin endpoint started	{"address": "tcp/localhost:2019", "enforce_origin": false, "origins": ["localhost:2019", "[::1]:2019", "127.0.0.1:2019"]}
2021/05/27 05:31:10.479	INFO	tls.cache.maintenance	started background certificate maintenance	{"cache": "0xc0003bca10"}
2021/05/27 05:31:10.486	INFO	http	server is listening only on the HTTP port, so no automatic HTTPS will be applied to this server	{"server_name": "srv0", "http_port": 8081}
2021/05/27 05:31:10.486	INFO	http	server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS	{"server_name": "srv1", "https_port": 8082}
2021/05/27 05:31:10.486	INFO	http	enabling automatic HTTP->HTTPS redirects	"server_name": "srv1"}
2021/05/27 05:31:10.486	WARN	http	user server is listening on same interface as automatic HTTP->HTTPS redirects; user-configured routes might override these redirects	"server_name": "srv0", "interface": "tcp/:8081"}
2021/05/27 05:31:10.486	INFO	tls	cleaned up storage units
2021/05/27 05:31:10.486	INFO	autosaved config	{"file": "/home/user/.local/share/caddy/autosave.json"}
2021/05/27 05:31:10.486	INFO	serving initial configuration
2021/05/27 06:18:46.591	INFO	tls.on_demand	obtaining new certificate	{"server_name": "192.168.1.42"}

And here’s the curl output.

$ curl -v "http://192.168.1.42:8081"
*   Trying 192.168.1.42:8081...
* Connected to 192.168.1.42 (192.168.1.42) port 8081 (#0)
> GET / HTTP/1.1
> Host: 192.168.1.42:8081
> User-Agent: curl/7.76.1
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 302 Found
< Content-Length: 29
< Content-Type: text/plain; charset=utf-8
< Date: Wed, 26 May 2021 20:37:36 GMT
< Location: ./login
< Server: Caddy
< Vary: Accept, Accept-Encoding
<
* Connection #0 to host 192.168.1.42 left intact
Found. Redirecting to ./login
                                                      ~
$ curl -v "http://192.168.1.42:8082"
*   Trying 192.168.1.42:8082...
* Connected to 192.168.1.42 (192.168.1.42) port 8082 (#0)
> GET / HTTP/1.1
> Host: 192.168.1.42:8082
> User-Agent: curl/7.76.1
> Accept: */*
>
* Mark bundle as not supporting multiuse
* HTTP 1.0, assume close after body
< HTTP/1.0 400 Bad Request
<
Client sent an HTTP request to an HTTPS server.
* Closing connection 0

$ curl -v "https://192.168.1.42:8082"
*   Trying 192.168.1.42:8082...
* Connected to 192.168.1.42 (192.168.1.42) port 8082 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS alert, internal error (592):
* error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error
* Closing connection 0
curl: (35) error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error

Also, I need 8081 to stay on http and 8082 to upgrade to https (if the request is http). Does this config do that?

That’s not possible. You need to request HTTP over the HTTP port, and HTTPS over the HTTPS port.

Update -
Apparently for the user instance, the data folder ~/.local/share/caddy was owned by root. Must have been when I ran caddy with superuser privileges without changing XDG_DATA_HOME. So instead of taking /var/lib/caddy, it took ~/.local/share/caddy and so the certificates could not be accessed.

I’ve since changed the owner back to the non-root user. Here is the Caddyfile I’m using.

:8081 {
    tls {
        on_demand
    }
    reverse_proxy localhost:8080
}

curl output -

$ curl -v "http://localhost:8081"
*   Trying 127.0.0.1:8081...
* Connected to localhost (127.0.0.1) port 8081 (#0)
> GET / HTTP/1.1
> Host: localhost:8081
> User-Agent: curl/7.76.1
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 302 Found
< Content-Length: 29
< Content-Type: text/plain; charset=utf-8
< Date: Thu, 27 May 2021 16:39:31 GMT
< Location: ./login
< Server: Caddy
< Vary: Accept, Accept-Encoding
<
* Connection #0 to host localhost left intact
Found. Redirecting to ./login
$ curl -v "https://localhost:8081"
*   Trying 127.0.0.1:8081...
* Connected to localhost (127.0.0.1) port 8081 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* error:1408F10B:SSL routines:ssl3_get_record:wrong version number
* Closing connection 0
curl: (35) error:1408F10B:SSL routines:ssl3_get_record:wrong version number

And caddy’s log -

$ caddy run
2021/05/27 16:39:16.828	INFO	using adjacent Caddyfile
2021/05/27 16:39:16.830	INFO	admin	admin endpoint started	{"address": "tcp/localhost:2019", "enforce_origin": false, "origins": ["127.0.0.1:2019", "localhost:2019", "[::1]:2019"]}
2021/05/27 16:39:16.830	INFO	tls.cache.maintenance	started background certificate maintenance	{"cache": "0xc000352e70"}
2021/05/27 16:39:16.831	INFO	autosaved config	{"file": "/home/user/.local/share/caddy/autosave.json"}
2021/05/27 16:39:16.831	INFO	serving initial configuration
2021/05/27 16:39:16.831	INFO	tls	cleaned up storage units

Isn’t on-demand tls supposed to redirect http to https here?

Yes, if a request comes on the HTTP port, it will respond with a redirect to the HTTPS port. An HTTP request to the HTTPS port is a bad request (wrong type of data on the wire).

A redirect is an HTTP response which contains the Location: header telling the client to try again with a different URL.

An HTTPS server cannot respond to HTTP requests.

This is all simplified if you use ports 80 and 443 because the browser assumes those port numbers automatically when you type http:// and https:// in the browser.

Like I said, if you run Caddy as a service, you can run Caddy as a non-root user, with permission to bind to ports 80 and 443.

I do run it as a service for the actual server, Im just seeing what all can be done as a non-root user.

So one last thing.
With on-demand tls, when a https request is sent, I’m getting an error in the log.

2021/05/27 18:26:57.599	INFO	using adjacent Caddyfile
2021/05/27 18:26:57.601	INFO	admin	admin endpoint started	{"address": "tcp/localhost:2019", "enforce_origin": false, "origins": ["[::1]:2019", "127.0.0.1:2019", "localhost:2019"]}
2021/05/27 18:26:57.601	INFO	http	server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS	{"server_name": "srv0", "https_port": 8082}
2021/05/27 18:26:57.601	INFO	http	enabling automatic HTTP->HTTPS redirects	{"server_name": "srv0"}
2021/05/27 18:26:57.601	INFO	tls.cache.maintenance	started background certificate maintenance	{"cache": "0xc00034f3b0"}
2021/05/27 18:26:57.601	INFO	http	enabling automatic TLS certificate management	{"domains": ["localhost"]}
2021/05/27 18:26:57.601	INFO	autosaved config	{"file": "/home/user/.local/share/caddy/autosave.json"}
2021/05/27 18:26:57.601	INFO	serving initial configuration
2021/05/27 18:26:57.602	INFO	tls	cleaned up storage units
2021/05/27 18:26:57.608	INFO	tls.on_demand	obtaining new certificate	{"server_name": "localhost"}
2021/05/27 18:26:57.608	INFO	tls.obtain	acquiring lock	{"identifier": "localhost"}
2021/05/27 18:26:57.609	INFO	tls.obtain	lock acquired	{"identifier": "localhost"}
2021/05/27 18:26:57.616	ERROR	tls.obtain	will retry	{"error": "[localhost] Obtain: subject does not qualify for a public certificate: localhost", "attempt": 1, "retrying_in": 60, "elapsed": 0.007231348, "max_duration": 2592000}

And it doesn’t proceed from there.

The caddy user is non-root though.

Yeah, but to start/enable the service you have to be root eh?

Can you look at the above question too (edited reply)?
Error says localhost doesn’t qualify for a public certificate. So isn’t it supposed to get a local CA signed one instead? I’ve already made the system trust the root certificate, so I thought it was supposed to work.

If you’re using systemd, then yes, systemd runs as root. But I don’t see the concern here.

You need to also enable the internal issuer by specifying tls internal, otherwise it will always try to use the public issuers (Let’s Encrypt/ZeroSSL).

1 Like

Yes. this is what I needed, thanks.

systemctl start or systemctl enable requires root access, doesn’t it?

This topic was automatically closed after 30 days. New replies are no longer allowed.