How to "pass-thru" destination ports?

Help
1

1. The problem I’m having:

I’m trying to run a server that utilizes websockets on a random high port. The port number would be within the range 49152-65535.

As far as I can tell, caddy has no way to “pass-thru” a port number to the upstream without using the layer 4 module, which I don’t want to do as that would remove TLS.

2. Error messages and/or full log output:

Feb 28 00:52:30 caddy systemd[1]: Reloading caddy.service - Caddy...
░░ Subject: A reload job for unit caddy.service has begun execution
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░
░░ A reload job for unit caddy.service has begun execution.
░░
░░ The job identifier is 7286.
Feb 28 00:52:30 caddy caddy[5914]: {"level":"info","ts":1740721950.87227,"msg":"using config from file","file":"/etc/caddy/Caddyfile"}
Feb 28 00:52:30 caddy caddy[5914]: Error: adapting config using caddyfile: parsing key: invalid port '49152-65535': strconv.Atoi: parsing "49152-65535": invalid syntax
Feb 28 00:52:30 caddy systemd[1]: caddy.service: Control process exited, code=exited, status=1/FAILURE
░░ Subject: Unit process exited
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░
░░ An ExecReload= process belonging to unit caddy.service has exited.
░░
░░ The process' exit code is 'exited' and its exit status is 1.
Feb 28 00:52:30 caddy systemd[1]: Reload failed for caddy.service - Caddy.
░░ Subject: A reload job for unit caddy.service has finished
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░
░░ A reload job for unit caddy.service has finished.
░░
░░ The job identifier is 7286 and the job result is failed.

3. Caddy version:

v2.9.1 h1:OEYiZ7DbCzAWVb6TNEkjRcSCRGHVoZsJinoDR/n9oaY=

4. How I installed and ran Caddy:

Currently running caddy in a debian LXC container on Proxmox using the community script from here: Proxmox VE Helper-Scripts

a. System environment:

OS: Debian
Caddy is running as a systemd service.

c. Service/unit/compose file:

# caddy.service
#
# For using Caddy with a config file.
#
# Make sure the ExecStart and ExecReload commands are correct
# for your installation.
#
# See https://caddyserver.com/docs/install for instructions.
#
# WARNING: This service does not use the --resume flag, so if you
# use the API to make changes, they will be overwritten by the
# Caddyfile next time the service is restarted. If you intend to
# use Caddy's API to configure it, add the --resume flag to the
# `caddy run` command or use the caddy-api.service file instead.

[Unit]
Description=Caddy
Documentation=https://caddyserver.com/docs/
After=network.target network-online.target
Requires=network-online.target

[Service]
Type=notify
User=caddy
Group=caddy
ExecStart=/usr/bin/caddy run --environ --config /etc/caddy/Caddyfile
ExecReload=/usr/bin/caddy reload --config /etc/caddy/Caddyfile --force
TimeoutStopSec=5s
LimitNOFILE=1048576
PrivateTmp=true
ProtectSystem=full
AmbientCapabilities=CAP_NET_ADMIN CAP_NET_BIND_SERVICE

[Install]
WantedBy=multi-user.target

d. My complete Caddy config:

# The Caddyfile is an easy way to configure your Caddy web server.
#
# Unless the file starts with a global options block, the first
# uncommented line is always the address of your site.
#
# To use your own domain name (with automatic HTTPS), first make
# sure your domain's A/AAAA DNS records are properly pointed to
# this machine's public IP, then replace ":80" below with your
# domain name.

# Refer to the Caddy docs for more information:
# https://caddyserver.com/docs/caddyfile

{
    email <redacted>
}

archipelago.cryptospartan.net:443 {
    reverse_proxy http://10.0.12.19
}

archipelago.cryptospartan.net:49152-65535 {
    @websockets {
        header Connection *Upgrade*
        header Upgrade    websocket
    }
    reverse_proxy @websockets http://10.0.12.19:49152-65535
}

5. Links to relevant resources:

2

Do you mean you want to use the same port number on which the request is received? Your upstream address can be reverse_proxy @websockets http://10.0.12.19:{port}

Not true. The layer-4 module can receive TLS traffic, inspect the SNI (without TLS termination), and proxy it as-is to upstream.

1 Like
3

Do you mean you want to use the same port number on which the request is received? Your upstream address can be reverse_proxy @websockets http://10.0.12.19:{port}

Yes this is what I’m looking for! But when I do this:

archipelago.cryptospartan.net:443 {
    reverse_proxy http://10.0.12.19:80
}

archipelago.cryptospartan.net:49152-65535 {
    @websockets {
        header Connection *Upgrade*
        header Upgrade    websocket
    }
    reverse_proxy @websockets http://10.0.12.19:{port}
}

Caddy fails to start and I get this error:

Mar 02 21:05:14 caddy caddy[4698]: Error: adapting config using caddyfile: parsing caddyfile tokens for 'reverse_proxy': parsing upstream 'http://10.0.12.19:{http.request.port}': due to parsing difficulties, placeholders are not allowed
when an upstream address contains a scheme, at /etc/caddy/Caddyfile:27
Mar 02 21:05:14 caddy systemd[1]: caddy.service: Control process exited, code=exited, status=1/FAILURE

Also, In regards to this:

Not true. The layer-4 module can receive TLS traffic, inspect the SNI (without TLS termination), and proxy it as-is to upstream.

Caddy is handling all of the TLS for this service, the upstream doesn’t support it. So in this case, I don’t believe layer-4 would solve my issue.

4

The error message means you need to remove the http:// from here. Caddy assumes http in reverse_proxy.

The Caddyfile format does not support port range here, so you will have to use JSON.

I converted the config to JSON by:
1- removing the port range, keeping the starting port only
2- removing http:// from the reverse_proxy upstream address
3- running caddy adapt -p to get the JSON equivalent.
4- inserting the port range into the binding address.

Here’s the result:

{
	"apps": {
		"http": {
			"servers": {
				"srv0": {
					"listen": [
						":443"
					],
					"routes": [
						{
							"match": [
								{
									"host": [
										"archipelago.cryptospartan.net"
									]
								}
							],
							"handle": [
								{
									"handler": "subroute",
									"routes": [
										{
											"handle": [
												{
													"handler": "reverse_proxy",
													"upstreams": [
														{
															"dial": "10.0.12.19:80"
														}
													]
												}
											]
										}
									]
								}
							],
							"terminal": true
						}
					]
				},
				"srv1": {
					"listen": [
						":49152-65535"
					],
					"routes": [
						{
							"match": [
								{
									"host": [
										"archipelago.cryptospartan.net"
									]
								}
							],
							"handle": [
								{
									"handler": "subroute",
									"routes": [
										{
											"handle": [
												{
													"handler": "reverse_proxy",
													"upstreams": [
														{
															"dial": "10.0.12.19:{http.request.port}"
														}
													]
												}
											],
											"match": [
												{
													"header": {
														"Connection": [
															"*Upgrade*"
														],
														"Upgrade": [
															"websocket"
														]
													}
												}
											]
										}
									]
								}
							],
							"terminal": true
						}
					]
				}
			}
		}
	}
}
1 Like