To clarify, you don’t want Caddy to terminate TLS? If so then you definitely need caddy-l4. Vanilla Caddy ships with an HTTP server, so if you need raw TCP (including TLS passthrough) then you need caddy-l4.
Unfortunately JSON config doesn’t support port ranges currently (Caddyfile does in certain places though such as reverse_proxy as a shortcut) so you’ll need to list out each port you want to listen on in the config.
You can then use the proxy handler to route the traffic wherever. I would suggest using a connection placeholder to get the connecting port to make the proxy config simple, but I’m not sure if we have a placeholder for the port alone. /cc @matt if you know.