FQDN IP reverse proxy for a ProxMox 7.3 Cluster LAN

Help
1

I have an ISP assigned static IP connection delivered by a Fiber SFP connection on the WAN side of a TPLink 7206 Router that acts as the DHCP server for my internal LAN.

On that LAN I have a ProxMox PVE 7.3-4 Cluster built from recycled Dell rack hardware.
5 Nodes, 2 R610’s and 3 R710’s with the equivalent of 120 Xeon CPU’s and 450 GB RAM and 30 TB of raid 5 disk storage.

This provides the platform for a host of Ubuntu Server VM’s to provide web services both internal and external. The internal servers have been easy to roll out as they are all on the shared LAN address space, 192.168.0.1 /24 served by the LAN DHCP with static IP’s, and the internet is accessible to all the local devices.

Serving out through the fiber router though has been difficult in this regard.

The router will NAT route any single VM, but only one at a time. I have three Nextcloud servers, two ERP servers and 2 Apache 2.4 web servers on the cluster, but only one can be externally accessible at a time.

I have been trying to find a reverse proxy solution to allow the web servers to be accessible from the net. I was told that “Caddy is the perfect solution” to this problem as it can be installed on a VM and route all the internal VM’s by traffic by their A records to subdomain names through the single hardware NAT router IP I have, What I am stuck on is the Caddyfile instructions to make this happen.

The TPLink NAT router will serve out any single internal LAN IP address, port range, and protocol set, but only one at a time. How can I configure a Caddyfile to proxy a cluster full of VM’s at the same time?

Yes I have a Caddy 2.62 VM as well on an Ubuntu 22.04.1 LTS server that I can easily have the hardware router connect to, its the Caddyfile construction that has me scratching my head. I have the domains, DNS A records and FQDN static IP, and the VM’s all configured with statically assigned LAN addresses.

Whats the best practices to be able to build and assign entries for the servers on the LAN?

2

What it sounds like you’re trying to do is set up Caddy as a reverse proxy to all the services you have running internally.

Basically, Caddy will listen on ports 80 and 443 (port forward those ports to your Caddy server) and route requests based on the config; typically routing is done by hostname (domain).

It might look like this:

nextcloud1.example.com {
	reverse_proxy 192.168.0.10:8080
}

nextcloud2.example.com {
	reverse_proxy 192.168.0.11:8080
}

something-else.example.com {
	reverse_proxy 192.168.0.12:9000
}

Caddy will automate issuing TLS certificates for each of those domains; you just need to make sure your DNS is correct for those domains, pointing to your WAN IP, and that your network router is routing traffic on ports 80/443 to Caddy.

For most apps, just the default reverse_proxy directive should probably work. But some apps have some tendencies to not work behind a reverse proxy out-of-the-box without some additional config, so you’ll need to look into that. For example, I know that NextCloud typically requires configuring trusted proxies to work properly – search their docs for details on that.

Hopefully that’s enough to get you started; I’m not really sure what else to say. Make sure to read the Caddy docs, especially the Caddyfile Concepts page which should give you an idea of how the config is structured:

3

I will move ahead with your suggestions and the structure examples. I will post the results as I go. Thanks for your feedback.

1 Like
4

Hi Francois

Your structured example solved a bunch of problems. I still have some complex issues with the NextCloud instances, but the proxy routing works for all of them and the performance is excellent.

5

Francis* :wink:

Great! Glad to hear.

6

Francis,

My apologies for my error. I am only now beginning to really appreciate the large number of variable components involved in reverse proxy routing for HTTPS protocols with the large range of variables in how that content can be served and the spectrum of platforms that serve it. That said, the ability to get maximum mileage from the very limited and expensive availability of IPV4 address space makes this a very valuable solution. When I have completed this project and have it fully online I will produce a workbook and project manual for others with the same needs. I am satisfied now that Caddy and ProxMox Cluster is a solid, scalable and interoperable solution for services deployment.

1 Like
7

This topic was automatically closed after 30 days. New replies are no longer allowed.