OpenVPN over reverse-proxy

1. Caddy version (caddy version):

V2.2.1

2. How I run Caddy:

On Ubuntu as service

a. System environment:

Ubuntu 20.04

b. Command:

runs from service default (installed using apt)

c. Service/unit/compose file:

default systemd

d. My complete Caddyfile or JSON config:

{
  experimental_http3
  admin :2019 {
    origins localhost, 127.0.0.1, 192.168.17.100
  }
}

# Add gzip compression to requests
(webconf) {
  encode gzip
}

#define logging
(logging) {
  log {
    output file /var/log/caddy/caddy.log {
      roll_size 1gb
      roll_keep 5
    }
  }
}

#define wildcard certificate
(certs) {
  tls /etc/caddy/STAR_eltomation_com.pem /etc/caddy/STAR_eltomation_com.key
}

# Add forward headers to requests
(theheaders) {
  header_up X-Forwarded-Ssl on
  header_up Host {http.request.host}
  header_up X-Real-IP {http.request.remote}
  header_up X-Forwarded-For {http.request.remote}
  header_up X-Forwarded-Port {http.request.port}
  header_up X-Forwarded-Proto {http.request.scheme}
  header_up X-Url-Scheme {http.request.scheme}
  header_up X-Forwarded-Host {http.request.host}
}
openvpn.eltomation.com {
  import logging
  import certs
  reverse_proxy https://192.168.16.4 {
    #import theheaders
    transport http {
      tls_insecure_skip_verify
    }
  }
  import webconf
}
openvpn.eltomation.com:943 {
  import logging
  import certs
  reverse_proxy https://192.168.16.4:943 {
   #import theheaders
    transport http {
      tls_insecure_skip_verify
    }
  }
  import webconf
}
openvpn.eltomation.com:1194 {
  import logging
  import certs
  reverse_proxy https://192.168.16.4:1194 {
    #import theheaders
    transport http {
      tls_insecure_skip_verify
    }
  }
  import webconf
}

3. The problem I’m having:

OpenVPN can’t connect.
Opening the webbrowser does work, but it shows the IP of the reverse-proxy and not the WAN IP of the one connecting.
Enabling the import theheaders doesn’t make any difference.

4. Error messages and/or full log output:

5. What I already tried:

Tried adding the headers, and tried (as first step) to not add the headers.
Also tried to force the connection over only 443. Without Caddy this works, but when the connection needs to run over Caddy it doesn’t work.

To be clear, browsing the openvpn.eltomation.com website does work, but connecting to it using openvpn doesn’t.

6. Links to relevant resources:

A little more insight on how OpenVPN is ran would be useful. So you’re reverse proxying the webserver to OpenVPN multiple times. The first one is on port 443, the second is on 943 and the third is on 1194. I’m going to assume you’re using OpenVPN as UDP. You have experimental HTTP/3 support enabled as well. HTTP/3 now runs on UDP. Try removing the whole 1194 block from your webserver and disabling experimental HTTP/3 support. I run a WireGuard VPN on port 443 (every other port at school is blocked) and Caddy on 443 as well. You can see how mine works at https://vpn.telesphoreo.me. Also, are you running (a different) OpenVPN on port 943 as well? I can connect to the website on 443 and 943 but not 1194. If you could also SSH into your server and run netstat -tulpn | grep 1194 that would be great. Here is my port bindings: You can run both on port 1194, but I think what’s conflicting is both are on UDP.

1 Like

To get the stuff started, I run one OpenVPN a.t.m. but in the end I will run 2. One for use as domain users and one for our customers.
443 = WebGUI and VPN
1194 = VPN
943 = Admin page
Multi-daemon mode, so TCP and UDP is possible.

I tried to reverse proxy with one rule (Reverse proxy matching on port)
but there was suggested to split it up to each port.

Right, so Caddy shouldn’t be routing VPN traffic itself. The VPN should be going
Caddy hosting the certificates to connect to VPN ----> Client ----> Server

Traffic itself shouldn’t be going through Caddy. What you want to do is have all of your Caddy webserver traffic on port 443. Your reverse proxies should be on different ports Your Caddyfile should look something like this

vpn.eltomation.com {
  import logging
  import certs
  reverse_proxy https://192.168.16.4:1443 # You should be able to set a different port for the Web GUI here and have Caddy reverse proxy it
}
admin.vpn.eltomation.com {
  import logging
  import certs
  reverse_proxy https://192.168.16.4:943 # You should be able to reverse proxy the admin page with Caddy using this
}

Note that since you are only hosting the VPN on port 1194, you don’t need a webpage for it. Since you have a VPN on port 443, you have a few different options:

  • Make Caddy use TCP and the VPN use UDP. That way you can have both binded at the same time. You can do this by removing experimental HTTP/3 support
  • Move the VPN on a different port like 1195 or 2443

I assume you’re using OpenVPN Access Server? You may even be able to set the root to where the web files are located so you would replace the reverse_proxy line with this:

root * /path/to/access-server-webfiles

So, unless you have a reason to keep the VPN itself on port 443, you should move it off of it and use something like 1195 or 1294 for the other one.

Also, for the sake of making it easier on yourself, remove the experimental http3 from your file. Try and get it working and once you do, you can try adding back the experimental http3 block and see if it breaks anything.

1 Like

The OpenVPN and Caddy are not on the same server.
I have more services running trough Caddy. a password manager, file server and unifi.
All those services also run on port 443 and do work perfectly.

Are you sure that mapping it to another port will fix this?
Because it should matter to what port it is proxy-ed, should it? (as long as that service does run at that port)

Not exactly sure how you’re doing that. Generally only one service with one protocol can be on a port. If the VPN and Caddy are on different servers, then how are you using a local IP for all of those services?

Caddy shouldn’t have anything to do with the actual connection to the VPN itself. I think Access Server only lets you see statistics, create users, and download a preprovisioned installer for OpenVPN. You should be able to connect to the VPN itself without using Caddy at all.

At this point I’m honestly kind of lost because you haven’t told me anything you’ve done to try and fix the problem and I don’t have any additional info about your server.

1 Like

Ok, so to be complete.

We have a WAN ip with multiple FQDN’s pointing to it.
From our firewall (FortiGate) we have a rule that NAT’s (and allows) traffic from WAN to the Caddy server.
Port 80 to port 80, port 443 to 443, 1194 to 1194 and so on.

The IP of the Caddy server is 10.99.99.100. this runs in our DMZ.

We have a few services running in our network, most of them on its own server.
OpenVPN for domain users on 192.168.16.9
OpenVPN for customers on 192.168.16.4
a File service on 192.168.16.5
a password manager on 192.168.16.6
Ubiquiti Unifi on 192.168.16.7
VoIP system with web interface on 192.168.16.10

Most of them are running on 443 as webservice.
the FQDN internally resolves to the 192.168.16.* IP. These services have a valid wildcard certificate.
From the outside these FQDN’s point to the WAN address. Where it is NAT-ed to the reverse-proxy (Caddy) who gets the information that is requested from the local 192.168.16.* devices.

So, let’s say you’re on the outside, requesting openvpn.eltomation.com. Then you’ll be pointed to the WAN of our router.
There the request gets NAT-ed to 10.99.99.100 (Caddy) where it finds the openvpn part and sees that the request should go to 192.168.16.4.

And this does work, for the WebGUI of OpenVPN, for the file service, for the password manager, for the VoIP WebGUI and so on.
But it doesn’t work for the actual OpenVPN VPN connection.
Not on the 1194 nor on the 443 ports.

So it looks like Caddy cán reverse-proxy the webpages and files, but can’t reverse-proxy the VPN connection.

To my understanding, a reverse-proxy will always stay between the system making the request en the service that needs to answer.
So the system making the request talks to Caddy and Caddy talks to the service.
Thus the system making the request doesn’t know that the service is on another IP address and/or an other port.

Just as an extra piece of info.
The reason I need/want OpenVPN to be available on 443 is that some of our customers are behind a very strict firewall. They can only go to their ‘outside’ on port 80 and 443. Thus making port 1194 (or any other port) unavailable.
But, connecting over 443 does work.
However. We also run our file server, password manager and so on.
We need/want those on 443 as well.

I know, ideally we could run one OpenVPN on 1194 and the other at 1195 and NAT them to the correct OpenVPN servers. But that means some customers with the port restrictions can’t connect to us.
thus the need for a system (reverse-proxy) that points 443 traffic to the correct local IP depending on the FQDN request.

Thanks for all of the informative info. Yes, this is correct. Caddy does not (as far as I’m aware, out of the box) proxy the VPN connection itself. That should be the job of the OpenVPN software itself on your server. Caddy is reverse proxying the web gui, not the actual VPN connection itself. You can run Caddy and the VPN itself on port 443. My first original post shows that it is possible. However, Caddy is running on TCP and WireGuard on UDP. If I were to enable experimental HTTP/3 support, Caddy would fail to start (or WireGuard would, whichever first) because the protocol is taken). You’re right that each internal IP can listen on the same port, I didn’t know that’s what you meant.

If Caddy were routing the traffic, then it would sort of be in the middle. The point of a VPN is encrypted traffic only between the software itself and the client itself. Does that make sense? It looks like you have everything setup correctly, but no, Caddy will not be able to route that actual VPN traffic itself. If it can, then someone else will have to correct me on that.

If you want to play with proxying TCP/UDP traffic (raw traffic, not HTTP), then the caddy-l4 plugin can do that:

But like @Telesphoreo said, Caddy by default is an HTTP server, not a TCP/UDP server. It doesn’t really make sense to use Caddy to proxy that traffic, because there’s not really any value-add; Caddy can’t magically add TLS to TCP/UDP connections without the other end actually supporting the appropriate protocols to make that happen (i.e. HTTP2 for TCP + TLS, and/or HTTP3 for UDP + TLS).

1 Like

What should I use to proxy TCP/UDP traffic?

I am looking for a system/method to nat/route/proxy the traffic to the correct internal services using its FQDN.
So openvpn.eltomation.com 1194 and 443 are pointed to 192.168.16.9
the files part is pointed to 192.168.16.5 and so on.

You won’t be able to proxy raw TCP/UDP traffic by hostname, because the hostname isn’t sent along in a standard way in raw traffic. That part is an extension of the HTTP protocol called SNI

TCP and UDP packets only have the IP address of their intended destination. The client will resolve the DNS then just send packets in its own format.

You can still use caddy-l4 to proxy the traffic but you won’t be able to make a decision based on hostname.

Caddy-l4 does let you do this.

This topic was automatically closed after 30 days. New replies are no longer allowed.