Reverse Proxy for Automatic HTTPS - Refuses to connect when not local

[rkechols (Ryan Echols)]

1. Caddy version (caddy version):

$ caddy version
v2.3.0 h1:fnrqJLa3G5vfxcxmOH/+kJOcunPLhSBnjgIvjXV/QTA=

2. How I run Caddy:

a. System environment:

  Operating System: Ubuntu 20.04.2 LTS
            Kernel: Linux 5.4.0-62-generic
      Architecture: x86-64

Running a server on localhost:8081 using Python 3.8.5.
Running caddy without a Caddyfile.

b. Command:

$ sudo caddy reverse-proxy --from cs236.cs.byu.edu --to localhost:8081

c. Service/unit/compose file:

None

d. My complete Caddyfile or JSON config:

Auto-generated file /root/.config/caddy/autosave.json:

{
    "admin":{
        "disabled":true
    },
    "apps":{
        "http":{
            "servers":{
                "proxy":{
                    "listen":[
                        ":443"
                    ],
                    "routes":[
                        {
                            "handle":[
                                {
                                    "handler":"reverse_proxy",
                                    "transport":{
                                        "protocol":"http"
                                    },
                                    "upstreams":[
                                        {
                                            "dial":"localhost:8081"
                                        }
                                    ]
                                }
                            ],
                            "match":[
                                {
                                    "host":[
                                        "cs236.cs.byu.edu"
                                    ]
                                }
                            ]
                        }
                    ]
                }
            }
        }
    }
}

3. The problem I’m having:

I’m trying to set up an HTTPS server by running the server normally (HTTP) on port 8081 and running caddy reverse-proxy (full command given above).
I can do curl localhost:8081 and get the correct response.
I can do curl https://cs236.cs.byu.edu from the local machine and get the correct response.
If I do curl https://cs236.cs.byu.edu from any other machine then I get a “Connection refused” error.

4. Error messages and/or full log output:

This is what it looks like when I start caddy

$ sudo caddy reverse-proxy --from cs236.cs.byu.edu --to localhost:8081
2021/04/02 08:18:03.657 WARN    admin   admin endpoint disabled
2021/04/02 08:18:03.658 INFO    http    server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS {"server_name": "proxy", "https_port": 443}
2021/04/02 08:18:03.659 INFO    http    enabling automatic HTTP->HTTPS redirects        {"server_name": "proxy"}
2021/04/02 08:18:03.660 INFO    http    enabling automatic TLS certificate management   {"domains": ["cs236.cs.byu.edu"]}
2021/04/02 08:18:03.658 INFO    tls.cache.maintenance   started background certificate maintenance      {"cache": "0xc00022c7e0"}
2021/04/02 08:18:03.662 INFO    tls     cleaned up storage units
2021/04/02 08:18:03.680 INFO    autosaved config        {"file": "/root/.config/caddy/autosave.json"}
Caddy proxying https://cs236.cs.byu.edu -> http://localhost:8081

If I then do curl https://cs236.cs.byu.edu/ from that same machine, I get the correct response.
If I then do curl https://cs236.cs.byu.edu/ from a different machine, I get this error:

$ curl https://cs236.cs.byu.edu
curl: (7) Failed to connect to cs236.cs.byu.edu port 443: Connection refused

5. What I already tried:

I’ve tried:

• Stopping the caddy process and re-starting it without deleting /root/.config/caddy/autosave.json
• Stopping the caddy process and re-starting it, including deleting /root/.config/caddy/autosave.json
• Trying both curl and a browser (Google Chrome) to connect from the other machine.
• Connecting from the remote machine using various URLs that use different combinations of http/https and specifying the port vs not.
  With this, I’ve noticed that going to http://cs236.cs.byu.edu does indeed redirect to https://cs236.cs.byu.edu, but then it gives the same Connection refused error.

[francislavoie]

Is that different machine in the LAN? If so, it might be because your router doesn’t have NAT hairpinning enabled. You could try from a device outside your LAN, like your cell phone over your cell connection to see if it connects.

If it also doesn’t work from outside your network, then you must have some firewall or port forwarding problem.

[rkechols (Ryan Echols)]

It looks like it was the port forwarding. Turns out there was something already set up that was interfering
Here’s what was already set up in port forwarding:

$ sudo iptables -t nat -L --line-numbers
Chain PREROUTING (policy ACCEPT)
num  target     prot opt source               destination
1    REDIRECT   tcp  --  anywhere             anywhere             tcp dpt:https redir ports 8080

Chain INPUT (policy ACCEPT)
num  target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
num  target     prot opt source               destination

Chain POSTROUTING (policy ACCEPT)
num  target     prot opt source               destination

So I ran sudo iptables -t nat -D PREROUTING 1 to get rid of that forwarding and then set things back up, and that fixed it! Thanks for pointing out that it might be port forwarding!