Reverse-proxy request not seen on second server

Help
2

Howdy @Aditya_Rajagopal, welcome to the Caddy community! Thanks for filling out the Help template and in particular providing those debug logs - they will help show what’s going on.

So, in order to explain why this is happening and how to fix it, first I need to demonstrate a specific nuance when it comes to Caddy server: Caddy responds to UNCONFIGURED requests with a Status 200 (OK). For a little bit of background on this, you can have a read of this informational post: Why Caddy emits empty 200 OK responses by default

To quickly show you what this means, though, I can start Caddy with this Caddyfile:

http://localhost {
  respond 403
}

And try to curl this address:

➜ curl -I localhost
HTTP/1.1 403 Forbidden
Server: Caddy
Date: Wed, 03 Jul 2024 05:16:11 GMT

A response of 403, exactly what we expected. But if we try to access it via the loopback IP address…

➜ curl -I 127.0.0.1
HTTP/1.1 200 OK
Server: Caddy
Date: Wed, 03 Jul 2024 05:16:07 GMT

A response of 200. Why?

Because in the Caddyfile, I have only configured respond 403 for http://localhost. I did not tell Caddy what I want it to do for requests to http:///127.0.0.1. Therefore, since it has no configuration for that site, it will respond with an empty status 200.

If I wanted BOTH to give 403 responses, I would need to add the second site address - so, I’d have http://localhost, http://127.0.0.1 in my Caddyfile. Or, if I wanted it to respond 403 for ANY hostname over HTTP, I could use a site address of just http://.

Now, with that information in mind, lets have a look at your http.handlers.reverse_proxy debug log:

{
  "upstream": "3.235.124.246:80",
  "duration": 0.004377609,
  "request": {
    "remote_ip": "143.58.200.28",
    "remote_port": "55204",
    "client_ip": "143.58.200.28",
    "proto": "HTTP/1.1",
    "method": "GET",
    "host": "52.87.249.192",
    "uri": "/caddyajlfk",
    "headers": {
      "User-Agent": [
        "curl/8.4.0"
      ],
      "Accept": [
        "*/*"
      ],
      "X-Forwarded-For": [
        "143.58.200.28"
      ],
      "X-Forwarded-Proto": [
        "http"
      ],
      "X-Forwarded-Host": [
        "52.87.249.192"
      ]
    }
  },
  "headers": {
    "Server": [
      "Caddy"
    ],
    "Date": [
      "Tue, 02 Jul 2024 17:50:00 GMT"
    ],
    "Content-Length": [
      "0"
    ]
  },
  "status": 200
}

Caddy responded with an empty 200 status. This is the same response we expect when Caddy is UNCONFIGURED for this route. Why?

Take a look at the request.host in the debug log: “52.87.249.192”. When you made a curl request to the first Caddy via its IP address, the first Caddy contacted the upstream Caddy and passed along the hostname you originally requested - that is, the IP address.

But you haven’t configured the upstream Caddy to respond to requests for this IP address. You have only configured the upstream Caddy to respond to requests for the host 3.235.124.246 over HTTP on port 80.

So how do you fix it?

There’s two ways to fix this issue. You can either configure the first Caddy to tell the upstream Caddy what it wants to see: that involves overriding the Host header being sent upstream, so that the upstream Caddy sees you requesting 3.235.124.246.

Or, you can configure the upstream Caddy to accept and respond to requests for 52.87.249.192. This might seem a little unintuitive since the upstream Caddy isn’t listening on this IP, but all it’s really doing is matching the requested Host.

Alternatively - you can sidestep this little nuance of IP addresses by using actual hostnames; domain names that actually resolve to IPs. Regardless of the IP address that’s being connected to, the original domain name that your client requests is going to become the Host header that carries through to the end. So if both Caddy servers were configured with the site mydomain.example.com and you reverse-proxy from one to the other, it’ll just work, regardless of IPs.

1 Like