Caddy fails (with 502) on half my reverse_proxy entries

1. The problem I’m having:

I have a Caddyfile (see below) whose only job it is to reverse proxy a bunch of localhost domains to localhost:port addresses, one backed by a nodejs server running on port 8000, and the rest backed by Docker containers. The docker ps command confirms that my containers are running, and shows correct port bindings, and loading each http://localhost: using curl works fine, resolving to the webservers running in each container.

However, caddy only works for the first two of the four reverse proxies, resolving to seemingly completely different ports and generating 502’s for the latter two others.

For completeness, the docker ps output is:

CONTAINER ID   IMAGE     COMMAND         CREATED        STATUS                  PORTS                                           NAMES
0cd073ed4964   nodejs    "sh ./run.sh"   37 hours ago   Up 37 hours (healthy)   0.0.0.0:54253->8000/tcp, [::]:54253->8000/tcp   nodejs
fabef5acc8ba   pomax     "sh ./run.sh"   37 hours ago   Up 37 hours (healthy)   0.0.0.0:54191->8000/tcp, [::]:54191->8000/tcp   pomax
0395f24ecdac   lame      "sh ./run.sh"   2 days ago     Up 2 days (healthy)     0.0.0.0:57830->8000/tcp, [::]:57830->8000/tcp   lame

2. Error messages and/or full log output:

A curl lookup for https://nodejs.app.localhost shows:

* Host nodejs.app.localhost:443 was resolved.
* IPv6: ::1
* IPv4: 127.0.0.1
*   Trying [::1]:443...
* Connected to nodejs.app.localhost (::1) port 443
* ALPN: curl offers h2,http/1.1
* (304) (OUT), TLS handshake, Client hello (1):
*  CAfile: /etc/ssl/cert.pem
*  CApath: none
* (304) (IN), TLS handshake, Server hello (2):
* (304) (IN), TLS handshake, Unknown (8):
* (304) (IN), TLS handshake, Certificate (11):
* (304) (IN), TLS handshake, CERT verify (15):
* (304) (IN), TLS handshake, Finished (20):
* (304) (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / AEAD-CHACHA20-POLY1305-SHA256 / [blank] / UNDEF
* ALPN: server accepted h2
* Server certificate:
*  subject: [NONE]
*  start date: Jul 27 12:29:41 2025 GMT
*  expire date: Jul 28 00:29:41 2025 GMT
*  subjectAltName: host "nodejs.app.localhost" matched cert's "nodejs.app.localhost"
*  issuer: CN=Caddy Local Authority - ECC Intermediate
*  SSL certificate verify ok.
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://nodejs.app.localhost/
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: nodejs.app.localhost]
* [HTTP/2] [1] [:path: /]
* [HTTP/2] [1] [user-agent: curl/8.7.1]
* [HTTP/2] [1] [accept: */*]
> GET / HTTP/2
> Host: nodejs.app.localhost
> User-Agent: curl/8.7.1
> Accept: */*
> 
* Request completely sent off
< HTTP/2 502 
< alt-svc: h3=":443"; ma=2592000
< server: Caddy
< content-length: 0
< date: Sun, 27 Jul 2025 16:29:21 GMT
< 
* Connection #0 to host nodejs.app.localhost left intact

With this associated caddy error:

ERROR
http.log.error
dial tcp 127.0.0.1:56359: connect: connection refused   
{
  "request": {
    "remote_ip": "::1",
    "remote_port": "58506",
    "client_ip": "::1",
    "proto": "HTTP/2.0",
    "method": "GET",
    "host": "nodejs.app.localhost",
    "uri": "/",
    "headers": { "User-Agent": ["curl/8.7.1"], "Accept": ["*/*"] },
    "tls": {
      "resumed": false,
      "version": 772,
      "cipher_suite": 4867,
      "proto": "h2",
      "server_name": "nodejs.app.localhost"
    }
  },
  "duration": 0.002264209,
  "status": 502,
  "err_id": "4bp27jsez",
  "err_trace": "reverseproxy.statusError (reverseproxy.go:1390)"
}

For completeness, a curl for the associated http://localhost:54253 shows:

* Host localhost:54253 was resolved.
* IPv6: ::1
* IPv4: 127.0.0.1
*   Trying [::1]:54253...
* connect to ::1 port 54253 from ::1 port 58687 failed: Connection refused
*   Trying 127.0.0.1:54253...
* Connected to localhost (127.0.0.1) port 54253
> GET / HTTP/1.1
> Host: localhost:54253
> User-Agent: curl/8.7.1
> Accept: */*
> 
* Request completely sent off
< HTTP/1.1 200 OK
< X-Powered-By: Express
< Content-Type: text/html; charset=utf-8
< Content-Length: 15
< ETag: W/"f-r1NvCP6NiRUAkgXPHbwx7mVakRg"
< Date: Sun, 27 Jul 2025 16:55:43 GMT
< Connection: keep-alive
< Keep-Alive: timeout=5
< 
Node server running on port 8000, exec for "python3 --version" shows: Python 3.12.11
* Connection #0 to host localhost left intact

3. Caddy version:

v2.10.0 h1:fonubSaQKF1YANl8TXqGcn4IbIRUDdfAkpcsfI/vX5U=

4. How I installed and ran Caddy:

This is a standard brew install on MacOS

a. System environment:

MacOS 13.7.5 (22H527)

b. Command:

brew install caddy

c. Service/unit/compose file:

Caddy is not being used inside a container configuration, it is simply invoked using caddy start, caddy reload, and caddy stop.

d. My complete Caddy config:

editor.com.localhost {
	reverse_proxy localhost:8000
}

lame.app.localhost {
	reverse_proxy localhost:57830
}

nodejs.app.localhost {
	reverse_proxy localhost:54253
}

pomax.app.localhost {
	reverse_proxy localhost:54191
}

5. Links to relevant resources:

No links, but the editor.com.localhost and lame.app.localhost lookups seem to work fine, resolving to services running on :8000 and :57830, but nodejs.app.localhost and pomax.app.localhost seem to use completely random ports that aren’t found anywhere in the config.

Running caddy start notes that it autosaved the config to /Users/pomax/Library/Application Support/Caddy/autosave.json but when I open that up, the ports as noted above are the same as the ports listed in that JSON file.

Running caddy reload does not change the routing behaviour

As it turns out, this was caused by caddy running more than one instance. Apparently, some previous run did not end up killing its caddy process and that instance kept “winning” the right to reverse proxy requests.

That’s fairly unexpected, you’d expect caddy to throw an error going “actually, another instance is already running for the following domain(s) in your config, exiting: [list of domains]” so that you can’t get into mystery routing territory =/

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.