Reverse proxy upstream help

1. Output of caddy version:


2. How I run Caddy:


a. System environment:

xcaddy build docker 2.6.2 alpine

b. Command:

 caddy start

c. Service/unit/compose file:

FROM caddy:2.6.2-builder AS builder

RUN xcaddy build \
FROM caddy:2.6.2

COPY --from=builder /usr/bin/caddy /usr/bin/caddy

RUN mkdir -p /usr/caddy
COPY Caddyfile /etc/caddy

RUN caddy start

d. My complete Caddy config:

	on_demand_tls {
		interval 2m
		burst 5

	storage redis {
		address "" // no default, but is build from host+":"+port, if set, then host and port is ignored
		username ""
		key_prefix "caddytls"
		value_prefix "caddy-storage-redis"
		timeout 5
		tls_enabled "false"
		tls_insecure "true"

	http:// {
		respond /health 200

	https:// {
		tls {
	reverse_proxy {$SSL_PROXY_UPSTREAM} 

3. The problem I’m having:

i’m getting a

{"level":"error","ts":1669253345.7200336,"logger":"http.handlers.reverse_proxy","msg":"aborting with incomplete response","error":"http2: stream closed"}

error when it attempts to connect to the upstream; this upstream works with nginx and other reverse proxies so i’m a bit stumped . (the upstream is listening on http port 80 and the variable is an http:// address

This error happens immediately upon accessing a page that would make caddy hit the upstream.

4. Error messages and/or full log output:

{"level":"info","ts":1669253138.9718034,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2","h3"]}
{"level":"info","ts":1669253138.9718316,"logger":"http.log","msg":"server running","name":"srv1","protocols":["h1","h2","h3"]}
{"level":"info","ts":1669253138.9723413,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
{"level":"info","ts":1669253138.9723556,"msg":"serving initial configuration"}
{"level":"error","ts":1669253345.7200336,"logger":"http.handlers.reverse_proxy","msg":"aborting with incomplete response","error":"http2: stream closed"}

5. What I already tried:

Removing and adding transport http; putting the full url in instead of the env variable for the proxy upstream (http://prometheus1.342434.svc.cluster.local)
changing the https:// to https:// / .

From the logs it’s connecting redis fine; requesting certificates fine and just refusing to do anything related to the reverse proxy with no errors and i’m stumped but hoping it’s a simple syntax fubar on my part

6. Links to relevant resources:

Caddy passes the Host header to the upstream unmodified.
So the hostname/domain your Client (e.g. web browser) sends to Caddy will be the one seen by the upstream nginx).

Maybe your nginx is looking for a specific Host header, not the one Caddy is passing all the way from the client.

You can override it the Host header to the upstream by using

reverse_proxy {$SSL_PROXY_UPSTREAM} {
	header_up Host {upstream_hostport}

as shown all the way down in docs/caddyfile/directives/reverse_proxy#https.
But that’s just a guess from me :woman_shrugging:

Could you share some curl outputs from wherever Caddy is running to your upstream target?
That might provide more details :innocent:

1 Like