Reverse proxy upstream help

[Elden (Jonathan Elden)]

1. Output of caddy version:

2.6.2

2. How I run Caddy:

Docker

a. System environment:

xcaddy build docker 2.6.2 alpine

b. Command:

 caddy start

c. Service/unit/compose file:

FROM caddy:2.6.2-builder AS builder

RUN xcaddy build \
    --with github.com/gamalan/caddy-tlsredis 
    
FROM caddy:2.6.2

COPY --from=builder /usr/bin/caddy /usr/bin/caddy

RUN mkdir -p /usr/caddy
COPY Caddyfile /etc/caddy


RUN caddy start

d. My complete Caddy config:

{
	on_demand_tls {
		interval 2m
		burst 5
	}

	storage redis {
		host "{$SSL_PROXY_REDIS_HOST}"
		port {$SSL_PROXY_REDIS_PORT}
		address "" // no default, but is build from host+":"+port, if set, then host and port is ignored
		username ""
		password "{$SSL_PROXY_REDIS_PASSWORD}"
		db {$SSL_PROXY_REDIS_DB}
		key_prefix "caddytls"
		value_prefix "caddy-storage-redis"
		timeout 5
		tls_enabled "false"
		tls_insecure "true"
	}
}

	http:// {
		respond /health 200
	}


	https:// {
		tls {
		
		on_demand
	}
	reverse_proxy {$SSL_PROXY_UPSTREAM} 
}

3. The problem I’m having:

edited:
i’m getting a

{"level":"error","ts":1669253345.7200336,"logger":"http.handlers.reverse_proxy","msg":"aborting with incomplete response","error":"http2: stream closed"}

error when it attempts to connect to the upstream; this upstream works with nginx and other reverse proxies so i’m a bit stumped . (the upstream is listening on http port 80 and the variable is an http:// address

This error happens immediately upon accessing a page that would make caddy hit the upstream.

4. Error messages and/or full log output:

{"level":"info","ts":1669253138.9718034,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2","h3"]}
{"level":"info","ts":1669253138.9718316,"logger":"http.log","msg":"server running","name":"srv1","protocols":["h1","h2","h3"]}
{"level":"info","ts":1669253138.9723413,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
{"level":"info","ts":1669253138.9723556,"msg":"serving initial configuration"}
{"level":"error","ts":1669253345.7200336,"logger":"http.handlers.reverse_proxy","msg":"aborting with incomplete response","error":"http2: stream closed"}

5. What I already tried:

Removing and adding transport http; putting the full url in instead of the env variable for the proxy upstream (http://prometheus1.342434.svc.cluster.local)
changing the https:// to https:// / .

From the logs it’s connecting redis fine; requesting certificates fine and just refusing to do anything related to the reverse proxy with no errors and i’m stumped but hoping it’s a simple syntax fubar on my part

6. Links to relevant resources:

[emilylange]

Caddy passes the Host header to the upstream unmodified.
So the hostname/domain your Client (e.g. web browser) sends to Caddy will be the one seen by the upstream nginx).

Maybe your nginx is looking for a specific Host header, not the one Caddy is passing all the way from the client.

You can override it the Host header to the upstream by using

reverse_proxy {$SSL_PROXY_UPSTREAM} {
	header_up Host {upstream_hostport}
}

as shown all the way down in docs/caddyfile/directives/reverse_proxy#https.
But that’s just a guess from me

Could you share some curl outputs from wherever Caddy is running to your upstream target?
That might provide more details