1. The problem I’m having:
Hi there 
I’m running 2 Servers hosting different things, when using forward_auth on Server 2, after authenticating successfully, request to the page protected by caddy’s forward_auth fail with http 502.
Here’s an overview:
Server 1
- caddy, with port 80 and 443 exposed
- authentik, with port 9000 exposed to my tailscale network
- kopia
Server 2
- caddy, with port 80 and 443 exposed
- kopia
Here’s my Caddyfile for Server 1:
{
email admin@watn3y.de
debug
}
auth.watn3y.de {
reverse_proxy authentik-server:9000
}
backup.balthasar.watn3y.net {
reverse_proxy /outpost.goauthentik.io/* http://100.75.208.132:9000
forward_auth http://100.75.208.132:9000 {
uri /outpost.goauthentik.io/auth/caddy
copy_headers X-Authentik-Username X-Authentik-Groups X-Authentik-Email X-Authentik-Name X-Authentik-Uid X-Authentik-Jwt X-Authentik-Meta-Jwks X-Authentik-Meta-Outpost X-Authentik-Meta-Provider X-Authentik-Meta-App X-Authentik-Meta-Version
trusted_proxies private_ranges
}
reverse_proxy kopia:51515
}
Here’s my Caddyfile for Server 2:
{
email admin@watn3y.de
debug
}
backup.melchior.watn3y.net {
reverse_proxy /outpost.goauthentik.io/* http://100.75.208.132:9000
forward_auth http://100.75.208.132:9000 {
uri /outpost.goauthentik.io/auth/caddy
copy_headers X-Authentik-Username X-Authentik-Groups X-Authentik-Email X-Authentik-Name X-Authentik-Uid X-Authentik-Jwt X-Authentik-Meta-Jwks X-Authentik-Meta-Outpost X-Authentik-Meta-Provider X-Authe> trusted_proxies private_ranges
}
# actual site configuration below, for example
reverse_proxy kopia:51515
}
(before you ask, those names are technically a reference to Neon Genesis Evangelion, not the bible :D)
Now the actual Problem:
When trying to access backup.balthasar.watn3y.net everything works just fine, caddy redirects me on to auth.watn3y.de, i authenticate myself and again get redirected to backup.balthasar.watn3y.net.
When I then try to access backup.melchior.watn3y.net I also get redirected to auth.watn3y.de, I authenticate myself, but loading backup.melchior.watn3y.net fails with http 502.
Here’s the specific line in the logs (full debug log in #2:
ERR | ts=1698641317.4063654 logger=http.log.error msg=net/http: invalid header field name "" request={"remote_ip":"192.168.0.21","remote_port":"58490","client_ip":"192.168.0.21","proto":"HTTP/3.0","method":"GET","host":"backup.melchior.watn3y.net","uri":"/","headers":{"Accept-Language":["en-DE,en;q=0.9"],"Sec-Fetch-User":["?1"],"Accept-Encoding":["gzip, deflate, br"],"Sec-Fetch-Site":["cross-site"],"Sec-Fetch-Mode":["navigate"],"Cookie":[],"Sec-Ch-Ua-Mobile":["?0"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7"],"Cache-Control":["max-age=0"],"Sec-Ch-Ua":["\"Chromium\";v=\"118\", \"Google Chrome\";v=\"118\", \"Not=A?Brand\";v=\"99\""],"Sec-Fetch-Dest":["document"],"Sec-Ch-Ua-Platform":["\"Windows\""],"Upgrade-Insecure-Requests":["1"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h3","server_name":"backup.melchior.watn3y.net"}} duration=0.025956439 status=502 err_id=8pdhe32k7 err_trace=reverseproxy.statusError (reverseproxy.go:1265)
2. Error messages and/or full log output:
3. Caddy version:
Is the same on both Servers
v2.7.5 h1:HoysvZkLcN2xJExEepaFHK92Qgs7xAiCFydN5x5Hs6Q=
4. How I installed and ran Caddy:
According to Keep Caddy Running — Caddy Documentation with very few changes
a. System environment:
Server 1:
- Ubuntu 22.04.3 LTS
- Docker Engine 24.0.7
Server 2:
- Debian GNU/Linux trixie/sid
- Docker Engine 24.0.7
