Forward_auth copy_headers value not replaced

Help
1

1. Output of caddy version:

v2.5.2 h1:eCJdLyEyAGzuQTa5Mh3gETnYWDClo1LjtQm2q9RNZrs

2. How I run Caddy:

I’m developing a custom auth API to work with Caddy.

request ---> Caddy (1) ---> Backend API Server (3), localhost:58806
               |  ^
               |  | 200 OK, 401, etc.
               V  |
            Auth Server (2), localhost:8080

(2) will resolve the Authorization header like most authentication server would do, and return a 200 with a few X-UBR-* headers with user information. And I’m trying to config Caddy to copy the values of those headers to (3).

$env:SITE_ADDRESS='uber.localtest.me'; caddy run --watch

a. System environment:

OS: Windows 10, 19044.1889

b. Command:

In PowerShell

$env:SITE_ADDRESS='uber.localtest.me'; caddy run --watch

c. Service/unit/compose file:

N/A

d. My complete Caddy config:

{
	admin localhost:3019
}

(headers) {
	header @origin Access-Control-Allow-Origin "{args.0}"
	header @origin Access-Control-Allow-Methods "OPTIONS,HEAD,GET,POST,PUT,PATCH,DELETE"
	header @origin Access-Control-Allow-Headers "Content-Type,X-Requested-With,Authorization"
}

(cors) {
	@origin header Origin "{args.0}"
	import headers {args.0}
}

localhost {
	# respond "Hello, world!"
	# file_server
	reverse_proxy 127.0.0.1:58806
}

api.{$SITE_ADDRESS} {
	tls internal
	# log
	log

	@preflight method OPTIONS
	handle @preflight {
		respond "OK"
		import cors {header.origin}
	}

	forward_auth localhost:8080 {
		uri /auth
		copy_headers {
			X-UBR-AUTH-OK
			X-UBR-AUTH-USER
			X-UBR-SUB
			X-UBR-USERPOOL
		}
	}

	reverse_proxy localhost:58806 {
		header_up Host "localhost"
		# header_up X-UBR-AUTH_USER {http.reverse_proxy.header.X-UBR-AUTH_USER}
		header_down Access-Control-Allow-Origin {header.origin}
		header_down Access-Control-Allow-Methods "OPTIONS,HEAD,GET,POST,PUT,PATCH,DELETE"
		header_down Access-Control-Allow-Headers "Content-Type,X-Requested-With"
	}

	reverse_proxy localhost:8090 {
		header_up Host "localhost"
		@notimplemented1 status 501
		@notimplemented2 status 502

		handle_response @notimplemented1 {
			# encode zstd gzip br
			reverse_proxy 127.0.0.1:58806 {
				header_up Host "localhost"
				header_down Access-Control-Allow-Origin {header.origin}
				header_down Access-Control-Allow-Methods "OPTIONS,HEAD,GET,POST,PUT,PATCH,DELETE"
				header_down Access-Control-Allow-Headers "Content-Type,X-Requested-With"
			}
		}

		handle_response @notimplemented2 {
			reverse_proxy 127.0.0.1:58807 {
				# header_up Host "localhost"
				header_down Access-Control-Allow-Origin {header.origin}
				header_down Access-Control-Allow-Methods "OPTIONS,HEAD,GET,POST,PUT,PATCH,DELETE"
				header_down Access-Control-Allow-Headers "Content-Type,X-Requested-With"
			}
		}

		header_down Access-Control-Allow-Origin {header.origin}
		header_down Access-Control-Allow-Methods "OPTIONS,HEAD,GET,POST,PUT,PATCH,DELETE"
		header_down Access-Control-Allow-Headers "Content-Type,X-Requested-With"
	}
}

{$SITE_ADDRESS} {
	tls internal
	# log
	# import cors uber.localtest.me
	reverse_proxy 127.0.0.1:58806 {
		header_up Host "localhost"
	}
}

3. The problem I’m having:

My auth server is responding with X-UBR-* headers, that I want Caddy to copy to the backend server requests, but I couldn’t see the header value from (3). My server is returning:

{:status 200,
 :headers
 {"X-UBR-AUTH-OK" "Yes",
  "X-UBR-AUTH-USER" "yuan.lin@nykgroup.com",
  "X-UBR-SUB" "2c0d1037-dbe5-4027-8786-c50c0c3810a9",
  "X-UBR-USERPOOL" "us-east-1_hbLoUmYGr"},
 :body "OK"}

But I’m seeing the below value from (3)

Accept: application/json, text/javascript, */*; q=0.01
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US, en; q=0.9
Authorization: Basic ZXlKcmFXUWlPaUo1UzJ4eU0xTnVWVnBLUjF3dlJHZzVZVWxpTkVWcU9ETjFjMmR0YVVkYVluRlNhMmRHUm5GR1UzSkRSVDBpTENKaGJHY2lPaUpTVXpJMU5pSjkuZXlKaGRGOW9ZWE5vSWpvaWJWZFdRbXh5TlVZMFJXaHBhaTFKTkU1d1MzY3daeUlzSW5OMVlpSTZJakpqTUdReE1ETTNMV1JpWlRVdE5EQXlOeTA0TnpnMkxXTTFNR013WXpNNE1UQmhPU0lzSW1WdFlXbHNYM1psY21sbWFXVmtJanAwY25WbExDSnBjM01pT2lKb2RIUndjenBjTDF3dlkyOW5ibWwwYnkxcFpIQXVkWE10WldGemRDMHhMbUZ0WVhwdmJtRjNjeTVqYjIxY0wzVnpMV1ZoYzNRdE1WOW9Za3h2VlcxWlIzSWlMQ0pqYjJkdWFYUnZPblZ6WlhKdVlXMWxJam9pTW1Nd1pERXdNemN0WkdKbE5TMDBNREkzTFRnM09EWXRZelV3WXpCak16Z3hNR0U1SWl3aVlYVmtJam9pTW1ScmFUbDFaWFkwTUhCeU1XWnhZVGhoZFhCcVpYQmpaakVpTENKbGRtVnVkRjlwWkNJNklqTmxPR014WVRneExXVTVPVFl0TkRsaU5TMWlOR0l4TFRBMU9UUTBaV0ZqWldSaFlpSXNJblJ2YTJWdVgzVnpaU0k2SW1sa0lpd2lZWFYwYUY5MGFXMWxJam94TmpZeE56UXdNVFU1TENKbGVIQWlPakUyTmpFM05EWTVPRE1zSW1saGRDSTZNVFkyTVRjME16TTRNeXdpWlcxaGFXd2lPaUo1ZFdGdUxteHBia0J1ZVd0bmNtOTFjQzVqYjIwaWZRLmlEc2RyX3N1bzBJNG9HWHJJc2ZLUmlkTHI5enI0WmNvWVN5WUo2eDNqZzdBUGhJdHAwRUVzYUZXdm9VemJneVdZOWpmdVp2RFBzdUpmdGhBbzdCbmlDSHdMcHJFeFd5RHo3R19Rck5GNWd2UW1iWnNqZE1Rc1htUXNwME1kX3F3Tm5makM4THdpWTNxYmpDWW5ZZl9wRk9pX1JocWpMc2VJbE9FdGpuVTZfYVFBbFUxRXd3WHBDWXVjWE4tVW5xRXVZQkVpaEU4eF9Oejc4XzhvY0NJYlF6U2tKLVNsNlVrUUIwR0Q0aTBFbzE4a0laZzd3ajJ3LVF5UERYT09YZl9SVmFFLXYweFhsNzBIN1Zwc0ZIaVNzT2lENy1McV9zQmR4c0k0VV9BYmZIbzZjUWM2RjF5Z0JPX2VoSDdOa0hkZjFBb2RlcnRHdlh3ZEVUWVB5bk9SZw==
Host: localhost
Referer: https://uber.localtest.me/
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36
Origin: https://uber.localtest.me
Sec-Ch-Ua: "Chromium";v="104", " Not A;Brand";v="99", "Google Chrome";v="104"
Sec-Ch-Ua-Mobile: ?0
Sec-Ch-Ua-Platform: "Windows"
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: same-site
X-Forwarded-For: 127.0.0.1
X-Forwarded-Host: api.uber.localtest.me
X-Forwarded-Proto: https
X-Requested-With: XMLHttpRequest
X-Ubr-Auth-Ok: {http.reverse_proxy.header.X-UBR-AUTH-OK}
X-Ubr-Auth-User: {http.reverse_proxy.header.X-UBR-AUTH-USER}
X-Ubr-Sub: {http.reverse_proxy.header.X-UBR-SUB}
X-Ubr-Userpool: {http.reverse_proxy.header.X-UBR-USERPOOL}

4. Error messages and/or full log output:

2022/08/29 04:12:12.683 INFO    http.log.access handled request {"request": {"remote_ip": "127.0.0.1", "remote_port": "3218", "proto": "HTTP/2.0", "method": "GET", "host": "api.uber.localtest.me", "uri": "/internal/api/TMS/?skip=0&take=20&sort=&filter=%7B%22logic%22%3A%22and%22%2C%22filters%22%3A%5B%5D%7D&isAdmin=true&type=NA&effective=8%2F29%2F2022", "headers": {"Authorization": [], "Sec-Ch-Ua-Platform": ["\"Windows\""], "Referer": ["https://uber.localtest.me/"], "Sec-Ch-Ua": ["\"Chromium\";v=\"104\", \" Not A;Brand\";v=\"99\", \"Google Chrome\";v=\"104\""], "Origin": ["https://uber.localtest.me"], "Accept-Encoding": ["gzip, deflate, br"], "Sec-Ch-Ua-Mobile": ["?0"], "Accept": ["application/json, text/javascript, */*; q=0.01"], "Sec-Fetch-Dest": ["empty"], "Accept-Language": ["en-US,en;q=0.9"], "Sec-Fetch-Mode": ["cors"], "Content-Type": ["application/json"], "User-Agent": ["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36"], "X-Requested-With": ["XMLHttpRequest"], "Sec-Fetch-Site": ["same-site"]}, "tls": {"resumed": false, "version": 772, "cipher_suite": 4865, "proto": "h2", "server_name": "api.uber.localtest.me"}}, "user_id": "", "duration": 1.1878341, "size": 3887, "status": 200, "resp_headers": {"Vary": ["Accept-Encoding"], "Access-Control-Allow-Headers": ["Content-Type,X-Requested-With"], "Content-Encoding": ["gzip"], "Access-Control-Allow-Methods": ["OPTIONS,HEAD,GET,POST,PUT,PATCH,DELETE"], "Content-Type": ["application/json; charset=utf-8"], "Expires": ["-1"], "Date": ["Mon, 29 Aug 2022 04:12:12 GMT"], "Pragma": ["no-cache"], "Content-Length": ["3887"], "Access-Control-Allow-Origin": ["https://uber.localtest.me"], "Cache-Control": ["no-cache"], "X-Sourcefiles": ["=?UTF-8?B?RDpcaG9tZVxsaW55MDFcd29ya3NwYWNlXDEtcHJvamVjdHNcbmJzYVxyb3JvLXViZXItd29ya3NwYWNlXE5ZS1ViZXJcaW50ZXJuYWxcYXBpXFRNU1w=?="], "Server": ["Caddy", "Microsoft-IIS/10.0"]}}
2022/08/29 04:12:13.364 INFO    http.log.access handled request {"request": {"remote_ip": "127.0.0.1", "remote_port": "3218", "proto": "HTTP/2.0", "method": "GET", "host": "api.uber.localtest.me", "uri": "/internal/api/TMS/?skip=0&take=20&sort=&filter=%7B%22logic%22%3A%22and%22%2C%22filters%22%3A%5B%5D%7D&isAdmin=true&type=NA&effective=8%2F29%2F2022", "headers": {"Sec-Ch-Ua-Mobile": ["?0"], "X-Requested-With": ["XMLHttpRequest"], "Sec-Fetch-Site": ["same-site"], "Sec-Fetch-Mode": ["cors"], "Accept-Language": ["en-US,en;q=0.9"], "Authorization": [], "Content-Type": ["application/json"], "Sec-Ch-Ua-Platform": ["\"Windows\""], "Accept": ["application/json, text/javascript, */*; q=0.01"], "User-Agent": ["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36"], "Referer": ["https://uber.localtest.me/"], "Sec-Ch-Ua": ["\"Chromium\";v=\"104\", \" Not A;Brand\";v=\"99\", \"Google Chrome\";v=\"104\""], "Origin": ["https://uber.localtest.me"], "Sec-Fetch-Dest": ["empty"], "Accept-Encoding": ["gzip, deflate, br"]}, "tls": {"resumed": false, "version": 772, "cipher_suite": 4865, "proto": "h2", "server_name": "api.uber.localtest.me"}}, "user_id": "", "duration": 0.677728, "size": 3892, "status": 200, "resp_headers": {"Content-Encoding": ["gzip"], "Access-Control-Allow-Headers": ["Content-Type,X-Requested-With"], "Server": ["Caddy", "Microsoft-IIS/10.0"], "Content-Type": ["application/json; charset=utf-8"], "Vary": ["Accept-Encoding"], "X-Sourcefiles": ["=?UTF-8?B?RDpcaG9tZVxsaW55MDFcd29ya3NwYWNlXDEtcHJvamVjdHNcbmJzYVxyb3JvLXViZXItd29ya3NwYWNlXE5ZS1ViZXJcaW50ZXJuYWxcYXBpXFRNU1w=?="], "Access-Control-Allow-Methods": ["OPTIONS,HEAD,GET,POST,PUT,PATCH,DELETE"], "Pragma": ["no-cache"], "Date": ["Mon, 29 Aug 2022 04:12:12 GMT"], "Access-Control-Allow-Origin": ["https://uber.localtest.me"], "Content-Length": ["3892"], "Cache-Control": ["no-cache"], "Expires": ["-1"]}}

5. What I already tried:

It looks like the value is not properly replaced.

I have tried in the official doc and the community to search and read about examples for using copy_headers, however, mostly posts are about existing auth services. And copy_headers are most configured like copy_header Remote-User Remote-Groups ....

6. Links to relevant resources:

2

I also tried to return the user info in Remote-User header, and copy_header works correctly for Remote-User header. Is there any way for Caddy to copy X-UBR-* values in my use case?

The case I also tried:

Info from the auth server (2)

{:status 200,
 :headers
 {"X-UBR-AUTH-OK" "Yes",
  "X-UBR-AUTH-USER" "yuan.lin@nykgroup.com",
  "Remote-User" "yuan.lin@nykgroup.com",
  "X-UBR-SUB" "2c0d1037-dbe5-4027-8786-c50c0c3810a9",
  "X-UBR-USERPOOL" "us-east-1_hbLoUmYGr"},
 :body "OK"}

And app server (3) sees:

...
Remote-User: yuan.lin@nykgroup.com
...
X-Ubr-Auth-Ok: {http.reverse_proxy.header.X-UBR-AUTH-OK}
X-Ubr-Auth-User: {http.reverse_proxy.header.X-UBR-AUTH-USER}
X-Ubr-Sub: {http.reverse_proxy.header.X-UBR-SUB}
X-Ubr-Userpool: {http.reverse_proxy.header.X-UBR-USERPOOL}
3

After some more try-and-error it looks the case for the header matters.

This currently works for me:

{:status 200,
 :headers
 {"X-Ubr-Auth-Ok" "Yes",
  "X-Ubr-Auth-User" "yuan.lin@nykgroup.com",
  "X-Ubr-Sub" "2c0d1037-dbe5-4027-8786-c50c0c3810a9",
  "X-Ubr-User-Pool" "us-east-1_hbLoUmYGr"},
 :body "OK"}

with this caddy config

copy_headers X-Ubr-Auth-Ok X-Ubr-Auth-User X-Ubr-Sub X-Ubr-User-Pool

And I can see the values in app server:

...
X-Ubr-Auth-Ok: Yes
X-Ubr-Auth-User: yuan.lin@nykgroup.com
X-Ubr-Sub: 2c0d1037-dbe5-4027-8786-c50c0c3810a9
X-Ubr-User-Pool: us-east-1_hbLoUmYGr
...

My question then, since HTTP header is case insensitive, this behavior is a bit confusing, maybe add doc about the auth server needs to name headers in Capitalized-kebab-form?

1 Like
4

Yeah, you must use the canonical form of the header fields, which means uppercase for the first letter of each part separated by dashes.

The placeholder replacer is doing a lookup for the header in the response by the exact case you told it to, but only the canonical form will work because that’s how they’re stored in the header map.

The code that sets up the replacer looks like this:

		// set up the replacer so that parts of the original response can be
		// used for routing decisions
		for field, value := range res.Header {
			repl.Set("http.reverse_proxy.header."+field, strings.Join(value, ","))
		}
		repl.Set("http.reverse_proxy.status_code", res.StatusCode)
		repl.Set("http.reverse_proxy.status_text", res.Status)

So you can see that we’re setting these to the field name of the header, which is the canonical form, so your placeholders (and by proxy, copy_headers because it’s just a shortcut for setting up these placeholders) needs to use the canonical form as well.

In some other places like the header directive, it’ll let you use whatever, because we can canonicalize the inputs before using them. But the replacer is just some string replacement magic.

5

Thanks, @francislavoie

1 Like
6

This topic was automatically closed after 30 days. New replies are no longer allowed.