Caddy v2 header_upstream

1. My Caddy version (caddy -version):

❯ git log | head
commit dccba71276796999f3e16a6560016dc5dd0a952a
Author: Matthew Holt <mholt@users.noreply.github.com>
Date:   Tue Oct 29 16:02:40 2019 -0600

    reverse_proxy: Structured logs

2. How I run Caddy:

./caddy
curl -v -X POST "http://localhost:2019/load" -H "Content-Type: application/json" -d @caddy.json

My caddy.json:-

{
    "apps": {
	"http": {
	    "servers": {
		"example": {
                    "automatic_https": {
                        "disable": false
                    },
		    "listen": [":443"],
		    "routes": [
			{
                            "match": [
                                {
                                    "host": ["sub.myhost.com"]
                                }
                            ],
			    "handle": [{
				"handler": "reverse_proxy",
				"upstreams": [
                                    {
                                        "dial": "localhost:8000"
                                    }
				]
			    }]
			}
		    ]
		}
	    }
	}
    }
}

a. System environment:

cat /etc/lsb-release 
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04.6 LTS"

3. The problem I’m having:

I need to set equivalent of header_upstream in caddy v1 when doing reverse proxy.

4. Error messages and/or full log output:

curl -X POST "http://localhost:2019/config/apps/http/servers/example/routes/0/handle/0/headers/" -H "Content-Type: application/json" -d'{"request": {"set": {"Host": "dev.myhost.com"}}}'
running: loading app module 'http': provision http: server example: setting up server routes: loading handler module in position 0: loading module 'reverse_proxy': decoding module config: http.handlers.reverse_proxy: json: cannot unmarshal string into Go struct field HeaderOps.headers.request.set of type []string

This is my current config:-

curl "http://localhost:2019/config/apps/http/servers/"
{"example":{"automatic_https":{"disable":false},"listen":[":443"],"routes":[{"handle":[{"handler":"reverse_proxy","upstreams":[{"dial":"localhost:8000"}]}],"match":[{"host":["sub.myhost.com"]}]}]}}

I try both with POST and PUT but got same error.

Aah, the error really said it. It need an array while I’m passing a string, so this was accepted:-

curl -v -X POST "http://localhost:2019/config/apps/http/servers/example/routes/0/handle/0/headers/" -H "Content-Type: application/json" -d'{"request": {"set": {"Host": ["sub.myhost.com"]}}}'

But still I’m not getting the expected header from my upstream.

Thanks for the report and for trying Caddy 2 while it’s in beta!

Sorry if I’m just not seeing something, but in your caddy.json, where are you setting the headers to upstream?

Oh, sorry. I’m not setting it in caddy.json actually. So the idea is to set the Host header for upstream dynamically so that I can easily point it to different app without changing the whole upstream definition.

Do you need a different value per-request? Something like a value of "{http.request.host}", to pass the hostname of the incoming request through to the upstream one?

No. So the use case actually to build something similar to ngrok. Let say I point *.myngrok.com to this caddy server. I want to allow my user to have domain like sub1.myngrok.com being forwardded to their local server. Their local server might be expecting different Host header (not sub1.myngrok.com) so they should be able to set that when adding new forwarding. The user then would do reverse ssh port forwarding from the server to their local, such as:-

ssh -R 8000:localhost:8000 myngrok.com 'curl http://localhost:2019/config/apps/http/servers/....'

To add new forwardding rule that will forward to their local server. Currently I’m using caddy1 but limited to just single subdomain as adding new config to Caddyfile is non-trivial. But after looking at caddy2 dynamic configuration I thought maybe it possible to this easily now.

Here’s more background on what I want to do - https://dev.to/k4ml/poor-man-ngrok-with-tcp-proxy-and-ssh-reverse-tunnel-1fm.

Ah, so you have the bare config that you posted above, and then you want to update parts of it using the API.

Now your previous post (and the initial post) makes sense! Thanks for clarifying, sorry, I’m tired today. :slight_smile: You say this command was accepted:

curl -v -X POST "http://localhost:2019/config/apps/http/servers/example/routes/0/handle/0/headers/" -H "Content-Type: application/json" -d'{"request": {"set": {"Host": ["sub.myhost.com"]}}}'

Okay. Right. So, to answer the first question: headers are a map of string to []string, because you can have header fields that are repeated, i.e. with multiple values. So your POST above looks correct.

And by doing a GET after that, can you confirm that this is your current, running config?

{
	"apps": {
		"http": {
			"servers": {
				"example": {
					"automatic_https": {
						"disable": false
					},
					"listen": [
						":443"
					],
					"routes": [
						{
							"handle": [
								{
									"handler": "reverse_proxy",
									"headers": {
										"request": {
											"set": {
												"Host": [
													"sub.myhost.com"
												]
											}
										}
									},
									"upstreams": [
										{
											"dial": "localhost:8000"
										}
									]
								}
							],
							"match": [
								{
									"host": [
										"sub.myhost.com"
									]
								}
							]
						}
					]
				}
			}
		}
	}
}

And with that, you are not seeing Host set properly on the upstream, is that right? What is it set to?

Oh dear. I’m so sorry. The header was actually set correctly and passed to upstream. I should have checked that first instead of just basing on the result that I’m not getting the website I was expecting to see.

So it’s totally my fault. The Host to set was wrong, not the one I’m expecting on upstream. Again, I’m sorry for wasting your time. I’ll try to make it up by doing some write up (or fix the docs to be more clear) on this feature.

1 Like

No worries, thank you! I’m super relieved actually.