HTTP proxy doesn't work for HTTPS server config

1. The problem I’m having:

I’m configuring HTTP proxy as an upstream behind Caddy as reverse proxy
Here is working Caddyfile:


{
	auto_https disable_redirects
	debug
}

:8000 {
	log {
		output stdout
		format json
	}
	reverse_proxy http://1.2.3.4:8000 {
		header_up Proxy-Authorization "Basic SOMECREDENTIALS"
	}
}

Here are the curl for the good case:

curl -v -x http://example.com:8000 http://ifcfg.me

GET http://ifcfg.me/ HTTP/1.1

Host: ifcfg.me

User-Agent: curl/8.7.1

Accept: /

Proxy-Connection: Keep-Alive

  • Request completely sent off

< HTTP/1.1 200 OK

< Content-Length: 13

< Content-Type: text/plain; charset=utf-8

< Date: Mon, 07 Oct 2024 15:26:53 GMT

< Server: Caddy

<

And the Caddy logs:

{“level”:“debug”,“ts”:1728314812.9717736,“logger”:“http.handlers.reverse_proxy”,“msg”:“selected upstream”,“dial”:“1.2.3.4:8000”,“total_upstreams”:1}

{“level”:“debug”,“ts”:1728314813.1839786,“logger”:“http.handlers.reverse_proxy”,“msg”:“upstream roundtrip”,“upstream”:“1.2.3.4:8000”,“duration”:0.212123151,“request”:{“remote_ip”:“1.1.1.1”,“remote_port”:“63054”,“client_ip”:“1.1.1.1”,“proto”:“HTTP/1.1”,“method”:“GET”,“host”:“ifcfg.me”,“uri”:“http://ifcfg.me/“,“headers”:{“Proxy-Authorization”:[“REDACTED”],“User-Agent”:[“curl/8.7.1”],“Accept”:[”*/*"],“X-Forwarded-For”:[“1.1.1.1”],“X-Forwarded-Proto”:[“http”],“X-Forwarded-Host”:[“ifcfg.me”]}},“headers”:{“Date”:["Mon, 07 Oct 2024 15:26:53 GMT”],“Content-Length”:[“13”],“Content-Type”:[“text/plain; charset=utf-8”]},“status”:200}

{“level”:“info”,“ts”:1728314813.1841192,“logger”:“http.log.access.log0”,“msg”:“handled request”,“request”:{“remote_ip”:“1.1.1.1”,“remote_port”:“63054”,“client_ip”:“1.1.1.1”,“proto”:“HTTP/1.1”,“method”:“GET”,“host”:“ifcfg.me”,“uri”:“http://ifcfg.me/“,“headers”:{“Accept”:[”*/*“],“Proxy-Connection”:[“Keep-Alive”],“User-Agent”:[“curl/8.7.1”]}},“bytes_read”:0,“user_id”:”",“duration”:0.212374046,“size”:13,“status”:200,“resp_headers”:{“Server”:[“Caddy”],“Date”:["Mon, 07 Oct 2024 15:26:53 GMT”],“Content-Length”:[“13”],“Content-Type”:[“text/plain; charset=utf-8”]}}

So we can clearly see that request is proxied, reaches the proxy server(upstream) and we get the response.

Now I just need to change this setup to work via HTTPS, so I change configuration to this:


{
	auto_https disable_redirects
	debug
}

example.com {
	log {
		output stdout
		format json
	}
	reverse_proxy http://1.2.3.4:8000 {
		header_up Proxy-Authorization "Basic SOMECREDENTIALS"
	}
}

Bad curl:

curl -v -x https://example.com http://ifcfg.me

* Host example.com:443 was resolved.

* IPv6: (none)

* IPv4: 3.3.3.3

* Trying 3.3.3.3:443...

* Connected to example.com (3.3.3.3) port 443

* (304) (OUT), TLS handshake, Client hello (1):

* CAfile: /etc/ssl/cert.pem

* CApath: none

* (304) (IN), TLS handshake, Server hello (2):

* (304) (IN), TLS handshake, Unknown (8):

* (304) (IN), TLS handshake, Certificate (11):

* (304) (IN), TLS handshake, CERT verify (15):

* (304) (IN), TLS handshake, Finished (20):

* (304) (OUT), TLS handshake, Finished (20):

* SSL connection using TLSv1.3 / AEAD-AES128-GCM-SHA256 / [blank] / UNDEF

* Proxy certificate:

* subject: CN=example.com

* start date: Oct 7 06:44:32 2024 GMT

* expire date: Jan 5 06:44:31 2025 GMT

* subjectAltName: host "example.com" matched cert's "example.com"

* issuer: C=US; O=Let's Encrypt; CN=E6

* SSL certificate verify ok.

> GET http://ifcfg.me/ HTTP/1.1

> Host: ifcfg.me

> User-Agent: curl/8.7.1

> Accept: */*

> Proxy-Connection: Keep-Alive

>

* Request completely sent off

< HTTP/1.1 200 OK

< Alt-Svc: h3=":443"; ma=2592000

< Server: Caddy

< Date: Mon, 07 Oct 2024 15:24:13 GMT

< Content-Length: 0

<

* Connection #0 to host example.com left intact

The only log I can see:

{“level”:“info”,“ts”:1728314653.108712,“logger”:“http.log.access”,“msg”:“NOP”,“request”:{“remote_ip”:“1.1.1.1”,“remote_port”:“63014”,“client_ip”:“1.1.1.1”,“proto”:“HTTP/1.1”,“method”:“GET”,“host”:“ifcfg.me”,“uri”:“http://ifcfg.me/“,“headers”:{“User-Agent”:[“curl/8.7.1”],“Accept”:[”*/*“],“Proxy-Connection”:[“Keep-Alive”]},“tls”:{“resumed”:false,“version”:772,“cipher_suite”:4865,“proto”:”“,“server_name”:“example.com”}},“bytes_read”:0,“user_id”:”“,“duration”:0.000004624,“size”:0,“status”:0,“resp_headers”:{“Server”:[“Caddy”],“Alt-Svc”:[“h3=\”:443\”; ma=2592000”]}}

2. Error messages and/or full log output:

No error messages, request is just not proxied, but handled by Caddy itself

3. Caddy version:

v2.8.4 h1:q3pe0wpBj1OcHFZ3n/1nl4V4bxBrYoSoab7rL9BMYNk=

4. How I installed and ran Caddy:

apt package manager + systemd

a. System environment:

Ubuntu 224.04.1 LTS (Noble Numbat)

b. Command:

Default systemd unit

PASTE OVER THIS, BETWEEN THE ``` LINES.
Please use the preview pane to ensure it looks nice.

c. Service/unit/compose file:

PASTE OVER THIS, BETWEEN THE ``` LINES.
Please use the preview pane to ensure it looks nice.

d. My complete Caddy config:

PASTE OVER THIS, BETWEEN THE ``` LINES.
Please use the preview pane to ensure it looks nice.

5. Links to relevant resources:

Your request has this host, but Caddy is only configured to match example.com, so your logs show a “NOP” (no-operation). You just made a bad request. Not sure what you’re trying to do with that.

1 Like

Seems like you’re trying to implement a forward proxy, not a reverse proxy. See this project for that GitHub - caddyserver/forwardproxy: Forward proxy plugin for the Caddy web server

1 Like

Hello! Thank you for looking into this issue! From the documentation I assumed that such syntax sugar means that Caddy actually serves “https://domainname.com” also, since https is a scheme. Do I miss something?
And do you mean that reverse_proxy logic will not work for me, even PoC tests on :port were ok?

1 Like

What do you mean by “such syntax sugar”? Which documentation page? Please be specific.

Do you mean this?

You wrote example.com here, not domainname.com.

Please be consistent, I’m not sure what you’re talking about.

Well, you haven’t been clear at all about what you’re trying to do. Are you sure you understand what a reverse proxy is, and how it differs from a forward proxy? You’re using the curl -x option which has to do with connecting to a forward proxy.

1 Like

Based on Caddyfile Concepts — Caddy Documentation

Address Effect
example.com HTTPS with managed publicly-trusted certificate

So I assume when we configure example.com - this server section also matches and handles https://example.com ( and perhaps https://example.com:443 also, that’s why it’s kinda a syntax sugar for a user)

Frankly speaking I think I haven’t used forward proxy at all, so tried to apply more familiar to me reverse proxy approach.
My use case is the following:

  1. A client connects to a server
  2. server proxies/balances request to a HTTP proxy (I need single entry point/balancer in order to have multiple proxies)
  3. HTTP proxy sends this request further

And it seems to be working when request arrives at proper server section.
I tried to specify schema also, as you suggested:


https://example.com {
	log {
		output stdout
		format json
	}
	reverse_proxy http://1.2.3.4:8000 {
		header_up Proxy-Authorization "Basic SOMECREDENTIALS"
	}
}

But initial curl request still triggers NOP log record.

Right but that’s not what I was talking about earlier. You used -x https://example.com which has has curl try to use the server at example.com as a proxy, but sends the Host header with ifcfg.me. Caddy uses the Host header for routing, and you don’t have a route for ifcfg.me, so it writes an empty response.

Okay but when you say “proxy” here, you’re talking about a forward proxy. Not a reverse proxy. Not the same.

I think you need to go do some reading then come back.

1 Like

Thanks for your time @francislavoie it was really educative topic :slight_smile:
Your pain is very close to me, I know how you feel when a user can’t explain the problem clearly and appreciate your efforts on this wonderful project .

I spent some time on this, and besides of Host header issue learnt abt HTTP Connect method used for proxies. Such a pity that Caddy doesn’t support it. Unfortunately forwardproxy also was not able to cover my needs (as it doesn’t allow https->http chaining).
So after a lot of suffering I had to use Envoy (as it supports CONNECT method out of the box). Absolutely hate it after Caddy :sob:

Once again, if you need CONNECT, you need a forward proxy, i.e. GitHub - caddyserver/forwardproxy: Forward proxy plugin for the Caddy web server

What do you mean? In what way does it not support that?

1 Like

It’s about this issue insecure schemes are only allowed to localhost upstreams · Issue #116 · caddyserver/forwardproxy · GitHub

For future community members facing a similar use case, I would recommend taking a look at the Glider project. It’s like Caddy but for proxies (in terms of simplicity), and my proxy chaining task can be solved there in just 1 line. Both Caddy and Envoy don’t look like good choices for this particular case to me at the moment.