Reverse_proxy and file_server

Help
1

1. The problem I’m having:

I am trying to simultaneously use a file_server and reverse_proxy to a service that returns the dynamic content of index html.

If proxying is turned off, absolutely all static files are processed properly by Caddy.
But if I enable proxying, the file server does not return anything.

I’ve added a config below that basically does the trick, but if there’s a greater depth of static files, everything breaks.

Therefore, the question is simple:
Is there a configuration pattern that will process all static files through the file server, and if the index file does not exist, then proxy to another service?

I’ll try to describe file system… Root for Caddy is /pages

/pages/123/styles/style.css
/pages/123/styles/app.css
/pages/123/js/script.js
/pages/123/js/app.js

/pages/456/styles/style.css
/pages/456/styles/app.css
/pages/456/js/script.js
/pages/456/js/app.js

So, when request is /123/styles/style.css or /456/js/app.js then I want to see content of files from file_server.

And only if request is /123 or /456 then I want to see content of dynamic html from reverse_proxy.

2. Error messages and/or full log output:

There isn't useful log

3. Caddy version:

image: caddy:2.7.5-alpine

4. How I installed and ran Caddy:

a. System environment:

➜ docker -v
Docker version 24.0.7, build afdd53b
~ 
➜ docker-compose -v
Docker Compose version v2.22.0
~ 
➜

d. My complete Caddy config:

:81 {
	encode gzip
	handle_path /*/* {
		root * /pages
        file_server
	}
	handle_path /*/*/* {
		root * /pages
		file_server
	}
	handle {
		reverse_proxy http://index-html:3000
	}
}

5. Links to relevant resources:

2

Enable the debug global option as requested in the template. The logs are important.

Make a request with curl -v showing the behaviour you’re experiencing.

3

Sorry.

Ok, full logs from very beggining

caddy-server  | {"level":"info","ts":1700136443.6844506,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":"caddyfile"}
caddy-server  | {"level":"info","ts":1700136443.6855986,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//127.0.0.1:2019","//localhost:2019","//[::1]:2019"]}
caddy-server  | {"level":"info","ts":1700136443.6857667,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc0002a9e00"}
caddy-server  | {"level":"debug","ts":1700136443.6856933,"logger":"http.auto_https","msg":"adjusted config","tls":{"automation":{"policies":[{}]}},"http":{"servers":{"srv0":{"listen":[":81"],"routes":[{"handle":[{"handler":"vars","root":"/pages"},{"encodings":{"gzip":{}},"handler":"encode","prefer":["gzip"]},{"handler":"reverse_proxy","upstreams":[{"dial":"index-html:3000"}]},{"handler":"file_server","hide":["/etc/caddy/Caddyfile"]}]}],"automatic_https":{}}}}}
caddy-server  | {"level":"debug","ts":1700136443.6859963,"logger":"http","msg":"starting server loop","address":"[::]:81","tls":false,"http3":false}
caddy-server  | {"level":"info","ts":1700136443.6860113,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2","h3"]}
caddy-server  | {"level":"info","ts":1700136443.6861398,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
caddy-server  | {"level":"info","ts":1700136443.686145,"msg":"serving initial configuration"}
caddy-server  | {"level":"info","ts":1700136443.6863124,"logger":"tls","msg":"cleaning storage unit","description":"FileStorage:/data/caddy"}
caddy-server  | {"level":"info","ts":1700136443.6863563,"logger":"tls","msg":"finished cleaning storage units"}
caddy-server  | {"level":"debug","ts":1700136447.920842,"logger":"http.handlers.reverse_proxy","msg":"selected upstream","dial":"index-html:3000","total_upstreams":1}
caddy-server  | {"level":"debug","ts":1700136448.0480502,"logger":"http.handlers.reverse_proxy","msg":"upstream roundtrip","upstream":"index-html:3000","duration":0.127095203,"request":{"remote_ip":"172.30.0.2","remote_port":"34870","client_ip":"172.30.0.2","proto":"HTTP/1.1","method":"GET","host":"pages.localhost","uri":"/01hf6n06t46v12b8e8jzj1xkyx/app.js","headers":{"X-Forwarded-Server":["204714e43edf"],"Cookie":[],"Accept-Encoding":["gzip, deflate, br"],"Cache-Control":["no-cache"],"X-Forwarded-Host":["pages.localhost"],"Accept-Language":["en-US,en;q=0.9"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7"],"Sec-Fetch-Site":["none"],"Upgrade-Insecure-Requests":["1"],"X-Forwarded-Port":["80"],"X-Forwarded-Proto":["http"],"Pragma":["no-cache"],"Sec-Ch-Ua":["\"Google Chrome\";v=\"119\", \"Chromium\";v=\"119\", \"Not?A_Brand\";v=\"24\""],"Sec-Fetch-Dest":["document"],"X-Forwarded-For":["172.30.0.2"],"X-Real-Ip":["172.30.0.1"],"Sec-Fetch-User":["?1"],"User-Agent":["Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36"],"Sec-Ch-Ua-Platform":["\"Linux\""],"Sec-Ch-Ua-Mobile":["?0"],"Sec-Fetch-Mode":["navigate"]}},"headers":{"Date":["Thu, 16 Nov 2023 12:07:28 GMT"],"Connection":["keep-alive"],"Keep-Alive":["timeout=5"],"Content-Type":["text/plain;charset=UTF-8"],"Content-Length":["13"]},"status":404}

request is /01hf6n06t46v12b8e8jzj1xkyx/app.js

This log according to the next Caddyfile

{
	debug
}

:81 {
	encode gzip
	root * /pages
	file_server
	reverse_proxy http://index-html:3000
}

When I disable proxy, next log

caddy-server  | {"level":"info","ts":1700136623.871673,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":"caddyfile"}
caddy-server  | {"level":"info","ts":1700136623.878083,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
caddy-server  | {"level":"debug","ts":1700136623.8784502,"logger":"http.auto_https","msg":"adjusted config","tls":{"automation":{"policies":[{}]}},"http":{"servers":{"srv0":{"listen":[":81"],"routes":[{"handle":[{"handler":"vars","root":"/pages"},{"encodings":{"gzip":{}},"handler":"encode","prefer":["gzip"]},{"handler":"file_server","hide":["/etc/caddy/Caddyfile"]}]}],"automatic_https":{}}}}}
caddy-server  | {"level":"info","ts":1700136623.8785992,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc0004a8080"}
caddy-server  | {"level":"debug","ts":1700136623.8788176,"logger":"http","msg":"starting server loop","address":"[::]:81","tls":false,"http3":false}
caddy-server  | {"level":"info","ts":1700136623.8788414,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2","h3"]}
caddy-server  | {"level":"info","ts":1700136623.8790474,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
caddy-server  | {"level":"info","ts":1700136623.879057,"msg":"serving initial configuration"}
caddy-server  | {"level":"info","ts":1700136623.8791041,"logger":"tls","msg":"cleaning storage unit","description":"FileStorage:/data/caddy"}
caddy-server  | {"level":"info","ts":1700136623.8791456,"logger":"tls","msg":"finished cleaning storage units"}
caddy-server  | {"level":"debug","ts":1700136626.0742764,"logger":"http.handlers.file_server","msg":"sanitized path join","site_root":"/pages","request_path":"/01hf6n06t46v12b8e8jzj1xkyx/app.js","result":"/pages/01hf6n06t46v12b8e8jzj1xkyx/app.js"}
caddy-server  | {"level":"debug","ts":1700136626.0752954,"logger":"http.handlers.file_server","msg":"opening file","filename":"/pages/01hf6n06t46v12b8e8jzj1xkyx/app.js"}
4

And that was with this config?

Because I can’t replicate the problem with that config, it should definitely route through handle_path /*/*

You didn’t disable the proxy in that config though. What you say doesn’t match with what you show.

5

This was config which makes me happy, but it looks weird.
Forget it.

Please, sorry, I’m a little bit confused with this situation.
Let’s start all over again.

I want to use simultaneously file_server and reverse_proxy.

So I write Caddyfile:

{
	debug
}

:81 {
	encode gzip
	root * /pages
	file_server
	reverse_proxy http://index-html:3000
}

And make request

➜ curl -v http://pages.localhost/01hf6n06t46v12b8e8jzj1xkyx/app.js
*   Trying ::1:80...
* Connected to pages.localhost (::1) port 80 (#0)
> GET /01hf6n06t46v12b8e8jzj1xkyx/app.js HTTP/1.1
> Host: pages.localhost
> User-Agent: curl/7.81.0
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 404 Not Found
< Content-Length: 13
< Content-Type: text/plain;charset=UTF-8
< Date: Thu, 16 Nov 2023 12:19:00 GMT
< Server: Caddy
< 
* Connection #0 to host pages.localhost left intact
404 Not Found

Caddy logs are:

caddy-server  | {"level":"info","ts":1700136443.6844506,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":"caddyfile"}
caddy-server  | {"level":"info","ts":1700136443.6855986,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//127.0.0.1:2019","//localhost:2019","//[::1]:2019"]}
caddy-server  | {"level":"info","ts":1700136443.6857667,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc0002a9e00"}
caddy-server  | {"level":"debug","ts":1700136443.6856933,"logger":"http.auto_https","msg":"adjusted config","tls":{"automation":{"policies":[{}]}},"http":{"servers":{"srv0":{"listen":[":81"],"routes":[{"handle":[{"handler":"vars","root":"/pages"},{"encodings":{"gzip":{}},"handler":"encode","prefer":["gzip"]},{"handler":"reverse_proxy","upstreams":[{"dial":"index-html:3000"}]},{"handler":"file_server","hide":["/etc/caddy/Caddyfile"]}]}],"automatic_https":{}}}}}
caddy-server  | {"level":"debug","ts":1700136443.6859963,"logger":"http","msg":"starting server loop","address":"[::]:81","tls":false,"http3":false}
caddy-server  | {"level":"info","ts":1700136443.6860113,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2","h3"]}
caddy-server  | {"level":"info","ts":1700136443.6861398,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
caddy-server  | {"level":"info","ts":1700136443.686145,"msg":"serving initial configuration"}
caddy-server  | {"level":"info","ts":1700136443.6863124,"logger":"tls","msg":"cleaning storage unit","description":"FileStorage:/data/caddy"}
caddy-server  | {"level":"info","ts":1700136443.6863563,"logger":"tls","msg":"finished cleaning storage units"}
caddy-server  | {"level":"debug","ts":1700136447.920842,"logger":"http.handlers.reverse_proxy","msg":"selected upstream","dial":"index-html:3000","total_upstreams":1}
caddy-server  | {"level":"debug","ts":1700136448.0480502,"logger":"http.handlers.reverse_proxy","msg":"upstream roundtrip","upstream":"index-html:3000","duration":0.127095203,"request":{"remote_ip":"172.30.0.2","remote_port":"34870","client_ip":"172.30.0.2","proto":"HTTP/1.1","method":"GET","host":"pages.localhost","uri":"/01hf6n06t46v12b8e8jzj1xkyx/app.js","headers":{"X-Forwarded-Server":["204714e43edf"],"Cookie":[],"Accept-Encoding":["gzip, deflate, br"],"Cache-Control":["no-cache"],"X-Forwarded-Host":["pages.localhost"],"Accept-Language":["en-US,en;q=0.9"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7"],"Sec-Fetch-Site":["none"],"Upgrade-Insecure-Requests":["1"],"X-Forwarded-Port":["80"],"X-Forwarded-Proto":["http"],"Pragma":["no-cache"],"Sec-Ch-Ua":["\"Google Chrome\";v=\"119\", \"Chromium\";v=\"119\", \"Not?A_Brand\";v=\"24\""],"Sec-Fetch-Dest":["document"],"X-Forwarded-For":["172.30.0.2"],"X-Real-Ip":["172.30.0.1"],"Sec-Fetch-User":["?1"],"User-Agent":["Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36"],"Sec-Ch-Ua-Platform":["\"Linux\""],"Sec-Ch-Ua-Mobile":["?0"],"Sec-Fetch-Mode":["navigate"]}},"headers":{"Date":["Thu, 16 Nov 2023 12:07:28 GMT"],"Connection":["keep-alive"],"Keep-Alive":["timeout=5"],"Content-Type":["text/plain;charset=UTF-8"],"Content-Length":["13"]},"status":404}

Then I try to disable reverse proxy, so new Caddyfile is:

{
	debug
}

:81 {
	encode gzip
	root * /pages
	file_server
}

I make request:

➜ curl -v http://pages.localhost/01hf6n06t46v12b8e8jzj1xkyx/app.js
*   Trying ::1:80...
* Connected to pages.localhost (::1) port 80 (#0)
> GET /01hf6n06t46v12b8e8jzj1xkyx/app.js HTTP/1.1
> Host: pages.localhost
> User-Agent: curl/7.81.0
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Content-Type: text/javascript; charset=utf-8
< Date: Thu, 16 Nov 2023 13:00:18 GMT
< Etag: "s46ill1en"
< Last-Modified: Wed, 15 Nov 2023 19:10:33 GMT
< Server: Caddy
< Vary: Accept-Encoding
< Transfer-Encoding: chunked
< 
...content of file...
* Connection #0 to host pages.localhost left intact

And logs for Caddyfile without proxy are:

caddy-server  | {"level":"info","ts":1700136623.871673,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":"caddyfile"}
caddy-server  | {"level":"info","ts":1700136623.878083,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
caddy-server  | {"level":"debug","ts":1700136623.8784502,"logger":"http.auto_https","msg":"adjusted config","tls":{"automation":{"policies":[{}]}},"http":{"servers":{"srv0":{"listen":[":81"],"routes":[{"handle":[{"handler":"vars","root":"/pages"},{"encodings":{"gzip":{}},"handler":"encode","prefer":["gzip"]},{"handler":"file_server","hide":["/etc/caddy/Caddyfile"]}]}],"automatic_https":{}}}}}
caddy-server  | {"level":"info","ts":1700136623.8785992,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc0004a8080"}
caddy-server  | {"level":"debug","ts":1700136623.8788176,"logger":"http","msg":"starting server loop","address":"[::]:81","tls":false,"http3":false}
caddy-server  | {"level":"info","ts":1700136623.8788414,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2","h3"]}
caddy-server  | {"level":"info","ts":1700136623.8790474,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
caddy-server  | {"level":"info","ts":1700136623.879057,"msg":"serving initial configuration"}
caddy-server  | {"level":"info","ts":1700136623.8791041,"logger":"tls","msg":"cleaning storage unit","description":"FileStorage:/data/caddy"}
caddy-server  | {"level":"info","ts":1700136623.8791456,"logger":"tls","msg":"finished cleaning storage units"}
caddy-server  | {"level":"debug","ts":1700136626.0742764,"logger":"http.handlers.file_server","msg":"sanitized path join","site_root":"/pages","request_path":"/01hf6n06t46v12b8e8jzj1xkyx/app.js","result":"/pages/01hf6n06t46v12b8e8jzj1xkyx/app.js"}
caddy-server  | {"level":"debug","ts":1700136626.0752954,"logger":"http.handlers.file_server","msg":"opening file","filename":"/pages/01hf6n06t46v12b8e8jzj1xkyx/app.js"}
6

Okay. So which request paths exactly do you want to go to your proxy? You showed examples for the fileserver, but that’s not a full picture.

Basically you need to set up matchers to split the traffic, to tell Caddy which requests go where.

Alternatively what you could do is the file matcher to test if the requested file exists on disk, and if not send it to the proxy.

:81 {
	root * /pages

	encode gzip

	@exists file
	handle @exists {
		file_server
	}

	handle {
		reverse_proxy index-html:3000
	}
}
2 Likes
7

Oh yeah! You are my savior!
This is exactly what I wanted, but I didn’t know how to set it up in the configuration.
This matcher was not completely clear to me, now I seem to understand everything.

Thank you very much for your help and sorry for taking so long to explain.

1 Like
8

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.