How to serve static files with security rules implemented in php?

[JeremyPasco]

Hi,
We are developing a web app based on api-platform distribution, which includes Caddy
Until now, we’ve been serving huge static files (ie: PDF) directly from PHP scripts because it allowed us to limit access based on business logic implemented in PHP (ACL, voters, roles…).
Now, we encounter memory issues when serving huge files from PHP and we’d like to serve them directly from Caddy as static files. But we don’t know how to do it while respecting our security rules.
Is there a way to let PHP inform Caddy when to serve or not these files?

[francislavoie]

Hi! Glad to see you follow up from the chat at API Platform Conf.

So the way this should work is you would have your server respond with an empty response body, but with an X-Accel-Redirect header (or whatever header name you prefer, but this is the one Nginx came up with for this) and then use handle_response to have Caddy serve that file from disk instead using a rewrite + file_server. It would look something like this:

php_fastcgi unix//run/php/php8.0-fpm.sock {
	@accel header X-Accel-Redirect *
	handle_response @accel {
		root    * /path/to/private/files
		rewrite   {http.reverse_proxy.header.X-Accel-Redirect}
		file_server
	}
}

There’s an example for this in the docs for reverse_proxy. The php_fastcgi directive can take all the same options as reverse_proxy in addition to its own options.

reverse_proxy (Caddyfile directive) - Caddy Documentation

BUT, there’s currently a bug that would prevent this from working, that I plan to work on very soon. The php_fastcgi directive doesn’t correctly parse handle_response options at the moment.

Unable to set root inside 'handle_response' block which is in turn inside 'php_fastcgi' directive

As an aside, I’d like to point out that PHP has a function readfile() which is specifically designed to respond with large files, without loading them entirely into memory. Have you tried using that function?

https://www.php.net/manual/en/function.readfile.php

[JeremyPasco]

Thank for the reply!
We are indeed exploring other PHP approach like readfile(). As we are expecting huge download hits the next months, we are benchmarking different options. And we thought that putting PHP out of the process may help to increase performance.

So, just to be sure I got this correctly:

1. A user request some endpoint /big_files/17
2. PHP script does its security part and returns a forbidden HTTP code, or allows it with a X-Accel-Redirect header with path to the static file to serve
3. Caddy intercepts the response, and rewrite it to include the static file

One last question: if a Varnish cache is in place, what would/could be cached if a user requests /big_files/17?

• the PHP response (with the X-Accel-Redirect header), then Caddy would get that response from cache
• the final response (after rewrite) which includes the static file, so the user would not trigger any rewrite

[francislavoie]

@JeremyPasco

So, just to be sure I got this correctly:

1. A user request some endpoint /big_files/17
2. PHP script does its security part and returns a forbidden HTTP code, or allows it with a X-Accel-Redirect header with path to the static file to serve
3. Caddy intercepts the response, and rewrite it to include the static file

Yep, that’s right

@JeremyPasco

One last question: if a Varnish cache is in place, what would/could be cached if a user requests /big_files/17?

• the PHP response (with the X-Accel-Redirect header), then Caddy would get that response from cache
• the final response (after rewrite) which includes the static file, so the user would not trigger any rewrite

If the request requires authentication, it’s probably not a good idea to involve caching, because it would also cache the authentication result. But I guess that might depend on how you’ve configured Varnish. I’m not sure I can answer that cleanly, because I don’t use Varnish.