V2: HTTP proxy in place of fastcgi in Wordpress

Help
1

1. Caddy version (caddy version):

2.3 rc2

2. How I run Caddy:

As a reverse proxy-cum-cache in front of another server running Nginx.

a. System environment:

Freshly installed by downloading the deb file on an Ubuntu 20.04 server

b. Command:

service caddy start

c. Service/unit/compose file:

It’s an official .deb file

d. My complete Caddyfile or JSON config:

ultra.news:443 {
tls /etc/letsencrypt/live/exam/fullchain.pem /etc/letsencrypt/live/exam/privkey.pem
root * /var/www/wordpress/

file_server
reverse_proxy 149.18.14.1:80
encode gzip

try_files {path} {path}/

header /menu/ test blah
header (.png|.js)$ Cache-Control “public, max-age=11116000”

log { output file /var/log/caddy/uaccess.log} }

3. The problem I’m having:

DESIRED OUTCOME:

A request comes.
Caddy checks if the corresponding file exists on the disk.
If yes, serves that file.
If not, passes the request to the origin server, fetches the response and passes it on.

This has to work for both static files as well as cached versions of dynamic pages if they exist. The request reach the origin server only if there is no cached version of that Wordpress post/page. If it’s a static asset, such as a .css file, or if a cached version of the html page exists, it should be delivered by the Caddy server.

Right now, I can either get Caddy to work as a file_server or as a reverse proxy. I can’t combine the two functionalities into a hybrid server – where it acts as a file_server if the file exists, and as a reverse proxy if it doesn’t.

If I define both, all requests are passed on to the origin server even if the file/page exists on the proxy server.

The problem seems to be that both file_server and reverse_proxy are terminating directives.

4. Error messages and/or full log output:

I can get either reverse_proxy to handle the request or file_server to handle the request, using route and handle directives. But if it’s file_server that handles it, and it doesn’t find the resource, it does not pass it on to the reverse_proxy directive, but issues a 404. Similarly, if it’s the reverse_proxy directive that handles it, it simply passes everything to the origin, irrespective of whether the resource exists locally or not.

5. What I already tried:

I tried various combinations of route and handle methods.

I believe the right handler to use here would be try_files. But it doesn’t seem to be working as I’m not sure how to use the so-called ‘named location’ [@location] format in Caddy.

In NGINX, I simply pass it to a named location after running it through the try_files directive like this

try_files $uri $uri/ /wp-content/cache/supercache/axam.com$request_uri/index-https.html @core;

location @core { proxy_pass http://http_backend; }

6. Links to relevant resources:

2

will it work if one replaces the “transport fastcgi” bit with transport http or something like that in the following code?

route {
	# Add trailing slash for directory requests
	@canonicalPath {
		file {
			try_files {path}/index.php
		}
		not path */
	}
	redir @canonicalPath {path}/ 308

	# If the requested file does not exist, try index files
	@indexFiles {
		file {
			try_files {path} {path}/index.php index.php
			split_path .php
		}
	}
	rewrite @indexFiles {http.matchers.file.relative}

	# Proxy PHP files to the FastCGI responder
	@phpFiles {
		path *.php
	}
	reverse_proxy @phpFiles <php-fpm_gateway> {
		transport fastcgi {
			split .php
		}
	}
}
3

replacing the above with http does indeed seem to work, except that I don’t know how to pass on the uri to the upstream in the proper format. I guess it has to do with the .php insertion meant for php-fpm processing?

For now, requests for the home page and any page ending in .php seem to work fine. But requests for other pages are being sent in a malformed way to the upstream server, resulting in HTTP 400 error from Nginx (which is helpfully forwarded to the end user by caddy)

PS: If I enable tls in the http transport block, it changes from 400 to 502\

{
    experimental_http3
#    order file_server after reverse_proxy
}


lbal.domain:443 {
#       tls off
tls /etc/letsencrypt/live/fffff/fullchain.pem /etc/letsencrypt/live/fffff/privkey.pem
root * /var/www/wordpress/
#       bind 139.162.31.167


route {
        # Add trailing slash for directory requests
        @canonicalPath {
                file {
                        try_files {path}/index.php
                }
                not path */
        }
        redir @canonicalPath {path}/ 308

        # If the requested file does not exist, try index files
        @indexFiles {
                file {
                        try_files {path} {path}/index.php index.php
                        split_path .php
                }
        }
        rewrite @indexFiles {http.matchers.file.relative}

        # Proxy PHP files to the FastCGI responder
        @phpFiles {
                path *.php
        }
        reverse_proxy @phpFiles 19.10.14.166 {
                transport http {

                        }
        }
}


file_server


#file_server
encode gzip

#try_files {path} {path}/index.php index.php

header /menu/ test blah
header (.png|.js)$ Cache-Control "public, max-age=11116000"

log {
        output file /var/log/caddy/uaccess.log
   }

}
4

Huh? We don’t have an RC2 release. Which version are you running exactly? Run the caddy version command to see.

Also, please use ``` on the lines before and after your config to use code formatting when posting your config and logs to the forums. It’ll preserve whitespace and make it easier to read.

I’m seeing some formatting problems with the Caddyfile you posted. Namely, it’s required to use newlines for blocks like log, you can’t put it on a single line like you did there.

Also, I think you need to read up on matchers; your second header directive doesn’t have a valid matcher. If you want to use regexp, you need to use a named matcher to do so. See the docs:

You can simplify ultra.news:443 to just ultra.news, it will have the same effect.

Yep, that’s correct. You need to use request matchers to tell Caddy when to do either. There’s a file matcher for what you want to do here.

Anyways, from everything I know about wordpress, what you’re trying to do here is pretty strange. Why not just use fastcgi? It should be as simple as this:

root * /var/www/wordpress
encode gzip
php_fastcgi unix//run/php/php-version-fpm.sock
file_server
5

some of it is from the earlier v1 config that I used to have a couple of years ago (such as the header directives).

Anyway, I think what foxed me is that try_files can both be a matcher as well as a directive, and when it’s used in the first sense, it doesn’t involve any rewriting, but when it’s used in the second way, it involves rewriting. At least that’s what I think is going on. I’m not sure.

I cannot use fastcgi as there is a secondary caching layer on the core server. Moreover, sending content from the reverse proxy to the application server without encryption is also not something I would like to do {I plan to upgrade the upstream connection to https after getting everything to work}.

[Of course, you can say why not connect the two using a private network etc, but there are some practical reasons why that’s not possible right now :frowning: ]

Anyway, I’m trying to see if the following works:

                file {
#                       try_files {path} {path}/index.php index.php
                        try_files {path} wp-content/cache/supercache/domain.name{path}/index-https.html
                        split_path .php
                }

I, however, don’t know if, in the above, try_files will simply return true or false or whether it will return the value of the rewritten uri/path.

6

Yep. The try_files directive is a shortcut for a file matcher plus a rewrite. It’s a common pattern, so it was made available as its own directive.

Seems to me like this is probably what you want :thinking: but this assumes there’s no .php files in the filesystem Caddy can see, because then Caddy would try to serve the .php files as-is without executing them… which is a horrible idea for all kinds of security reasons.

root * /var/www/wordpress

@exists file {path}
handle @exists {
	file_server
}

handle {
	reverse_proxy 149.18.14.1:80
}

This makes use of handle’s mutual exclusivity, so only the first matching handle will run (the last one has no matcher so it will always run as a fallback if no other was matched)

Ultimately, you need to decide which paths are safe to serve statically, and whitelist those, if you can’t remove the .php files from the server Caddy’s on. So like:

@exists {
	file {path}
	path /css/* /js/* /images/*
}
7

I can add this “wp-content/cache/supercache/domain.name{path}/index-https.html” after {path} to check if a cached page exists, right? I will also have to add a .br and .gz version later to make use of pre-compressed, cached files. I saw a readymade template someone had posted here.

8

I’m assuming Caddy doesn’t compress on the fly (br and gz)

9

“only the first matching handle will run (the last one has no matcher so it will always run as a fallback if no other was matched)” – This is super useful. I didn’t know this. I did go through the documentation for ‘handle’, but perhaps overlooked this part.

10

Instead of

path /css/ /js/ /images/***

can one do a not *.php and not .ini sort of rule?

11

:man_shrugging:

I don’t know your file structure, so I can’t confirm.

But yes, you can do that, but then you’ll need to add this inside of the handle I gave you to make the rewrite before serving:

rewrite * {http.matchers.file.relative}

So like

@exists file {path} /foo{path}/index.html
handle @exists {
	rewrite * {http.matchers.file.relative}
	file_server
}

That’s what encode gzip does. Caddy doesn’t support br encoding because it’s very slow to compress at runtime, it’s better if it’s pre-compressed. But gzip is fast.

Yeah:

I guess:

@exists {
	file {path}
	not path *.php *.ini
}

But still. This all still gives me the shivers. I would never do something like this tbh. Seems like a bad idea. Way overcomplicates things.

1 Like
12

Actually, the php and ini files are not required to be there on the proxy server at all, and I can put in a policy to not mirror them to it. I just need the static files (css, js, images and html) out front so as to prevent too many requests to the back-end.)

1 Like
13

anyway, let me put these ideas into practice and see… Thanks a lot.

1 Like
14

I can confirm everything works. [Not tried precompression yet.] I have a question on that topic. Suppose I don’t enable Brotli, and leave gzip for on-the-fly conversion.
However, there are times when stuff goes viral, and the same file will be requested repeatedly – could be like 5 times per second. In such cases, does caddy recompress each of those 5 times, or does it store the compressed file in RAM somewhere and re-use it? If it does that, then I can leave it at on-the-fly. Otherwise I might as well as look at precompression.

15

Caddy’s file_server and encode don’t have built-in caching, but you might use the cache plugin to help with that:

But really, gzip is like, really fast, so it’ll be faster with gzip than without, even with no caching.

16

Ok. I was just worried about the hit on the CPU. Because of social media, one has to be ready for spikes, even if it happens only once in three months. Even with precompressed/cached files, there are times when the server barely remained standing, and we’ve had to quickly spin up others to soak up the traffic. However, 99% of the time, it’s all very quiet.

17

This topic was automatically closed after 30 days. New replies are no longer allowed.