Help set up a WordPress bypass with a static cache

I’m trying to replicate this nginx config (source) using Caddy 2 :

server {
    set $cache_uri $request_uri;

    # bypass cache if POST requests or URLs with a query string
    if ($request_method = POST) {
        set $cache_uri 'nullcache';
    }

    if ($query_string != "") {
        set $cache_uri 'nullcache';
    }

    # bypass cache if URLs containing the following strings
    if ($request_uri ~* "(/wp-admin/|/xmlrpc.php|/wp-(app|cron|login|register|mail).php|wp-.*.php|/feed/|index.php|wp-comments-popup.php|wp-links-opml.php|wp-locations.php|sitemap(index)?.xml|[a-z0-9-]+-sitemap([0-9]+)?.xml)") {
        set $cache_uri 'nullcache';
    }

    # bypass cache if the cookies containing the following strings
    if ($http_cookie ~* "comment_author|wordpress_[a-f0-9]+|wp-postpass|wordpress_logged_in") {
        set $cache_uri 'nullcache';
    }

    # custom sub directory e.g. /blog
    set $custom_subdir '';

    # default html file
    set $cache_enabler_uri '${custom_subdir}/wp-content/cache/cache-enabler/${http_host}${cache_uri}index.html';
    location / {
        gzip_static on; # this directive is not required but recommended
        try_files $cache_enabler_uri $uri $uri/ $custom_subdir/index.php?$args;
    }

    ...
}

The idea is to bypass WordPress and PHP entirely and try first to serve the static file created by a cache plugin. If it does not exist, the server should then redirect to PHP and WordPress, so the website is never broken.

So far, that’s easy. The more difficult part is that we need to bypass this behaviour for connected users and for the admin and some more URL.

Here’s what I have come up with so far :

@cache {
    not {
		header_regexp Cookie "comment_author|wordpress_[a-f0-9]+|wp-postpass|wordpress_logged_in"
		path_regexp "(/wp-admin/|/xmlrpc.php|/wp-(app|cron|login|register|mail).php|wp-.*.php|/feed/|index.php|wp-comments-popup.php|wp-links-opml.php|wp-locations.php|sitemap(index)?.xml|[a-z0-9-]+-sitemap([0-9]+)?.xml)"
		method POST
		query *=*
	}
}

route @cache {
    header cache file
    try_files /wp-content/cache/cache-enabler/{host}{uri}/index.html {path} {path}/index.php?{query}
    php_fastcgi unix//run/php/php7.4-fpm-caddy.sock
}
	
header cache wp
php_fastcgi unix//run/php/php7.4-fpm-caddy.sock

It works great for a visitor of the website, the site is really fast when it can only serve a HTML file, and only fast when it can’t, i.e. when the page was not available in the cache.

It does not work right for the exclusions though. Even if I’m connected and so cookies are set and I should not see the cache, I am redirected to the static file if it exists.

I have added a custom “cache” header to monitor what happens. So I know for sure the path exclusion works fine : all wp-admin requests, for instance, have the “wp” value. Every other requests have the “file” value, so it is using the bypass.

Also, how can I handle the two missing exclusions from nginx, POST requests and queries ? What I have tried does not seem to work either.

What did I do wrong ?

Thanks in advance for the help !:+1:

1 Like

I think I made several syntax errors, but I’m not sure which and how to correct the file. :thinking:

Maybe it’s just me, but I think the documentation here could have more examples, one for each standard matcher would be really great : Request matchers (Caddyfile) — Caddy Documentation (I know, easier said then done… :slight_smile: )

I don’t know why method POST wouldn’t work, but for query you could do something like this:

expression {query} != ''

Like so, you mean ?

@cache {
		not {
			header_regexp Cookie "(comment_author|wordpress_[a-f0-9]+|wp-postpass|wordpress_logged_in)"
			path_regexp "(/wp-admin/|/xmlrpc.php|/wp-(app|cron|login|register|mail).php|wp-.*.php|/feed/|index.php|wp-comments-popup.php|wp-links-opml.php|wp-locations.php|sitemap(index)?.xml|[a-z0-9-]+-sitemap([0-9]+)?.xml)"
			method POST
			expression {query} != ''
		}
	}

	route @cache {
		header cache file
		try_files /wp-content/cache/cache-enabler/{host}{uri}/index.html {path} {path}/index.php?{query}
        php_fastcgi unix//run/php/php7.4-fpm-caddy.sock
    }
	
	header cache wp
	php_fastcgi unix//run/php/php7.4-fpm-caddy.sock

It does not change anything, I think, I still see the “file” header when performing a WordPress search (which should be a query, right ?).


And now that I (re-)think about it, the query and POST conditions do not seem really necessary, since the try_files request will take charge of those.

The most important is the cookie restriction, and this one does not work at all for me. If you know why…

Well, I give up for now…

I tried every combination I could think of, but I can’t manage to create a different route based on cookies, queries and method.

The only thing that works is the path, and it’s not really necessary, since the try_files works to redirect to the PHP version every time. The most important one was the cookie, but I even tried to make it really simple and it still did not work :

header Cookie "*wordpress_logged_in*"

So unless someone knows what’s wrong, or if it’s a bug in Caddy 2 (@matt ?), I have no other idea…

Just reading through other threads, I wonder if this actually has the same problem referenced here?

Re: header not working - I was able to prove that it’s functioning as expected. Maybe your browser isn’t actually sending the content you’re expecting?

~/Projects/test
➜ cat Caddyfile
http://:8080 {
  @header {
    header Cookie "*wordpress_logged_in*"
  }
  handle @header {
    respond "Got header" 200
  }
  respond "No header" 400
}

~/Projects/test
➜ caddy2 version
v2.0.0 h1:pQSaIJGFluFvu8KDGDODV8u4/QRED/OPyIR+MWYYse8=

~/Projects/test
➜ curl -iL localhost:8080
HTTP/1.1 400 Bad Request
Server: Caddy
Date: Mon, 11 May 2020 00:17:47 GMT
Content-Length: 9

No header⏎
~/Projects/test
➜ curl -iL localhost:8080 -H "Cookie:wordpress_logged_in"
HTTP/1.1 200 OK
Server: Caddy
Date: Mon, 11 May 2020 00:18:06 GMT
Content-Length: 10

Got header⏎
~/Projects/test
➜ curl -iL localhost:8080 -H "Cookie:foowordpress_logged_inbar"
HTTP/1.1 200 OK
Server: Caddy
Date: Mon, 11 May 2020 00:18:13 GMT
Content-Length: 10

Got header⏎

Yeah I don’t think it’s the same as that bug with merging in not because that was only an issue when you have the same kind of matcher more than once in a not block.

You can know for sure what’s going on by using caddy adapt and inspecting the JSON.

Thanks for your answers and help ! :+1:

@Whitestrake

I’m sure I have the correct cookie when I’m connected :

@matt

Using this full Caddyfile (I tried creating two routes to see if it helps, and reversing the logic to avoid the not directive) :

www.voiretmanger.fr,
nicolasfurno.fr,
www.nicolasfurno.fr,
nicolinux.fr,
www.nicolinux.fr {
	redir https://voiretmanger.fr{uri}/
}

(static) {
	@static {
		file
		path *.ico *.css *.js *.gif *.jpg *.jpeg *.png *.svg *.woff *.json
	}
	header @static Cache-Control max-age=5184000
}


voiretmanger.fr {
	root * /var/www/voiretmanger.fr
	encode zstd gzip
	file_server
	import static
	log {
		output file /var/log/caddy/voiretmanger.fr.access.log
	}

	# Redirect personnels
	redir /a-propos/publicite /soutien
	redir /archives/carte-des-restaurants /a-manger

	header {
		# enable HSTS
		Strict-Transport-Security max-age=31536000;
		# disable clients from sniffing the media type
		X-Content-Type-Options nosniff
		# keep referrer data off of HTTP connections
		Referrer-Policy no-referrer-when-downgrade
	}

	@notcache {
		header_regexp Cookie "(comment_author|wordpress_[a-f0-9]+|wp-postpass|wordpress_logged_in)"
		path_regexp "(/wp-admin/|/xmlrpc.php|/wp-(app|cron|login|register|mail).php|wp-.*.php|/feed/|index.php|wp-comments-popup.php|wp-links-opml.php|wp-locations.php|sitemap(index)?.xml|[a-z0-9-]+-sitemap([0-9]+)?.xml)"
		method POST
		expression {query} != ''
	}

	route @notcache {
		header cache wp
		php_fastcgi unix//run/php/php7.4-fpm-caddy.sock
    }

	route {
		header cache file
		try_files /wp-content/cache/cache-enabler/{host}{uri}/index.html {path} {path}/index.php?{query}
		php_fastcgi unix//run/php/php7.4-fpm-caddy.sock
	}
	
	request_header /wp-content Cache-Control "public, max-age=2592000, s-maxage=86400"
	request_header /wp-includes Cache-Control "public, max-age=2592000, s-maxage=86400"
	request_header /favicon.ico Cache-Control "public, max-age=2592000, s-maxage=86400"
}

files.voiretmanger.fr {
	root * /var/www/files.voiretmanger.fr
	encode zstd gzip
	file_server browse
	log {
		output file /var/log/caddy/files.voiretmanger.fr.access.log
	}
	import static
}

memoire.nicolasfurno.fr {
	root * /var/www/memoire.nicolasfurno.fr
	encode zstd gzip
	file_server
	log {
		output file /var/log/caddy/memoire.nicolasfurno.fr.access.log
	}
	import static
}

Here is the output of the caddy adapt command :

{
	"logging": {
		"logs": {
			"default": {
				"exclude": [
					"http.log.access.log2",
					"http.log.access.log1",
					"http.log.access.log0"
				]
			},
			"log0": {
				"writer": {
					"filename": "/var/log/caddy/voiretmanger.fr.access.log",
					"output": "file"
				},
				"include": [
					"http.log.access.log0"
				]
			},
			"log1": {
				"writer": {
					"filename": "/var/log/caddy/files.voiretmanger.fr.access.log",
					"output": "file"
				},
				"include": [
					"http.log.access.log1"
				]
			},
			"log2": {
				"writer": {
					"filename": "/var/log/caddy/memoire.nicolasfurno.fr.access.log",
					"output": "file"
				},
				"include": [
					"http.log.access.log2"
				]
			}
		}
	},
	"apps": {
		"http": {
			"servers": {
				"srv0": {
					"listen": [
						":443"
					],
					"routes": [
						{
							"match": [
								{
									"host": [
										"memoire.nicolasfurno.fr"
									]
								}
							],
							"handle": [
								{
									"handler": "subroute",
									"routes": [
										{
											"handle": [
												{
													"handler": "vars",
													"root": "/var/www/memoire.nicolasfurno.fr"
												},
												{
													"encodings": {
														"gzip": {},
														"zstd": {}
													},
													"handler": "encode"
												},
												{
													"handler": "file_server",
													"hide": [
														"Caddyfile"
													]
												}
											]
										}
									]
								}
							],
							"terminal": true
						},
						{
							"match": [
								{
									"host": [
										"files.voiretmanger.fr"
									]
								}
							],
							"handle": [
								{
									"handler": "subroute",
									"routes": [
										{
											"handle": [
												{
													"handler": "vars",
													"root": "/var/www/files.voiretmanger.fr"
												},
												{
													"encodings": {
														"gzip": {},
														"zstd": {}
													},
													"handler": "encode"
												},
												{
													"browse": {},
													"handler": "file_server",
													"hide": [
														"Caddyfile"
													]
												}
											]
										}
									]
								}
							],
							"terminal": true
						},
						{
							"match": [
								{
									"host": [
										"www.voiretmanger.fr",
										"nicolasfurno.fr",
										"www.nicolasfurno.fr",
										"nicolinux.fr",
										"www.nicolinux.fr"
									]
								}
							],
							"handle": [
								{
									"handler": "subroute",
									"routes": [
										{
											"handle": [
												{
													"handler": "static_response",
													"headers": {
														"Location": [
															"https://voiretmanger.fr{http.request.uri}/"
														]
													},
													"status_code": 302
												}
											]
										}
									]
								}
							],
							"terminal": true
						},
						{
							"match": [
								{
									"host": [
										"voiretmanger.fr"
									]
								}
							],
							"handle": [
								{
									"handler": "subroute",
									"routes": [
										{
											"handle": [
												{
													"handler": "vars",
													"root": "/var/www/voiretmanger.fr"
												},
												{
													"handler": "headers",
													"response": {
														"set": {
															"Referrer-Policy": [
															"no-referrer-when-downgrade"
															],
															"Strict-Transport-Security": [
															"max-age=31536000;"
															],
															"X-Content-Type-Options": [
															"nosniff"
															]
														}
													}
												}
											]
										},
										{
											"handle": [
												{
													"handler": "static_response",
													"headers": {
														"Location": [
															"/a-manger"
														]
													},
													"status_code": 302
												}
											],
											"match": [
												{
													"path": [
														"/archives/carte-des-restaurants"
													]
												}
											]
										},
										{
											"handle": [
												{
													"handler": "static_response",
													"headers": {
														"Location": [
															"/soutien"
														]
													},
													"status_code": 302
												}
											],
											"match": [
												{
													"path": [
														"/a-propos/publicite"
													]
												}
											]
										},
										{
											"handle": [
												{
													"handler": "headers",
													"request": {
														"set": {
															"Cache-Control": [
															"public, max-age=2592000, s-maxage=86400"
															]
														}
													}
												}
											],
											"match": [
												{
													"path": [
														"/wp-includes"
													]
												}
											]
										},
										{
											"handle": [
												{
													"handler": "headers",
													"request": {
														"set": {
															"Cache-Control": [
															"public, max-age=2592000, s-maxage=86400"
															]
														}
													}
												}
											],
											"match": [
												{
													"path": [
														"/wp-content"
													]
												}
											]
										},
										{
											"handle": [
												{
													"handler": "headers",
													"request": {
														"set": {
															"Cache-Control": [
															"public, max-age=2592000, s-maxage=86400"
															]
														}
													}
												}
											],
											"match": [
												{
													"path": [
														"/favicon.ic"
													]
												}
											]
										},
										{
											"handle": [
												{
													"encodings": {
														"gzip": {},
														"zstd": {}
													},
													"handler": "encode"
												}
											]
										},
										{
											"handle": [
												{
													"handler": "subroute",
													"routes": [
														{
															"handle": [
															{
															"handler": "headers",
															"response": {
															"set": {
															"Cache": [
															"wp"
															]
															}
															}
															}
															]
														},
														{
															"handle": [
															{
															"handler": "static_response",
															"headers": {
															"Location": [
															"{http.request.uri.path}/"
															]
															},
															"status_code": 308
															}
															],
															"match": [
															{
															"file": {
															"try_files": [
															"{http.request.uri.path}/index.php"
															]
															},
															"not": [
															{
															"path": [
															"*/"
															]
															}
															]
															}
															]
														},
														{
															"handle": [
															{
															"handler": "rewrite",
															"uri": "{http.matchers.file.relative}"
															}
															],
															"match": [
															{
															"file": {
															"split_path": [
															".php"
															],
															"try_files": [
															"{http.request.uri.path}",
															"{http.request.uri.path}/index.php",
															"index.php"
															]
															}
															}
															]
														},
														{
															"handle": [
															{
															"handler": "reverse_proxy",
															"transport": {
															"protocol": "fastcgi",
															"split_path": [
															".php"
															]
															},
															"upstreams": [
															{
															"dial": "unix//run/php/php7.4-fpm-caddy.sock"
															}
															]
															}
															],
															"match": [
															{
															"path": [
															"*.php"
															]
															}
															]
														}
													]
												}
											],
											"match": [
												{
													"expression": "{http.request.uri.query} != ''",
													"header_regexp": {
														"Cookie": {
															"pattern": "(comment_author|wordpress_[a-f0-9]+|wp-postpass|wordpress_logged_in)"
														}
													},
													"method": [
														"POST"
													],
													"path_regexp": {
														"pattern": "(/wp-admin/|/xmlrpc.php|/wp-(app|cron|login|register|mail).php|wp-.*.php|/feed/|index.php|wp-comments-popup.php|wp-links-opml.php|wp-locations.php|sitemap(index)?.xml|[a-z0-9-]+-sitemap([0-9]+)?.xml)"
													}
												}
											]
										},
										{
											"handle": [
												{
													"handler": "subroute",
													"routes": [
														{
															"handle": [
															{
															"handler": "headers",
															"response": {
															"set": {
															"Cache": [
															"file"
															]
															}
															}
															}
															]
														},
														{
															"group": "group0",
															"handle": [
															{
															"handler": "rewrite",
															"uri": "{http.matchers.file.relative}"
															}
															],
															"match": [
															{
															"file": {
															"try_files": [
															"/wp-content/cache/cache-enabler/{http.request.host}{http.request.uri}/index.html",
															"{http.request.uri.path}"
															]
															}
															}
															]
														},
														{
															"group": "group0",
															"handle": [
															{
															"handler": "rewrite",
															"uri": "{http.matchers.file.relative}?{http.request.uri.query}"
															}
															],
															"match": [
															{
															"file": {
															"try_files": [
															"{http.request.uri.path}/index.php"
															]
															}
															}
															]
														},
														{
															"handle": [
															{
															"handler": "static_response",
															"headers": {
															"Location": [
															"{http.request.uri.path}/"
															]
															},
															"status_code": 308
															}
															],
															"match": [
															{
															"file": {
															"try_files": [
															"{http.request.uri.path}/index.php"
															]
															},
															"not": [
															{
															"path": [
															"*/"
															]
															}
															]
															}
															]
														},
														{
															"handle": [
															{
															"handler": "rewrite",
															"uri": "{http.matchers.file.relative}"
															}
															],
															"match": [
															{
															"file": {
															"split_path": [
															".php"
															],
															"try_files": [
															"{http.request.uri.path}",
															"{http.request.uri.path}/index.php",
															"index.php"
															]
															}
															}
															]
														},
														{
															"handle": [
															{
															"handler": "reverse_proxy",
															"transport": {
															"protocol": "fastcgi",
															"split_path": [
															".php"
															]
															},
															"upstreams": [
															{
															"dial": "unix//run/php/php7.4-fpm-caddy.sock"
															}
															]
															}
															],
															"match": [
															{
															"path": [
															"*.php"
															]
															}
															]
														}
													]
												},
												{
													"handler": "file_server",
													"hide": [
														"Caddyfile"
													]
												}
											]
										}
									]
								}
							],
							"terminal": true
						}
					],
					"logs": {
						"logger_names": {
							"files.voiretmanger.fr": "log1",
							"memoire.nicolasfurno.fr": "log2",
							"voiretmanger.fr": "log0"
						},
						"skip_hosts": [
							"www.voiretmanger.fr",
							"nicolasfurno.fr",
							"www.nicolasfurno.fr",
							"nicolinux.fr",
							"www.nicolinux.fr"
						]
					}
				}
			}
		},
		"tls": {
			"automation": {
				"policies": [
					{
						"issuer": {
							"email": "nicolinux@gmail.com",
							"module": "acme"
						}
					}
				]
			}
		}
	}
}

Great - thanks. So, given those JSON routes, is the routing logic wrong (i.e. is the Caddyfile adapter producing the wrong JSON output)? Or do you think this is a bug in the routing logic itself? (Sorry I haven’t delved into the thread deeply so I don’t understand it as well as you do.)

Good question, I think I understand it, but I’m not sure I know the answer, to be perfectly honest with you. :smiley:

I tried to simplify the Caddyfile to produce a simpler JSON.

Caddyfile :

voiretmanger.fr {
    @cache {
        not {
            header_regexp Cookie "comment_author|wordpress_[a-f0-9]+|wp-postpass|wordpress_logged_in"
            path_regexp "(/wp-admin/|/xmlrpc.php|/wp-(app|cron|login|register|mail).php|wp-.*.php|/feed/|index.php|wp-comments-popup.php|wp-links-opml.php|wp-locations.php|sitemap(index)?.xml|[a-z0-9-]+-sitemap([0-9]+)?.xml)"
            method POST
            expression {query} != ''
        }
    }

    route @cache {
        header cache file
        try_files /wp-content/cache/cache-enabler/{host}{uri}/index.html {path} {path}/index.php?{query}
        php_fastcgi unix//run/php/php7.4-fpm-caddy.sock
    }
        
    header cache wp
    php_fastcgi unix//run/php/php7.4-fpm-caddy.sock
}

JSON :

{
	"apps": {
		"http": {
			"servers": {
				"srv0": {
					"listen": [
						":443"
					],
					"routes": [
						{
							"match": [
								{
									"host": [
										"voiretmanger.fr"
									]
								}
							],
							"handle": [
								{
									"handler": "subroute",
									"routes": [
										{
											"handle": [
												{
													"handler": "headers",
													"response": {
														"set": {
															"Cache": [
																"wp"
															]
														}
													}
												}
											]
										},
										{
											"handle": [
												{
													"handler": "subroute",
													"routes": [
														{
															"handle": [
																{
																	"handler": "headers",
																	"response": {
																		"set": {
																			"Cache": [
																				"file"
																			]
																		}
																	}
																}
															]
														},
														{
															"group": "group0",
															"handle": [
																{
																	"handler": "rewrite",
																	"uri": "{http.matchers.file.relative}"
																}
															],
															"match": [
																{
																	"file": {
																		"try_files": [
																			"/wp-content/cache/cache-enabler/{http.request.host}{http.request.uri}/index.html",
																			"{http.request.uri.path}"
																		]
																	}
																}
															]
														},
														{
															"group": "group0",
															"handle": [
																{
																	"handler": "rewrite",
																	"uri": "{http.matchers.file.relative}?{http.request.uri.query}"
																}
															],
															"match": [
																{
																	"file": {
																		"try_files": [
																			"{http.request.uri.path}/index.php"
																		]
																	}
																}
															]
														},
														{
															"handle": [
																{
																	"handler": "static_response",
																	"headers": {
																		"Location": [
																			"{http.request.uri.path}/"
																		]
																	},
																	"status_code": 308
																}
															],
															"match": [
																{
																	"file": {
																		"try_files": [
																			"{http.request.uri.path}/index.php"
																		]
																	},
																	"not": [
																		{
																			"path": [
																				"*/"
																			]
																		}
																	]
																}
															]
														},
														{
															"handle": [
																{
																	"handler": "rewrite",
																	"uri": "{http.matchers.file.relative}"
																}
															],
															"match": [
																{
																	"file": {
																		"split_path": [
																			".php"
																		],
																		"try_files": [
																			"{http.request.uri.path}",
																			"{http.request.uri.path}/index.php",
																			"index.php"
																		]
																	}
																}
															]
														},
														{
															"handle": [
																{
																	"handler": "reverse_proxy",
																	"transport": {
																		"protocol": "fastcgi",
																		"split_path": [
																			".php"
																		]
																	},
																	"upstreams": [
																		{
																			"dial": "unix//run/php/php7.4-fpm-caddy.sock"
																		}
																	]
																}
															],
															"match": [
																{
																	"path": [
																		"*.php"
																	]
																}
															]
														}
													]
												}
											],
											"match": [
												{
													"not": [
														{
															"expression": "{http.request.uri.query} != ''",
															"header_regexp": {
																"Cookie": {
																	"pattern": "comment_author|wordpress_[a-f0-9]+|wp-postpass|wordpress_logged_in"
																}
															},
															"method": [
																"POST"
															],
															"path_regexp": {
																"pattern": "(/wp-admin/|/xmlrpc.php|/wp-(app|cron|login|register|mail).php|wp-.*.php|/feed/|index.php|wp-comments-popup.php|wp-links-opml.php|wp-locations.php|sitemap(index)?.xml|[a-z0-9-]+-sitemap([0-9]+)?.xml)"
															}
														}
													]
												}
											]
										},
										{
											"handle": [
												{
													"handler": "static_response",
													"headers": {
														"Location": [
															"{http.request.uri.path}/"
														]
													},
													"status_code": 308
												}
											],
											"match": [
												{
													"file": {
														"try_files": [
															"{http.request.uri.path}/index.php"
														]
													},
													"not": [
														{
															"path": [
																"*/"
															]
														}
													]
												}
											]
										},
										{
											"handle": [
												{
													"handler": "rewrite",
													"uri": "{http.matchers.file.relative}"
												}
											],
											"match": [
												{
													"file": {
														"split_path": [
															".php"
														],
														"try_files": [
															"{http.request.uri.path}",
															"{http.request.uri.path}/index.php",
															"index.php"
														]
													}
												}
											]
										},
										{
											"handle": [
												{
													"handler": "reverse_proxy",
													"transport": {
														"protocol": "fastcgi",
														"split_path": [
															".php"
														]
													},
													"upstreams": [
														{
															"dial": "unix//run/php/php7.4-fpm-caddy.sock"
														}
													]
												}
											],
											"match": [
												{
													"path": [
														"*.php"
													]
												}
											]
										}
									]
								}
							],
							"terminal": true
						}
					]
				}
			}
		}
	}
}

It produces a lot of indentation and I don’t know the internals of Caddy 2 enough to see if there’s an obvious error here.

At first glance, it seems OK and I would guess there is a bug, but maybe I’m misunderstanding something ?

I think the JSON looks right as well, I don’t think it’s a Caddyfile parsing issue. Just for the sake of readability, this is what the matcher looks like in the JSON:

"match": [
    {
        "expression": "{http.request.uri.query} != ''",
        "header_regexp": {
            "Cookie": {
                "pattern": "(comment_author|wordpress_[a-f0-9]+|wp-postpass|wordpress_logged_in)"
            }
        },
        "method": [
            "POST"
        ],
        "path_regexp": {
            "pattern": "(/wp-admin/|/xmlrpc.php|/wp-(app|cron|login|register|mail).php|wp-.*.php|/feed/|index.php|wp-comments-popup.php|wp-links-opml.php|wp-locations.php|sitemap(index)?.xml|[a-z0-9-]+-sitemap([0-9]+)?.xml)"
        }
    }
]

Edit: waiiiit a minute… why isn’t there a not matcher wrapping that? :thinking:

Edit2: Hmm, when I adapt it myself it does have the not matcher. Did you mistakenly adapt that from your Caddyfile when it didn’t have not?

"match": [
    {
        "not": [
            {
                "expression": "{http.request.uri.query} != ''",
                "header_regexp": {
                    "Cookie": {
                        "pattern": "comment_author|wordpress_[a-f0-9]+|wp-postpass|wordpress_logged_in"
                    }
                },
                "method": [
                    "POST"
                ],
                "path_regexp": {
                    "pattern": "(/wp-admin/|/xmlrpc.php|/wp-(app|cron|login|register|mail).php|wp-.*.php|/feed/|index.php|wp-comments-popup.php|wp-links-opml.php|wp-locations.php|sitemap(index)?.xml|[a-z0-9-]+-sitemap([0-9]+)?.xml)"
                }
            }
        ]
    }
]
1 Like

I tried both, sorry for the confusion.

In this post, I tried with the not matcher and I can see it in the JSON output. In the previous post, I did not.

So, maybe I should recap what I would like to achieve and what my reasoning was. It could be helpful to help you understand precisely what is wrong with the code or the general logic.

I have a WordPress blog and I would like to bypass the CMS and PHP by using directly a static HTML cache stored on my server. The catch is that some URLs and more importantly some users should not access this cached version and should go directly to WordPress, as usual.


My idea was to set up two distincts routes :

  • one that can use the static cache directly and bypass WordPress, unless at least one of these conditions is true :
    • the path contains a specific pattern (for example, wp-admin) ;
    • the cookie contains a specific pattern (for example wordpress_logged_in) ;
    • the request is a query ;
    • the request is a POST request ;
  • one « standard » that go to WordPress and not the cache, when at least one of the previous conditions is true.

Using the Caddyfile, my idea was to define a route based on the exception first. So I created a named matcher @cache with the four conditions surrounded with the not matcher. Here’s what I came up with :

@cache {
    not {
		header_regexp Cookie "comment_author|wordpress_[a-f0-9]+|wp-postpass|wordpress_logged_in"
		path_regexp "(/wp-admin/|/xmlrpc.php|/wp-(app|cron|login|register|mail).php|wp-.*.php|/feed/|index.php|wp-comments-popup.php|wp-links-opml.php|wp-locations.php|sitemap(index)?.xml|[a-z0-9-]+-sitemap([0-9]+)?.xml)"
		method POST
		expression {query} != ""
	}
}

This named matcher should contain every requests that can go to the cache directly and bypass PHP/WordPress completely if the cached version already exists. If not, a backup plan redirects to PHP and the CMS.

I created this route, with the try_files that tries first the static cache, then fallbacks to the standard PHP, so that WordPress can generate the page and display something :

route @cache {
   try_files /wp-content/cache/cache-enabler/{host}{uri}/index.html {path} {path}/index.php?{query}
}

For the requests that should not be handled by this first route, I can just use the standard php_fastcgi of Caddy 2, which works perfectly fine for a WordPress site :

php_fastcgi unix//run/php/php7.4-fpm-caddy.sock

Until this point, have I done something wrong ? Is there a syntax or a logic error somewhere ? If we can agree it should work in theory, I suppose we can then try to find why it does not in practice.

In practice, every request is handled by the @cached route and never by the direct php_fastcgi way. I know that, because even if I’m connected to my site, I see only the cached version. I can still access the admin, since the try_filesfalls back to PHP, but the initial idea was to never see the cache when I’m connected.

I tried several ideas, one of which was to create two named handlers and two explicit routes. I also tried to simplify the @cached route by using only one condition at a time. One more experiment was to revert the whole logic, so I did not use the not matcher inside the named handler.

Until now, I have not found a way to make this work. If it’s a limitation of the Caddyfile at this moment, that’s fine and understandable, I would like to know to stop searching. :slight_smile:

And if something is not clear, let me know !

Thanks again ! :+1:

1 Like

What do your cache files look like on disk? I just took a look at your OP again and you have the filename like this:

/cache-enabler/${http_host}${cache_uri}index.html

But in the Caddyfile you have this:

/cache-enabler/{host}{uri}/index.html

Notice that you have a / before index.html that isn’t there in the nginx version. Does that matter here? Do you create a cache directory with an index.html in it, or are you making files with long names?

Also, were you able to test each of the exclusion rules in isolation? If you remove the other 3 and just test the one, do each of them work?

An index.html file inside a folder per URL generated by WordPress, like this :

➜  007-spectre-mendes la
total 36K
-rw-r--r-- 1 caddy caddy 33K May 11 01:06 index.html
➜  007-spectre-mendes pwd
/var/www/voiretmanger.fr/wp-content/cache/cache-enabler/voiretmanger.fr/007-spectre-mendes

I think the / is there because Caddy removes it from {uri} placeholder ? I’m not sure if it’s even necessary, but I know it’s working fine this way. :slight_smile:

The only rule that worked for me was the path_regexp one. I could not make the other three work.

I forgot to mention it in the recap, but as you can see in earlier posts, I added a header to see what route was used for each request.

Right now, using this setup, I always have the “file” value, so the named matcher takes every request.

    @cache {
        not {
            header_regexp Cookie "comment_author|wordpress_[a-f0-9]+|wp-postpass|wordpress_logged_in"
            path_regexp "(/wp-admin/|/xmlrpc.php|/wp-(app|cron|login|register|mail).php|wp-.*.php|/feed/|index.php|wp-comments-popup.php|wp-links-opml.php|wp-locations.php|sitemap(index)?.xml|[a-z0-9-]+-sitemap([0-9]+)?.xml)"
            method POST
            expression {query} != ''
        }
    }

    route @cache {
        header cache file
        try_files /wp-content/cache/cache-enabler/{host}{uri}/index.html {path} {path}/index.php?{query}
    }
        
    header cache wp
    php_fastcgi unix//run/php/php7.4-fpm-caddy.sock
}
1 Like

Oh geez, I think I might know the solution. So currently the logic is essentially:

not (header && path && method && expression)

But I think what you actually need is (unless my brain is just fried tonight, sorry if it is):

(not header) || (not path) || (not method) || (not expression)

So… let’s try:

@cache {
    not header_regexp Cookie "comment_author|wordpress_[a-f0-9]+|wp-postpass|wordpress_logged_in"
    not path_regexp "(/wp-admin/|/xmlrpc.php|/wp-(app|cron|login|register|mail).php|wp-.*.php|/feed/|index.php|wp-comments-popup.php|wp-links-opml.php|wp-locations.php|sitemap(index)?.xml|[a-z0-9-]+-sitemap([0-9]+)?.xml)"
    not method POST
    not expression {query} != ''
}

Maybe…?

:man_shrugging:

The key here is that when you use multiple not matchers beside eachother, they get ORed together, but if you use one big not matcher then the contents are ANDed. It’s tricky.

4 Likes

Think you’re on to something with the logic issue. Damn those logic issues.

With this:

@cache {
  header_regexp Cookie "comment_author|wordpress_[a-f0-9]+|wp-postpass|wordpress_logged_in"
  path_regexp "(/wp-admin/|/xmlrpc.php|/wp-(app|cron|login|register|mail).php|wp-.*.php|/feed/|index.php|wp-comments-popup.php|wp-links-opml.php|wp-locations.php|sitemap(index)?.xml|[a-z0-9-]+-sitemap([0-9]+)?.xml)"
  method POST
  expression {query} != ''
}

For a request with the header, but none of the other elements, this doesn’t match :x:

If we invert it, therefore:

@cache {
  not {
    header_regexp Cookie "comment_author|wordpress_[a-f0-9]+|wp-postpass|wordpress_logged_in"
    path_regexp "(/wp-admin/|/xmlrpc.php|/wp-(app|cron|login|register|mail).php|wp-.*.php|/feed/|index.php|wp-comments-popup.php|wp-links-opml.php|wp-locations.php|sitemap(index)?.xml|[a-z0-9-]+-sitemap([0-9]+)?.xml)"
    method POST
    expression {query} != ''
  }
}

A request with the header but no other element now does match :white_check_mark:

But we’re sending those requests to the cache! That’s not what we wanted!

Well, what kind of request now doesn’t match the above? Any request that actually has all four elements now bypasses the cache. So to bypass it you need to POST a request to one of those paths, with a query, and send a Cookie with it.

Logic, man.

2 Likes

You’re right ! It was the issue and it seems to work fine with multiple not marchers ! :+1:

At first glance, everything works now. Visitors see directly the cache, but as I’m logged in, I access directly WordPress instead. I tried, a search (so a query request) works also, and the WP admin is always using WordPress.

I don’t know why I did not thought of trying this logic… :man_facepalming:

4 Likes

Dang. Nice job, team. :mag:

3 Likes