Custom writer only gets body if request is encoded with gzip

Plugins
1

I’m currently trying to fix FrankenWP’s Sidekick-cache plugin, as it seems to only be able to save requests if the request is encoded with gzip or brotli (added by another plugin). The cache works simply as a handler/middleware that then runs next.ServeHTTP(<custom response writer>, r) as seen here

With encoded requests it works as expected, the custom writer gets the body and saves it to cache. But if it isn’t encoded with gzip the body never reaches the custom writer. I’d suspect this happens because there isn’t a handler after it just gives the no-op handler :thinking:

Is there any way to implement this properly :person_shrugging:?

I have a modified version running that adds the header even if cache misses:
curl https://scalewp.mikroni.fi -I → x-mikroniwp-cache: OHI (= miss)
curl -H 'Accept-encoding: zstd' https://scalewp.mikroni.fi -I → x-mikroniwp-cache: OHI (= miss)
curl -H 'Accept-encoding: br' https://scalewp.mikroni.fi -I → x-mikrowp-cache: HIT

But this problem exists even on the vanilla frankenwp compose example (just remove the CACHE_RESPONSE_CODES: ${CACHE_RESPONSE_CODES:-000} line so it will cache requests

And here’s the caddy log for the “wordpress” container of the example compose.yml, with added log { output stdout, level debug }. I did two requests without encoding and one with gzip.

wordpress-1  | WordPress not found in /var/www/html - copying now...
wordpress-1  | WARNING: '/var/www/html/wp-content/plugins/akismet' exists! (not copying the WordPress version)
wordpress-1  | WARNING: '/var/www/html/wp-content/themes/twentytwentyfour' exists! (not copying the WordPress version)
wordpress-1  | WARNING: '/var/www/html/wp-content/themes/twentytwentythree' exists! (not copying the WordPress version)
wordpress-1  | WARNING: '/var/www/html/wp-content/themes/twentytwentytwo' exists! (not copying the WordPress version)
wordpress-1  | Complete! WordPress has been successfully copied to /var/www/html
wordpress-1  | No 'wp-config.php' found in /var/www/html, but 'WORDPRESS_...' variables supplied; copying 'wp-config-docker.php' (WORDPRESS_CONFIG_EXTRA WORDPRESS_DB_HOST WORDPRESS_DB_NAME WORDPRESS_DB_PASSWORD WORDPRESS_DB_USER WORDPRESS_DEBUG WORDPRESS_TABLE_PREFIX)
wordpress-1  | 2024/07/25 20:02:40.711	INFO	using config from file	{"file": "/etc/caddy/Caddyfile"}
wordpress-1  | 2024/07/25 20:02:40.712	INFO	adapted config to JSON	{"adapter": "caddyfile"}
wordpress-1  | 2024/07/25 20:02:40.712	WARN	Caddyfile input is not formatted; run 'caddy fmt --overwrite' to fix inconsistencies	{"adapter": "caddyfile", "file": "/etc/caddy/Caddyfile", "line": 31}
wordpress-1  | 2024/07/25 20:02:40.712	INFO	redirected default logger	{"from": "stderr", "to": "stdout"}
wordpress-1  | 2024/07/25 20:02:40.712	INFO	admin	admin endpoint started	{"address": "localhost:2019", "enforce_origin": false, "origins": ["//localhost:2019", "//[::1]:2019", "//127.0.0.1:2019"]}
wordpress-1  | 2024/07/25 20:02:40.712	INFO	tls.cache.maintenance	started background certificate maintenance	{"cache": "0xc00062eb00"}
wordpress-1  | 2024/07/25 20:02:40.714	WARN	http.auto_https	server is listening only on the HTTP port, so no automatic HTTPS will be applied to this server	{"server_name": "srv0", "http_port": 80}
wordpress-1  | 2024/07/25 20:02:40.714	DEBUG	http.auto_https	adjusted config	{"tls": {"automation":{"policies":[{}]}}, "http": {"servers":{"srv0":{"listen":[":80"],"routes":[{"handle":[{"handler":"vars","root":"/var/www/html/"},{"BypassHome":false,"BypassPathPrefixes":["/wp-admin","/wp-content","/wp-includes","/wp-json","/feed"],"CacheResponseCodes":["2","404","405"],"Loc":"/var/www/html/wp-content/cache","PurgeKey":"","PurgePath":"/__cache/purge","Store":null,"TTL":80000,"handler":"wp_cache"},{"encodings":{"br":{},"gzip":{},"zstd":{}},"handler":"encode","prefer":["br","zstd","gzip"]}]},{"handle":[{"handler":"static_response","headers":{"Location":["{http.request.orig_uri.path}/"]},"status_code":308}]},{"handle":[{"handler":"rewrite","uri":"{http.matchers.file.relative}"}]},{"handle":[{"handler":"php","split_path":[".php"]}]},{"handle":[{"handler":"file_server"}]}],"automatic_https":{"disable":true,"disable_redirects":true},"logs":{"default_logger_name":"log0"}}}}}
wordpress-1  | 2024/07/25 20:02:40.716	INFO	FrankenPHP started 🐘	{"php_version": "8.3.7"}
wordpress-1  | 2024/07/25 20:02:40.716	WARN	pki.ca.local	installing root certificate (you might be prompted for password)	{"path": "storage:pki/authorities/local/root.crt"}
wordpress-1  | 2024/07/25 20:02:40.716	INFO	warning: "certutil" is not available, install "certutil" with "apt install libnss3-tools" or "yum install nss-tools" and try again
wordpress-1  | 2024/07/25 20:02:40.716	INFO	define JAVA_HOME environment variable to use the Java trust
wordpress-1  | 2024/07/25 20:02:41.179	INFO	certificate installed properly in linux trusts
wordpress-1  | 2024/07/25 20:02:41.179	DEBUG	http	starting server loop	{"address": "[::]:80", "tls": false, "http3": false}
wordpress-1  | 2024/07/25 20:02:41.179	INFO	http.log	server running	{"name": "srv0", "protocols": ["h1", "h2", "h3"]}
wordpress-1  | 2024/07/25 20:02:41.179	INFO	autosaved config (load with --resume flag)	{"file": "/config/caddy/autosave.json"}
wordpress-1  | 2024/07/25 20:02:41.179	INFO	serving initial configuration
wordpress-1  | 2024/07/25 20:02:41.183	INFO	tls	cleaning storage unit	{"storage": "FileStorage:/data/caddy"}
wordpress-1  | 2024/07/25 20:02:41.183	INFO	tls	finished cleaning storage units
wordpress-1  | 2024/07/25 20:02:57.031	DEBUG	http.handlers.wp_cache	HTTP Version	{"Version": "HTTP/1.1"}
wordpress-1  | 2024/07/25 20:02:57.031	DEBUG	http.handlers.wp_cache	Getting key from cache	{"key": "none::+"}
wordpress-1  | 2024/07/25 20:02:57.031	DEBUG	http.handlers.wp_cache	wp cache - error - none::/	{"error": "Key not found in cache"}
wordpress-1  | 2024/07/25 20:02:57.031	DEBUG	http.handlers.rewrite	rewrote request	{"request": {"remote_ip": "172.23.0.1", "remote_port": "38534", "client_ip": "172.23.0.1", "proto": "HTTP/1.1", "method": "HEAD", "host": "localhost:8100", "uri": "/", "headers": {"Accept": ["*/*"], "User-Agent": ["curl/7.81.0"]}}, "method": "HEAD", "uri": "/index.php"}
wordpress-1  | 2024/07/25 20:02:57.341	DEBUG	http.handlers.wp_cache	==========-SetHeader-==========
wordpress-1  | 2024/07/25 20:02:57.342	INFO	http.log.access.log0	handled request	{"request": {"remote_ip": "172.23.0.1", "remote_port": "38534", "client_ip": "172.23.0.1", "proto": "HTTP/1.1", "method": "HEAD", "host": "localhost:8100", "uri": "/", "headers": {"User-Agent": ["curl/7.81.0"], "Accept": ["*/*"]}}, "bytes_read": 0, "user_id": "", "duration": 0.311211509, "size": 0, "status": 200, "resp_headers": {"Link": ["<http://localhost:8100/index.php?rest_route=/>; rel=\"https://api.w.org/\""], "Server": ["Caddy"], "X-Powered-By": ["PHP/8.3.7"], "Content-Type": ["text/html; charset=UTF-8"]}}
wordpress-1  | 2024/07/25 20:02:58.040	DEBUG	http.handlers.wp_cache	HTTP Version	{"Version": "HTTP/1.1"}
wordpress-1  | 2024/07/25 20:02:58.040	DEBUG	http.handlers.wp_cache	Getting key from cache	{"key": "none::+"}
wordpress-1  | 2024/07/25 20:02:58.040	DEBUG	http.handlers.wp_cache	wp cache - error - none::/	{"error": "Key not found in cache"}
wordpress-1  | 2024/07/25 20:02:58.040	DEBUG	http.handlers.rewrite	rewrote request	{"request": {"remote_ip": "172.23.0.1", "remote_port": "58102", "client_ip": "172.23.0.1", "proto": "HTTP/1.1", "method": "HEAD", "host": "localhost:8100", "uri": "/", "headers": {"User-Agent": ["curl/7.81.0"], "Accept": ["*/*"]}}, "method": "HEAD", "uri": "/index.php"}
wordpress-1  | 2024/07/25 20:02:58.057	DEBUG	http.handlers.wp_cache	==========-SetHeader-==========
wordpress-1  | 2024/07/25 20:02:58.058	INFO	http.log.access.log0	handled request	{"request": {"remote_ip": "172.23.0.1", "remote_port": "58102", "client_ip": "172.23.0.1", "proto": "HTTP/1.1", "method": "HEAD", "host": "localhost:8100", "uri": "/", "headers": {"User-Agent": ["curl/7.81.0"], "Accept": ["*/*"]}}, "bytes_read": 0, "user_id": "", "duration": 0.01813057, "size": 0, "status": 200, "resp_headers": {"Link": ["<http://localhost:8100/index.php?rest_route=/>; rel=\"https://api.w.org/\""], "Server": ["Caddy"], "X-Powered-By": ["PHP/8.3.7"], "Content-Type": ["text/html; charset=UTF-8"]}}
wordpress-1  | 2024/07/25 20:03:10.544	DEBUG	admin.api	received request	{"method": "GET", "host": "localhost:2019", "uri": "/metrics", "remote_ip": "127.0.0.1", "remote_port": "53544", "headers": {"Accept":["*/*"],"User-Agent":["curl/7.88.1"]}}
wordpress-1  | 2024/07/25 20:03:40.585	DEBUG	admin.api	received request	{"method": "GET", "host": "localhost:2019", "uri": "/metrics", "remote_ip": "127.0.0.1", "remote_port": "33782", "headers": {"Accept":["*/*"],"User-Agent":["curl/7.88.1"]}}
wordpress-1  | 2024/07/25 20:04:10.633	DEBUG	admin.api	received request	{"method": "GET", "host": "localhost:2019", "uri": "/metrics", "remote_ip": "127.0.0.1", "remote_port": "34444", "headers": {"Accept":["*/*"],"User-Agent":["curl/7.88.1"]}}
wordpress-1  | 2024/07/25 20:04:40.671	DEBUG	admin.api	received request	{"method": "GET", "host": "localhost:2019", "uri": "/metrics", "remote_ip": "127.0.0.1", "remote_port": "47166", "headers": {"Accept":["*/*"],"User-Agent":["curl/7.88.1"]}}
wordpress-1  | 2024/07/25 20:05:10.711	DEBUG	admin.api	received request	{"method": "GET", "host": "localhost:2019", "uri": "/metrics", "remote_ip": "127.0.0.1", "remote_port": "35614", "headers": {"Accept":["*/*"],"User-Agent":["curl/7.88.1"]}}
wordpress-1  | 2024/07/25 20:05:40.749	DEBUG	admin.api	received request	{"method": "GET", "host": "localhost:2019", "uri": "/metrics", "remote_ip": "127.0.0.1", "remote_port": "58280", "headers": {"Accept":["*/*"],"User-Agent":["curl/7.88.1"]}}
wordpress-1  | 2024/07/25 20:06:10.799	DEBUG	admin.api	received request	{"method": "GET", "host": "localhost:2019", "uri": "/metrics", "remote_ip": "127.0.0.1", "remote_port": "58612", "headers": {"Accept":["*/*"],"User-Agent":["curl/7.88.1"]}}
^Ywordpress-1  | 2024/07/25 20:06:40.164	DEBUG	http.handlers.wp_cache	HTTP Version	{"Version": "HTTP/1.1"}
wordpress-1  | 2024/07/25 20:06:40.164	DEBUG	http.handlers.wp_cache	Getting key from cache	{"key": "gzip::+"}
wordpress-1  | 2024/07/25 20:06:40.164	DEBUG	http.handlers.wp_cache	Pulled key from memory	{"key": "gzip::+"}
wordpress-1  | 2024/07/25 20:06:40.164	DEBUG	http.handlers.wp_cache	Cache hit	{"key": "gzip::+"}
wordpress-1  | 2024/07/25 20:06:40.164	INFO	http.log.access.log0	handled request	{"request": {"remote_ip": "172.23.0.1", "remote_port": "36284", "client_ip": "172.23.0.1", "proto": "HTTP/1.1", "method": "HEAD", "host": "localhost:8100", "uri": "/", "headers": {"User-Agent": ["curl/7.81.0"], "Accept": ["*/*"], "Accept-Encoding": ["gzip"]}}, "bytes_read": 0, "user_id": "", "duration": 0.000169446, "size": 15028, "status": 200, "resp_headers": {"Server": ["Caddy"], "X-Wpeverywhere-Cache": ["HIT"], "Content-Type": ["text/html; charset=UTF-8"], "Vary": ["Accept-Encoding"], "Content-Encoding": ["gzip"]}}
wordpress-1  | 2024/07/25 20:06:40.838	DEBUG	admin.api	received request	{"method": "GET", "host": "localhost:2019", "uri": "/metrics", "remote_ip": "127.0.0.1", "remote_port": "37084", "headers": {"Accept":["*/*"],"User-Agent":["curl/7.88.1"]}}
2

Forgot curl -I means head :man_facepalming:

1 Like
3

So was the problem just with how you were using curl? Or is there still a problem?

2 Likes
4

This isn’t a problem anymore and I marked this thread resolved. The plugin worked as it should, I just used the wrong curl parameters. I use curl pretty rarely so I didn’t know it changes the method to HEAD, which of course doesn’t have the body :smile:

Plugin still has some bugs (regarding HTTP/2 and how the plugin caches responses) but I think I can handle it myself and looking at the replace-response plugin :smile:

Y’all keep being awesome :metal:

2 Likes
5

This topic was automatically closed after 30 days. New replies are no longer allowed.