The {scheme} placeholder will output http or https depending on the current connection, and {host} will output xxx.udance.com.au (or whatever the current hostname is, so it doesn’t need to be hardcoded), and * will match the rest, whatever the path/query of the request is.
header and path are basically simplified pattern matchers, and usually faster than their _regexp cousins for simple cases.
So do bots leave Referer empty or put something in it that does not match {scheme}://{host}* i.e. contain regexp https?://xxx\.udance\.com\.au
That’s the assumption, yes. This is by no means fool-proof. It only helps to block very unsophisticated bots. It’s trivial to just include the Referer header if someone figures out that’s the condition used to block requests.
I read into this that it should work with one and fail with the other. This isn’t the behaviour I’m seeing. My testing reveals it works for both.
root@wp-xxx:/usr/local/www # curl -v https://xxx.udance.com.au/wp-login.php
* Trying 10.1.1.4:443...
* Connected to xxx.udance.com.au (10.1.1.4) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /usr/local/share/certs/ca-root-nss.crt
* CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
* ALPN, server accepted to use h2
* Server certificate:
* subject: CN=*.udance.com.au
* start date: Jul 31 18:08:01 2021 GMT
* expire date: Oct 29 18:07:59 2021 GMT
* subjectAltName: host "xxx.udance.com.au" matched cert's "*.udance.com.au"
* issuer: C=US; O=Let's Encrypt; CN=R3
* SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x801480800)
> GET /wp-login.php HTTP/2
> Host: xxx.udance.com.au
> user-agent: curl/7.77.0
> accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* Connection state changed (MAX_CONCURRENT_STREAMS == 250)!
< HTTP/2 200
< cache-control: no-cache, must-revalidate, max-age=0
< content-type: text/html; charset=UTF-8
< date: Mon, 30 Aug 2021 07:27:30 GMT
< expires: Wed, 11 Jan 1984 05:00:00 GMT
< server: Caddy
< server: Caddy
< set-cookie: wordpress_test_cookie=WP%20Cookie%20check; path=/; secure
< vary: Accept-Encoding
< x-frame-options: SAMEORIGIN
< x-powered-by: PHP/7.4.21
<
<!DOCTYPE html>
<html lang="en-US">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<title>Log In ‹ Test — WordPress</title>
<meta name='robots' content='max-image-preview:large, noindex, noarchive' />
<link rel='dns-prefetch' href='//s.w.org' />
<link rel='stylesheet' id='dashicons-css' href='https://xxx.udance.com.au/wp-includes/css/dashicons.min.css?ver=5.8' media='all' />
<link rel='stylesheet' id='buttons-css' href='https://xxx.udance.com.au/wp-includes/css/buttons.min.css?ver=5.8' media='all' />
<link rel='stylesheet' id='forms-css' href='https://xxx.udance.com.au/wp-admin/css/forms.min.css?ver=5.8' media='all' />
<link rel='stylesheet' id='l10n-css' href='https://xxx.udance.com.au/wp-admin/css/l10n.min.css?ver=5.8' media='all' />
<link rel='stylesheet' id='login-css' href='https://xxx.udance.com.au/wp-admin/css/login.min.css?ver=5.8' media='all' />
<meta name='referrer' content='strict-origin-when-cross-origin' />
<meta name="viewport" content="width=device-width" />
</head>
<body class="login no-js login-action-login wp-core-ui locale-en-us">
<script type="text/javascript">
document.body.className = document.body.className.replace('no-js','js');
</script>
<div id="login">
<h1><a href="https://wordpress.org/">Powered by WordPress</a></h1>
<form name="loginform" id="loginform" action="https://xxx.udance.com.au/wp-login.php" method="post">
<p>
<label for="user_login">Username or Email Address</label>
<input type="text" name="log" id="user_login" class="input" value="" size="20" autocapitalize="off" />
</p>
<div class="user-pass-wrap">
<label for="user_pass">Password</label>
<div class="wp-pwd">
<input type="password" name="pwd" id="user_pass" class="input password-input" value="" size="20" />
<button type="button" class="button button-secondary wp-hide-pw hide-if-no-js" data-toggle="0" aria-label="Show password">
<span class="dashicons dashicons-visibility" aria-hidden="true"></span>
</button>
</div>
</div>
<p class="forgetmenot"><input name="rememberme" type="checkbox" id="rememberme" value="forever" /> <label for="rememberme">Remember Me</label></p>
<p class="submit">
<input type="submit" name="wp-submit" id="wp-submit" class="button button-primary button-large" value="Log In" />
<input type="hidden" name="redirect_to" value="https://xxx.udance.com.au/wp-admin/" />
<input type="hidden" name="testcookie" value="1" />
</p>
</form>
<p id="nav">
<a href="https://xxx.udance.com.au/wp-login.php?action=lostpassword">Lost your password?</a>
</p>
<script type="text/javascript">
function wp_attempt_focus() {setTimeout( function() {try {d = document.getElementById( "user_login" );d.focus(); d.select();} catch( er ) {}}, 200);}
wp_attempt_focus();
if ( typeof wpOnload === 'function' ) { wpOnload() } </script>
<p id="backtoblog">
<a href="https://xxx.udance.com.au/">← Go to Test</a> </p>
</div>
<script src='https://xxx.udance.com.au/wp-includes/js/jquery/jquery.min.js?ver=3.6.0' id='jquery-core-js'></script>
<script src='https://xxx.udance.com.au/wp-includes/js/jquery/jquery-migrate.min.js?ver=3.3.2' id='jquery-migrate-js'></script>
<script id='zxcvbn-async-js-extra'>
var _zxcvbnSettings = {"src":"https:\/\/xxx.udance.com.au\/wp-includes\/js\/zxcvbn.min.js"};
</script>
<script src='https://xxx.udance.com.au/wp-includes/js/zxcvbn-async.min.js?ver=1.0' id='zxcvbn-async-js'></script>
<script src='https://xxx.udance.com.au/wp-includes/js/dist/vendor/regenerator-runtime.min.js?ver=0.13.7' id='regenerator-runtime-js'></script>
<script src='https://xxx.udance.com.au/wp-includes/js/dist/vendor/wp-polyfill.min.js?ver=3.15.0' id='wp-polyfill-js'></script>
<script src='https://xxx.udance.com.au/wp-includes/js/dist/hooks.min.js?ver=a7edae857aab69d69fa10d5aef23a5de' id='wp-hooks-js'></script>
<script src='https://xxx.udance.com.au/wp-includes/js/dist/i18n.min.js?ver=5f1269854226b4dd90450db411a12b79' id='wp-i18n-js'></script>
<script id='wp-i18n-js-after'>
wp.i18n.setLocaleData( { 'text direction\u0004ltr': [ 'ltr' ] } );
</script>
<script id='password-strength-meter-js-extra'>
var pwsL10n = {"unknown":"Password strength unknown","short":"Very weak","bad":"Weak","good":"Medium","strong":"Strong","mismatch":"Mismatch"};
</script>
<script id='password-strength-meter-js-translations'>
( function( domain, translations ) {
var localeData = translations.locale_data[ domain ] || translations.locale_data.messages;
localeData[""].domain = domain;
wp.i18n.setLocaleData( localeData, domain );
} )( "default", { "locale_data": { "messages": { "": {} } } } );
</script>
<script src='https://xxx.udance.com.au/wp-admin/js/password-strength-meter.min.js?ver=5.8' id='password-strength-meter-js'></script>
<script src='https://xxx.udance.com.au/wp-includes/js/underscore.min.js?ver=1.8.3' id='underscore-js'></script>
<script id='wp-util-js-extra'>
var _wpUtilSettings = {"ajax":{"url":"\/wp-admin\/admin-ajax.php"}};
</script>
<script src='https://xxx.udance.com.au/wp-includes/js/wp-util.min.js?ver=5.8' id='wp-util-js'></script>
<script id='user-profile-js-extra'>
var userProfileL10n = {"user_id":"0","nonce":"ea886466a1"};
</script>
<script id='user-profile-js-translations'>
( function( domain, translations ) {
var localeData = translations.locale_data[ domain ] || translations.locale_data.messages;
localeData[""].domain = domain;
wp.i18n.setLocaleData( localeData, domain );
} )( "default", { "locale_data": { "messages": { "": {} } } } );
</script>
<script src='https://xxx.udance.com.au/wp-admin/js/user-profile.min.js?ver=5.8' id='user-profile-js'></script>
<script>
/(trident|msie)/i.test(navigator.userAgent)&&document.getElementById&&window.addEventListener&&window.addEventListener("hashchange",(function(){var t,e=location.hash.substring(1);/^[A-z0-9_-]+$/.test(e)&&(t=document.getElementById(e))&&(/^(?:a|select|input|button|textarea)$/i.test(t.tagName)||(t.tabIndex=-1),t.focus())}),!1);
</script>
<div class="clear"></div>
</body>
</html>
* Connection #0 to host xxx.udance.com.au left intact
If I use the not version of the no-referer matcher…
@no-referer not header Referer {scheme}://{host}*
it fails for both.
root@wp-xxx:/usr/local/www # curl -v https://xxx.udance.com.au/wp-login.php
* Trying 10.1.1.4:443...
* Connected to xxx.udance.com.au (10.1.1.4) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /usr/local/share/certs/ca-root-nss.crt
* CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
* ALPN, server accepted to use h2
* Server certificate:
* subject: CN=*.udance.com.au
* start date: Jul 31 18:08:01 2021 GMT
* expire date: Oct 29 18:07:59 2021 GMT
* subjectAltName: host "xxx.udance.com.au" matched cert's "*.udance.com.au"
* issuer: C=US; O=Let's Encrypt; CN=R3
* SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x801480800)
> GET /wp-login.php HTTP/2
> Host: xxx.udance.com.au
> user-agent: curl/7.77.0
> accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* Connection state changed (MAX_CONCURRENT_STREAMS == 250)!
< HTTP/2 502
< server: Caddy
< content-length: 0
< date: Mon, 30 Aug 2021 07:32:26 GMT
<
* Connection #0 to host xxx.udance.com.au left intact
I believe I need to use the not version of the matcher, but accessing wp-login.php from the browser shouldn’t trigger an abort.
After doing some reading on Referrer-Policy, I think everything is working as it should be. The default Referrer-Policy is strict-origin-when-cross-origin. This has been the case for about a year now (refer A new default Referrer-Policy for Chrome: strict-origin-when-cross-origin).
This MDN extract:
strict-origin-when-cross-origin(default)Send the origin, path, and querystring when performing a same-origin request. For cross-origin requests send the origin (only) when the protocol security level stays same (HTTPS→HTTPS). Don’t send the
Refererheader to less secure destinations (HTTPS→HTTP).
I think what’s happening is that I’m going from a frontend Caddy RP with TLS termination to an upstream Caddy webserver serving WP PHP files. This happens over HTTP. The Referer header isn’t being transmitted. This might explain why I’m not getting a match on either…
@noreferrer header_regexp Referer https?://xxx\.udance\.com\.au/(wp-comments-posts|wp-login)\.php$
abort @noreferrer
or…
@protected path_regexp (wp-comments-posts|wp-login)\.php$
handle @protected {
@no-referer header Referer {scheme}://{host}*
abort @no-referer
}
Does this explanation seem plausible? If so, what follows is that I is that I should withdraw any Caddy code submitted for the WP section [Deny Access to No Referrer Requests]
EDIT: Out of curiosity, if I enable mTLS, would Referer then be be transmitted to the upstream server?
Again, your access logs will show whether Caddy received the header. It should not be affected by going through another Caddy instance, since all headers are copied by reverse_proxy.
It might be a directive order issue. Try caddy adapt --pretty and see what order they’re in, to make sure some rewrite didn’t prevent the matcher from working.
As a proof of concept that the matcher is working:
Caddyfile:
{
admin off
debug
}
:8881 {
log
@protected path_regexp (wp-comments-posts|wp-login)\.php$
handle @protected {
@no-referer not header Referer {scheme}://{host}*
respond @no-referer "NOPE"
}
respond "YEP"
}
Trying it:
$ curl http://localhost:8881
YEP
$ curl http://localhost:8881/wp-login.php
NOPE
$ curl -H 'Referer: http://localhost:8881/wp-login.php' http://localhost:8881/wp-login.php
YEP
So basically, a plain request to not one of those two paths should return fine. A request to one of those two paths without a Referer header should fail. A request to one of those two paths with a Referer header should pass. (With abort, you’ll see curl: (52) Empty reply from server)
So if it’s not working for you, it must be something else in your config preventing it from getting hit, I guess.
Ooh. I’ve learnt something new!
I can confirm that it’s working too. I get the same result with this Caddyfile.
:80 {
log {
output file /var/log/caddy/access.log
}
root * /usr/local/www/wordpress
php_fastcgi 127.0.0.1:9000
file_server
@protected path_regexp (wp-comments-posts|wp-login)\.php$
handle @protected {
@no-referer not header Referer {scheme}://{host}*
respond @no-referer "NOPE"
}
respond "YEP"
}
An examination of the access log reveals the following though:
# curl http://localhost:80
YEP
Referer doesn’t appear in the access log.
{"level":"info","ts":1630379835.024139,"logger":"http.log.access.log0","msg":"handled request","request":{"remote_addr":"[::1]:28422","proto":"HTTP/1.1","method":"GET","host":"localhost","uri":"/","headers":{"User-Agent":["curl/7.77.0"],"Accept":["*/*"]}},"common_log":"::1 - - [31/Aug/2021:11:17:15 +0800] \"GET / HTTP/1.1\" 200 3","duration":0.000066548,"size":3,"status":200,"resp_headers":{"Content-Type":[],"Server":["Caddy"]}}
# curl http://localhost:80/wp-login.php
NOPE
Referer doesn’t appear in the access log.
{"level":"info","ts":1630380027.041773,"logger":"http.log.access.log0","msg":"handled request","request":{"remote_addr":"[::1]:15965","proto":"HTTP/1.1","method":"GET","host":"localhost","uri":"/wp-login.php","headers":{"User-Agent":["curl/7.77.0"],"Accept":["*/*"]}},"common_log":"::1 - - [31/Aug/2021:11:20:27 +0800] \"GET /wp-login.php HTTP/1.1\" 200 4","duration":0.000124746,"size":4,"status":200,"resp_headers":{"Server":["Caddy"],"Content-Type":[]}}
# curl -H 'Referer: http://localhost:80/wp-login.php' http://localhost:80/wp-login.php
YEP
Referer does appear in the access log.
{"level":"info","ts":1630380431.8426464,"logger":"http.log.access.log0","msg":"handled request","request":{"remote_addr":"[::1]:48939","proto":"HTTP/1.1","method":"GET","host":"localhost","uri":"/wp-login.php","headers":{"User-Agent":["curl/7.77.0"],"Accept":["*/*"],"Referer":["http://localhost:80/wp-login.php"]}},"common_log":"::1 - - [31/Aug/2021:11:27:11 +0800] \"GET /wp-login.php HTTP/1.1\" 200 3","duration":0.000151882,"size":3,"status":200,"resp_headers":{"Server":["Caddy"],"Content-Type":[]}}
The thing is, from a browser, I get the same result rather than the opposite, which I understand for the above statement is not expected.
Something in php_fastcgi maybe?
root@wp-xxx:/usr/local/www # caddy adapt --pretty
2021/08/31 03:37:44.119 INFO using adjacent Caddyfile
{
"logging": {
"logs": {
"default": {
"level": "DEBUG"
},
"log0": {
"writer": {
"filename": "/var/log/caddy/access.log",
"output": "file"
},
"level": "DEBUG",
"include": [
"http.log.access.log0"
]
}
}
},
"apps": {
"http": {
"servers": {
"srv0": {
"listen": [
":80"
],
"routes": [
{
"handle": [
{
"handler": "vars",
"root": "/usr/local/www/wordpress"
}
]
},
{
"match": [
{
"path_regexp": {
"pattern": "(wp-comments-posts|wp-login)\\.php$"
}
}
],
"handle": [
{
"handler": "subroute",
"routes": [
{
"handle": [
{
"body": "NOPE",
"handler": "static_response"
}
],
"match": [
{
"not": [
{
"header": {
"Referer": [
"{http.request.scheme}://{http.request.host}*"
]
}
}
]
}
]
}
]
}
]
},
{
"handle": [
{
"body": "YEP",
"handler": "static_response"
}
]
},
{
"match": [
{
"file": {
"try_files": [
"{http.request.uri.path}/index.php"
]
},
"not": [
{
"path": [
"*/"
]
}
]
}
],
"handle": [
{
"handler": "static_response",
"headers": {
"Location": [
"{http.request.uri.path}/"
]
},
"status_code": 308
}
]
},
{
"match": [
{
"file": {
"try_files": [
"{http.request.uri.path}",
"{http.request.uri.path}/index.php",
"index.php"
],
"split_path": [
".php"
]
}
}
],
"handle": [
{
"handler": "rewrite",
"uri": "{http.matchers.file.relative}"
}
]
},
{
"match": [
{
"path": [
"*.php"
]
}
],
"handle": [
{
"handler": "reverse_proxy",
"transport": {
"protocol": "fastcgi",
"split_path": [
".php"
]
},
"upstreams": [
{
"dial": "127.0.0.1:9000"
}
]
}
]
},
{
"handle": [
{
"handler": "file_server",
"hide": [
"./Caddyfile"
]
}
]
}
],
"logs": {
"default_logger_name": "log0"
}
}
}
}
}
}
Well, remove respond "Yep" if you have file_server etc, because otherwise it’ll just take precedence over those. php_fastcgi won’t even get reached.
Actually, that’s exactly as expected. If you directly load wp-login.php without having gotten there from a page navigation like clicking a link, then you should get rejected, because in that case Referer wouldn’t be there.
But to be honest, this entire approach to “hardening” is really stupid. It’s very much a hack, and in no way reliable. As you can see, some bot/bad actor could just add the Referer header to get through. It’s basically Security through obscurity - Wikipedia. This doesn’t really solve any problem.
I go to the main site and click the link shown…
I should get the login screen. Instead, I get rejected, which seems to suggest that Referer hasn’t been passed along from the page navigation.
However, examining the access log, I do see Referer in the first line.
{"level":"info","ts":1630384905.2133794,"logger":"http.log.access.log0","msg":"handled request","request":{"remote_addr":"10.1.1.4:50592","proto":"HTTP/1.1","method":"GET","host":"xxx.udance.com.au","uri":"/wp-login.php","headers":{"Sec-Fetch-Site":["same-origin"],"X-Forwarded-For":["10.1.1.222"],"Accept-Language":["en-US,en;q=0.9"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.159 Safari/537.36 Edg/92.0.902.84"],"Referer":["https://xxx.udance.com.au/"],"Sec-Fetch-Dest":["document"],"Sec-Fetch-Mode":["navigate"],"Sec-Fetch-User":["?1"],"Upgrade-Insecure-Requests":["1"],"X-Forwarded-Proto":["https"],"Accept-Encoding":["gzip, deflate, br"],"Sec-Ch-Ua":["\"Chromium\";v=\"92\", \" Not A;Brand\";v=\"99\", \"Microsoft Edge\";v=\"92\""],"Sec-Ch-Ua-Mobile":["?0"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"]}},"common_log":"10.1.1.4 - - [31/Aug/2021:12:41:45 +0800] \"GET /wp-login.php HTTP/1.1\" 200 4","duration":0.000106105,"size":4,"status":200,"resp_headers":{"Server":["Caddy"],"Content-Type":[]}}
{"level":"info","ts":1630384905.4891205,"logger":"http.log.access.log0","msg":"handled request","request":{"remote_addr":"10.1.1.4:50592","proto":"HTTP/1.1","method":"GET","host":"xxx.udance.com.au","uri":"/favicon.ico","headers":{"X-Forwarded-For":["10.1.1.222"],"X-Forwarded-Proto":["https"],"Accept":["image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8"],"Accept-Language":["en-US,en;q=0.9"],"Sec-Ch-Ua":["\"Chromium\";v=\"92\", \" Not A;Brand\";v=\"99\", \"Microsoft Edge\";v=\"92\""],"Sec-Fetch-Dest":["image"],"Sec-Fetch-Site":["same-origin"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.159 Safari/537.36 Edg/92.0.902.84"],"Accept-Encoding":["gzip, deflate, br"],"Referer":["https://xxx.udance.com.au/wp-login.php"],"Sec-Ch-Ua-Mobile":["?0"],"Sec-Fetch-Mode":["no-cors"]}},"common_log":"10.1.1.4 - - [31/Aug/2021:12:41:45 +0800] \"GET /favicon.ico HTTP/1.1\" 302 0","duration":0.246683975,"size":0,"status":302,"resp_headers":{"Server":["Caddy"],"X-Redirect-By":["WordPress"],"Location":["https://xxx.udance.com.au/wp-includes/images/w-logo-blue-white-bg.png"],"Status":["302 Found"],"X-Powered-By":["PHP/7.4.21"],"Content-Type":["text/html; charset=UTF-8"],"Link":["<https://xxx.udance.com.au/wp-json/>; rel=\"https://api.w.org/\""]}}
Something odd is happening around Referer, but I can’t put my finger on it. It seems pretty random whether it appears in the access log or not. Furthermore, the behaviour of the matcher doesn’t seem to correlate with what I’m seeing in the access log. It’s confusing.
Extract from the process log after clicking the link from the main page.
{"level":"info","ts":1630386207.4195123,"logger":"http.log.access.log0","msg":"handled request","request":{"remote_addr":"10.1.1.4:65082","proto":"HTTP/1.1","method":"GET","host":"xxx.udance.com.au","uri":"/wp-login.php","headers":{"Accept-Encoding":["gzip, deflate, br"],"Sec-Ch-Ua":["\"Chromium\";v=\"92\", \" Not A;Brand\";v=\"99\", \"Microsoft Edge\";v=\"92\""],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.159 Safari/537.36 Edg/92.0.902.84"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"],"Referer":["https://xxx.udance.com.au/"],"Sec-Fetch-Site":["same-origin"],"X-Forwarded-For":["10.1.1.222"],"X-Forwarded-Proto":["https"],"Accept-Language":["en-US,en;q=0.9"],"Sec-Fetch-Mode":["navigate"],"Sec-Fetch-User":["?1"],"Upgrade-Insecure-Requests":["1"],"Sec-Ch-Ua-Mobile":["?0"],"Sec-Fetch-Dest":["document"]}},"common_log":"10.1.1.4 - - [31/Aug/2021:13:03:27 +0800] \"GET /wp-login.php HTTP/1.1\" 200 4","duration":0.000105318,"size":4,"status":200,"resp_headers":{"Server":["Caddy"],"Content-Type":[]}}
{"level":"debug","ts":1630386207.4482172,"logger":"http.handlers.rewrite","msg":"rewrote request","request":{"remote_addr":"10.1.1.4:65082","proto":"HTTP/1.1","method":"GET","host":"xxx.udance.com.au","uri":"/favicon.ico","headers":{"X-Forwarded-For":["10.1.1.222"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.159 Safari/537.36 Edg/92.0.902.84"],"Accept-Encoding":["gzip, deflate, br"],"Referer":["https://xxx.udance.com.au/wp-login.php"],"Sec-Fetch-Dest":["image"],"Sec-Fetch-Site":["same-origin"],"Sec-Fetch-Mode":["no-cors"],"X-Forwarded-Proto":["https"],"Accept":["image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8"],"Accept-Language":["en-US,en;q=0.9"],"Sec-Ch-Ua":["\"Chromium\";v=\"92\", \" Not A;Brand\";v=\"99\", \"Microsoft Edge\";v=\"92\""],"Sec-Ch-Ua-Mobile":["?0"]}},"method":"GET","uri":"index.php"}
{"level":"debug","ts":1630386207.448344,"logger":"http.reverse_proxy.transport.fastcgi","msg":"roundtrip","request":{"remote_addr":"10.1.1.4:65082","proto":"HTTP/1.1","method":"GET","host":"xxx.udance.com.au","uri":"index.php","headers":{"Sec-Fetch-Site":["same-origin"],"X-Forwarded-For":["10.1.1.222, 10.1.1.4"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.159 Safari/537.36 Edg/92.0.902.84"],"Accept-Encoding":["gzip, deflate, br"],"Referer":["https://xxx.udance.com.au/wp-login.php"],"Sec-Fetch-Dest":["image"],"Sec-Ch-Ua-Mobile":["?0"],"Sec-Fetch-Mode":["no-cors"],"X-Forwarded-Proto":["https"],"Accept":["image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8"],"Accept-Language":["en-US,en;q=0.9"],"Sec-Ch-Ua":["\"Chromium\";v=\"92\", \" Not A;Brand\";v=\"99\", \"Microsoft Edge\";v=\"92\""]}},"dial":"127.0.0.1:9000","env":{"AUTH_TYPE":"","CONTENT_LENGTH":"","CONTENT_TYPE":"","DOCUMENT_ROOT":"/usr/local/www/wordpress","DOCUMENT_URI":"index.php","GATEWAY_INTERFACE":"CGI/1.1","HTTP_ACCEPT":"image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8","HTTP_ACCEPT_ENCODING":"gzip, deflate, br","HTTP_ACCEPT_LANGUAGE":"en-US,en;q=0.9","HTTP_HOST":"xxx.udance.com.au","HTTP_REFERER":"https://xxx.udance.com.au/wp-login.php","HTTP_SEC_CH_UA":"\"Chromium\";v=\"92\", \" Not A;Brand\";v=\"99\", \"Microsoft Edge\";v=\"92\"","HTTP_SEC_CH_UA_MOBILE":"?0","HTTP_SEC_FETCH_DEST":"image","HTTP_SEC_FETCH_MODE":"no-cors","HTTP_SEC_FETCH_SITE":"same-origin","HTTP_USER_AGENT":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.159 Safari/537.36 Edg/92.0.902.84","HTTP_X_FORWARDED_FOR":"10.1.1.222, 10.1.1.4","HTTP_X_FORWARDED_PROTO":"https","PATH_INFO":"","QUERY_STRING":"","REMOTE_ADDR":"10.1.1.4","REMOTE_HOST":"10.1.1.4","REMOTE_IDENT":"","REMOTE_PORT":"65082","REMOTE_USER":"","REQUEST_METHOD":"GET","REQUEST_SCHEME":"http","REQUEST_URI":"/favicon.ico","SCRIPT_FILENAME":"/usr/local/www/wordpress/index.php","SCRIPT_NAME":"/index.php","SERVER_NAME":"xxx.udance.com.au","SERVER_PROTOCOL":"HTTP/1.1","SERVER_SOFTWARE":"Caddy/v2.4.3"}}
{"level":"debug","ts":1630386207.7043395,"logger":"http.handlers.reverse_proxy","msg":"upstream roundtrip","upstream":"127.0.0.1:9000","request":{"remote_addr":"10.1.1.4:65082","proto":"HTTP/1.1","method":"GET","host":"xxx.udance.com.au","uri":"index.php","headers":{"Sec-Fetch-Mode":["no-cors"],"X-Forwarded-Proto":["https"],"Accept":["image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8"],"Accept-Language":["en-US,en;q=0.9"],"Sec-Ch-Ua":["\"Chromium\";v=\"92\", \" Not A;Brand\";v=\"99\", \"Microsoft Edge\";v=\"92\""],"Sec-Ch-Ua-Mobile":["?0"],"X-Forwarded-For":["10.1.1.222, 10.1.1.4"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.159 Safari/537.36 Edg/92.0.902.84"],"Accept-Encoding":["gzip, deflate, br"],"Referer":["https://xxx.udance.com.au/wp-login.php"],"Sec-Fetch-Dest":["image"],"Sec-Fetch-Site":["same-origin"]}},"headers":{"X-Powered-By":["PHP/7.4.21"],"Content-Type":["text/html; charset=UTF-8"],"Link":["<https://xxx.udance.com.au/wp-json/>; rel=\"https://api.w.org/\""],"X-Redirect-By":["WordPress"],"Location":["https://xxx.udance.com.au/wp-includes/images/w-logo-blue-white-bg.png"],"Status":["302 Found"]},"status":302}
{"level":"info","ts":1630386207.704579,"logger":"http.log.access.log0","msg":"handled request","request":{"remote_addr":"10.1.1.4:65082","proto":"HTTP/1.1","method":"GET","host":"xxx.udance.com.au","uri":"/favicon.ico","headers":{"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.159 Safari/537.36 Edg/92.0.902.84"],"Accept-Encoding":["gzip, deflate, br"],"Referer":["https://xxx.udance.com.au/wp-login.php"],"Sec-Fetch-Dest":["image"],"Sec-Fetch-Site":["same-origin"],"X-Forwarded-For":["10.1.1.222"],"Accept":["image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8"],"Accept-Language":["en-US,en;q=0.9"],"Sec-Ch-Ua":["\"Chromium\";v=\"92\", \" Not A;Brand\";v=\"99\", \"Microsoft Edge\";v=\"92\""],"Sec-Ch-Ua-Mobile":["?0"],"Sec-Fetch-Mode":["no-cors"],"X-Forwarded-Proto":["https"]}},"common_log":"10.1.1.4 - - [31/Aug/2021:13:03:27 +0800] \"GET /favicon.ico HTTP/1.1\" 302 0","duration":0.256450471,"size":0,"status":302,"resp_headers":{"X-Powered-By":["PHP/7.4.21"],"Content-Type":["text/html; charset=UTF-8"],"Link":["<https://xxx.udance.com.au/wp-json/>; rel=\"https://api.w.org/\""],"X-Redirect-By":["WordPress"],"Location":["https://xxx.udance.com.au/wp-includes/images/w-logo-blue-white-bg.png"],"Server":["Caddy"],"Status":["302 Found"]}}
It’s literally whether your browser arrived to that page via a page transition or not. That’s it. If you refresh in your browser, then it will not have that header, because the refresh is a fresh page load without the page transition.
Looking at the Apache config for this, looks like they only block POST requests, and not GET requests to that page. Doesn’t seem like the nginx makes that distinction, for some reason. So you could add a method POST matcher in there.
Also you could respond with respond "NOPE: {scheme}://{host}*" just to see what the placeholders spit out, but I’m pretty sure that should be correct.
refresh = clear cache?
If I go straight to wp-login.php without clicking a site link first:
If I clear the browser cache and then navigate to the page from a site link, I get the same result.
I’m assuming this is correct:
{
debug
}
:80 {
log {
output file /var/log/caddy/access.log
}
root * /usr/local/www/wordpress
php_fastcgi 127.0.0.1:9000
file_server
@protected path_regexp (wp-comments-posts|wp-login)\.php$
handle @protected {
@no-referer {
not header Referer {scheme}://{host}*
method POST
}
respond @no-referer "NOPE: {scheme}://{host}*"
}
# respond "YEP"
}
This time I get to the login page whether or not I navigate from a site link.
I don’t fully comprehend the purpose of this. A quick search suggests GET is idempotent (same result each time) What is the difference between POST and GET? [duplicate], while PUT is not. Is that the reason for adding this in?
Oh interesting, well that’s the issue. The scheme doesn’t match. http but the header is https. Because you have that proxy hop in between which is HTTP between them. So uh, I guess don’t use {scheme} and just hard-code https, or use {header.X-Forwarded-Proto} lol but might as well just hardcode it.
The GET request will be the actual page load where you receive the HTML in the browser. The POST is the actual form submit action on that page when you click “Log in”, sending the user/pass to the app. The “important part” is to prevent bots from POSTing, because that’s the “heavy” operation, the part that actually makes the app do password verification. Blocking the GET is not really useful because then it will just block legitimate users from loading the login page if they directly go to it. The important part is to block POST requests from clients who were not actually on the login page when sending it (e.g. bots… but again, in no way fool-proof)
There have been so many diversions, I’ve kinda lost my way 
Let me see if I can get the story straight.
Let’s start with this Caddyfile.
:80 {
log {
output file /var/log/caddy/access.log
}
root * /usr/local/www/wordpress
php_fastcgi 127.0.0.1:9000
file_server
@protected path_regexp (wp-comments-posts|wp-login)\.php$
handle @protected {
@no-referer {
not header Referer https://{host}*
method POST
}
abort @no-referer
}
}
Notes:
https://xxx.udance.com.au*), I temporarily remove the not from the matcher. Questions:
header, I can set, add and delete headers. Rather than relying on logs, Is there a way to display headers using respond?method POST, the result is the same. As long as the Referer originates from the site, does it matter if method is POST or GET? Another way of looking at this is…wouldn’t you also want to block GET requests from bots? Maybe this is why the NginX example didn’t make that distinction.{scheme} have worked?Assuming method POST is redundant, this is the replacement text I’m proposing for the WP support doc to deny access to no referrer requests: It should work whether or not the WP site is behind a RP. Please review.
For Caddy, deny access to logins and comments if an off-site
Refererattempts to submit content.
# Stop spam attack logins and comments
@protected path_regexp (wp-comments-posts|wp-login)\.php$
handle @protected {
@no-referer {
not header Referer https://{host}*
method POST
}
abort @no-referer
}
Using
abortfor blocking bots is more efficient because Caddy will just drop the connection immediately instead of sending back a response.
The text above will replace what was previously submitted:
For Caddy, use the
header_regexprequest matcher to deny access to no referrer requests.
# Stop spam attack logins and comments
@noreferrer not header_regexp Referer https?://example\.com/(wp-comments-posts|wp-login)\.php$
abort @noreferrer
Using
abortfor blocking bots is more efficient because Caddy will just drop the connection immediately instead of sending back a response.
Yes, using the placeholder {header.<field>}
It’s expected that you never get blocked in the browser. The point of this is to block bots but not to block legitimate browser usage.
It absolutely does matter if the it’s a GET or POST, because GETs are requests to load the HTML of the page, and POSTs are the form submissions.
It’s not useful to block GETs because then it would block legitimate users trying to access the login page directly or refreshing once they land on the page.
The part that this is intended to block is bots brute forcing the login form by making repeated requests over and over. There’s lots of bots just sending admin/admin and other common user/pass combinations to /wp-login.php on all kinds of websites until they get a successful login response so they can hijack your site.
(Like I’ve said before, bots would be stupid to not set the Referer header so I really doubt this would have any actual benefit, more important to just ensure strong password usage, consider using 2FA for WordPress logins since bots will never be able to brute force that)
A better test case is this:
# Should fail (I seem to get a 502 from Cloudflare, probably because "abort")
$ curl -v -d "log=admin&pwd=admin&wp-submit=Log+In" https://xxx.udance.com.au/wp-login.php
# Should get through due to Referer header
$ curl -v -d "log=admin&pwd=admin&wp-submit=Log+In" -H "Referer: https://xxx.udance.com.au" https://xxx.udance.com.au/wp-login.php
# Should also get through (GET request with no Referer, is OK)
$ curl -v https://xxx.udance.com.au/wp-login.php
And all requests in the browser should work fine without getting blocked.
To explain, -d is “data”, basically asks curl to send a POST request with this data. It looks like a query string, but it gets sent as Content-Type: application/x-www-form-urlencoded which is how HTML forms are submitted.
Yeah, it should. It’s just that to the perspective backend Caddy, the request that came in was over HTTP so that’s working “as intended”, it’s just that what we were trying to use it for required the “real” scheme coming from the client – which is in the X-Forwarded-Proto header, but it’s not worth faffing with that in this situation, because if you’re using Caddy, it’s like 99.9% likely you’re using HTTPS for the actual client requests anyways.
Now, this could be very useful for testing 
Would it also be correct to say that the intention is to block bad bots that POST, but allow good bots that GET e.g. search engine bots.
Is post #54 okay to go?
I guess, but no search engine bots care about your login page. They care about content.
I would say “Mitigate” rather than “Stop” because it really doesn’t stop anything reliably.
I need help with this sentence as well. Can you think of something better?
Maybe…
For Caddy, deny access to logins and comments if a bot attempts to submit content.
“For Caddy, mitigate brute force login and spam comment attempts from bots.”
I guess
Revised update:
For Caddy, mitigate brute force login and spam comment attempts from bots.
# Mitigate spam attack logins and comments
@protected path_regexp (wp-comments-posts|wp-login)\.php$
handle @protected {
@no-referer {
not header Referer https://{host}*
method POST
}
abort @no-referer
}
Using
abortfor blocking bots is more efficient because Caddy will just drop the connection immediately instead of sending back a response.