1. The problem I’m having:
I am running Caddy (v2.7.6) server as a proxy with automatic TLS, that proxies requests to a remote backend that serves a php application. I am also running an ask-server in order to mitigate abuse of auto-tls feature by rejecting tls-handshake for unknown hosts. Ask-server (simple php script) is configured on the same machine using same caddy server (makes Caddy very elegant).
This setup works great and smooth, with one minor issue (I believe we are missing some understating on our side). By analyzing access log of ask server - we do see a lot of bogus requests (e.g. host scanners and other) which can produce a severe load on our ask-server (we’ve also experimented with remote ask-server running on another server) so what we wanted to do is to block these requests on a lower level using os’ firewall based on request client ip addresses/request headers.
However, we are unable to obtain forwarded client_ip/remote_ip values from the access log of ask server. All we see is “::1” values in both, which makes sense since ask requests are originated by Caddy running on same machine.
What we tried to in attempt to solve this:
- Assuming (wrongly?) that caddy is somehow proxying request we’ve tried adding global trusted_proxies directive, but with no success:
servers {
trusted_proxies static ::1 "::1" 127.0.0.1
}
*EDIT: By skimming through a Caddy sources at it seems a false assumption. Caddy just builds a new request to ask server using ask-server url from a config and a host name to verify being passed as a query parameter.
2. The only way we’re able to obtain these values is by setting default logger’s level to debug:
log default {
output file /var/log/caddy/default.log
level debug
}
In my opinion this is not very convenient, since using this log level produces a really huge file size; we’re also unable to access some other useful info (which is present in access logs) as e.g. request headers (it can be useful in analyzing and deciding which ip address should be blocked).
2. Error messages and/or full log output:
Example of ask-server log:
{"level":"error","ts":1706102369.5341675,"logger":"http.log.access.log2","msg":"handled request","request":{"remote_ip":"::1","remote_port":"45282","client_ip":"::1","proto":"HTTP/1.1","method":"GET","host":"localhost:8080","uri":"/?domain=www.login.cpbpmyadmin.holidayheroes-de-len-v2-stage.travelyo-stage.stagewycrm.X","headers":{"User-Agent":["Go-http-client/1.1"],"Accept-Encoding":["gzip"]}},"bytes_read":0,"user_id":"","duration":0.002276843,"size":46,"status":400,"resp_headers":{"Server":["Caddy"],"Content-Type":["application/json; charset=utf-8"],"Status":["400 Bad Request"],"X-Powered-By":["PHP/7.3.33"]}}
Example of default log with debug level:
{"level":"debug","ts":1706099141.5186276,"logger":"tls","msg":"response from ask endpoint","client_ip":"59.110.175.210","domain":"www.phpmyadmin.admin.ns.secure.support.sitemap.cms.autodiscover.remote.api.speedycrm.X","url":"http://localhost:8080/?domain=www.phpmyadmin.admin.ns.secure.support.sitemap.cms.autodiscover.remote.api.speedycrm.X","status":400}
Note: private host was substituted with “X”
3. Caddy version:
v2.7.6 h1:w0NymbG2m9PcvKWsrXO6EEkY9Ru4FJK8uQbYcev1p3A=
4. How I installed and ran Caddy:
a. System environment:
NAME=“CentOS Linux”
VERSION=“7 (Core)”
ID=“centos”
ID_LIKE=“rhel fedora”
VERSION_ID=“7”
PRETTY_NAME=“CentOS Linux 7 (Core)”
ANSI_COLOR=“0;31”
CPE_NAME=“cpe:/o:centos:centos:7”
HOME_URL=“https://www.centos.org/”
BUG_REPORT_URL=“https://bugs.centos.org/”
CENTOS_MANTISBT_PROJECT=“CentOS-7”
CENTOS_MANTISBT_PROJECT_VERSION=“7”
REDHAT_SUPPORT_PRODUCT=“centos”
REDHAT_SUPPORT_PRODUCT_VERSION=“7”
d. My complete Caddy config:
{
log default {
output file /var/log/caddy/default.log
level debug
}
on_demand_tls {
ask http://localhost:8080/
}
servers {
trusted_proxies static ::1 "::1" 127.0.0.1
}
}
:443 {
tls admin@org.com {
on_demand
}
log {
output file /var/log/caddy/access.log {
roll_size 10mb
roll_keep 20
roll_keep_for 720h
}
}
reverse_proxy https://x.x.x.x:443 {
header_up Host {host}
header_up X-Real-IP {remote_host}
transport http {
tls
tls_insecure_skip_verify
compression off
}
}
}
http://localhost:8080 {
log {
output file /var/log/caddy/mc-caddy-ask-server-access.log {
roll_size 10mb
roll_keep 20
roll_keep_for 720h
}
}
root * /var/www/mc-caddy-ask-server/public
php_fastcgi tcp/localhost:9000 {
trusted_proxies ::1
}
}