1. The problem I’m having:
I have a requirement to serve static files (using file_server) which contain special characters like / and : - for example https://example.com/pie. The implementation follows the SAML Metadata Query semantics (see: Metadata Query Protocol - InCommon Metadata Service Wiki - Internet2 Wiki)
This is currently working under Apache httpd by having files on the filesystem named using urlencoded versions (eg. https:%2F%2Fexample.com%2Fpie) and AllowEncodedSlashes NoDecode defined in the Apache config.
I’m trying to port this to Caddy v2 (using caddy:alpine docker container running v2.7.6) but I only get 404s when requesting the same URI as was being served successfully by Apache.
Both of these files exist on the filesystem:
# ls -il ./pub/mdq/entities/*example.com*
306692 -rw-r--r-- 4 root root 5 Jan 18 12:09 ./pub/mdq/entities/https%3A%2F%2Fexample.com%2Fpie
306692 -rw-r--r-- 4 root root 5 Jan 18 12:09 ./pub/mdq/entities/https%3a%2f%2fexample.com%2fpie
306692 -rw-r--r-- 4 root root 5 Jan 18 12:09 ./pub/mdq/entities/https:%2F%2Fexample.com%2Fpie
306692 -rw-r--r-- 4 root root 5 Jan 18 12:09 ./pub/mdq/entities/https:%2f%2fexample.com%2fpie
The debug output suggests that Caddy is urldecoding %2a into / and then expecting that to be in the filesystem path. What I need is to be able to pass through the %2a into the actual filename on disk.
2. Error messages and/or full log output:
$ curl -vs -H 'Host: example' http://hal.lan:8082/entities/https:%2f%2fexample.com%2fpie
* Trying 192.168.37.22:8082...
* Connected to hal.lan (192.168.37.22) port 8082
> GET /entities/https:%2f%2fexample.com%2fpie HTTP/1.1
> Host: example
> User-Agent: curl/8.4.0
> Accept: */*
>
< HTTP/1.1 404 Not Found
< Content-Type: application/samlmetadata+xml
< Server: Caddy
< Date: Thu, 18 Jan 2024 12:18:31 GMT
< Content-Length: 0
<
* Connection #0 to host hal.lan left intact
{"level":"info","ts":1705580627.7879965,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":"caddyfile"}
{"level":"warn","ts":1705580627.8091376,"msg":"Caddyfile input is not formatted; run 'caddy fmt --overwrite' to fix inconsistencies","adapter":"caddyfile","file":"/etc/caddy/Caddyfile","line":2}
{"level":"info","ts":1705580627.8116891,"msg":"redirected default logger","from":"stderr","to":"stdout"}
{"level":"info","ts":1705580627.8151548,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
{"level":"warn","ts":1705580627.8173888,"logger":"http.auto_https","msg":"automatic HTTPS is completely disabled for server","server_name":"srv0"}
{"level":"info","ts":1705580627.8174314,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc0005a3200"}
{"level":"debug","ts":1705580627.8174934,"logger":"http.auto_https","msg":"adjusted config","tls":{"automation":{"policies":[{}]}},"http":{"servers":{"srv0":{"listen":[":80"],"routes":[{"handle":[{"handler":"subroute","routes":[{"handle":[{"handler":"vars","root":"/pub/mdq"}]},{"handle":[{"handler":"headers","response":{"set":{"Content-Type":["application/samlmetadata+xml"]}}}],"match":[{"path":["/entities/*"]}]},{"handle":[{"canonical_uris":false,"handler":"file_server","hide":["/etc/caddy/Caddyfile"],"precompressed":{"gzip":{}},"precompressed_order":["gzip"]}]}]}],"terminal":true}],"automatic_https":{"disable":true},"logs":{"logger_names":{"example":"log0"}}}}}}
{"level":"debug","ts":1705580627.8233988,"logger":"http","msg":"starting server loop","address":"[::]:80","tls":false,"http3":false}
{"level":"info","ts":1705580627.8235157,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2","h3"]}
{"level":"info","ts":1705580627.824527,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
{"level":"info","ts":1705580627.8276577,"msg":"serving initial configuration"}
{"level":"warn","ts":1705580627.8347867,"logger":"tls","msg":"storage cleaning happened too recently; skipping for now","storage":"FileStorage:/data/caddy","instance":"4979a702-4f27-4e4b-8742-69a09721d8b0","try_again":1705667027.8347764,"try_again_in":86399.999997861}
{"level":"info","ts":1705580627.8350356,"logger":"tls","msg":"finished cleaning storage units"}
{"level":"debug","ts":1705580634.837426,"logger":"http.handlers.file_server","msg":"sanitized path join","site_root":"/pub/mdq","request_path":"/entities/https://example.com/pie","result":"/pub/mdq/entities/https:/example.com/pie"}
{"level":"debug","ts":1705580634.8377106,"logger":"http.log.error.log0","msg":"{id=z8ax53q2w} fileserver.(*FileServer).notFound (staticfiles.go:629): HTTP 404","request":{"remote_ip":"192.168.37.221","remote_port":"53028","client_ip":"192.168.37.221","proto":"HTTP/1.1","method":"GET","host":"example","uri":"/entities/https:%2f%2fexample.com%2fpie","headers":{"Accept":["*/*"],"User-Agent":["curl/8.4.0"]}},"duration":0.000516803,"status":404,"err_id":"z8ax53q2w","err_trace":"fileserver.(*FileServer).notFound (staticfiles.go:629)"}
{"level":"error","ts":1705580634.8378222,"logger":"http.log.access.log0","msg":"handled request","request":{"remote_ip":"192.168.37.221","remote_port":"53028","client_ip":"192.168.37.221","proto":"HTTP/1.1","method":"GET","host":"example","uri":"/entities/https:%2f%2fexample.com%2fpie","headers":{"User-Agent":["curl/8.4.0"],"Accept":["*/*"]}},"bytes_read":0,"user_id":"","duration":0.000516803,"size":0,"status":404,"resp_headers":{"Server":["Caddy"],"Content-Type":["application/samlmetadata+xml"]}}
3. Caddy version:
# docker compose exec mps caddy version
v2.7.6 h1:w0NymbG2m9PcvKWsrXO6EEkY9Ru4FJK8uQbYcev1p3A=
4. How I installed and ran Caddy:
a. System environment:
- Docker 24.0.7 running under Alpine Linux v3.19
caddy:alpineimage reporting asv2.7.6
# uname -a
Linux hal 6.6.7-0-virt #1-Alpine SMP PREEMPT_DYNAMIC Thu, 14 Dec 2023 08:49:17 +0000 x86_64 GNU/Linux
b. Command:
docker compose up -d
c. Service/unit/compose file:
Docker Compose file
version: '3.6'
services:
mps:
image: caddy:alpine
ports:
- 8082:80
volumes:
- ./pub:/pub:ro
- ./Caddyfile:/etc/caddy/Caddyfile:ro
d. My complete Caddy config:
{
auto_https off
log default {
output stdout
}
debug
}
(common) {
log {
output stdout
}
file_server {
precompressed gzip
disable_canonical_uris
}
}
example:80 {
root * /pub/mdq
header /entities/* Content-Type application/samlmetadata+xml
import common
}
5. Links to relevant resources:
https://httpd.apache.org/docs/2.4/mod/core.html#allowencodedslashes
https://spaces.at.internet2.edu/display/MDQ/Metadata+Query+Protocol
Thanks!
