Error 404 when accessing a Url containing Unicode characters

1. The problem I’m having:

Hosted a static website on a VDS server based on vuepress2. The site is multilingual and some image files have Unicode characters in the address

public/assets/news/cover/талисман/0-microsoftteams-image_3_.png

Accordingly, when assembling the site, the links are converted to:

https://myname-site/assets/news/cover/%D1%82%D0%B0%D0%BB%D0%B8%D1%81%D0%BC%D0%B0%D0%BD/0-microsoftteams-image_3_.png

When placing the test version on shared hosting behind Nginx, there are no problems with loading such files.
On the production server, I decided to try caddy v2 for the first time. And I discovered that all files containing Unicode characters are not loaded.

2. Error messages and/or full log output:

This is the log immediately after enabling the debug option. And two attempts to access the file through the browser. And one try via curl

{"level":"info","ts":"2024/02/22 18:57:38","name":"admin.api","msg":"received request","method":"POST","host":"localhost:2019","uri":"/load","remote_ip":"127.0.0.1","remote_port":"54934","headers":{"Accept-Encoding":["gzip"],"Cache-Control":["must-revalidate"],"Content-Length":["906"],"Content-Type":["application/json"],"Origin":["http://localhost:2019"],"User-Agent":["Go-http-client/1.1"]}}
{"level":"info","ts":"2024/02/22 18:57:38","name":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
{"level":"info","ts":"2024/02/22 18:57:38","name":"http.auto_https","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443}
{"level":"info","ts":"2024/02/22 18:57:38","name":"http.auto_https","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
{"level":"debug","ts":"2024/02/22 18:57:38","name":"http.auto_https","msg":"adjusted config","tls":{"automation":{"policies":[{}]}},"http":{"servers":{"remaining_auto_https_redirects":{"listen":[":80"],"routes":[{},{}],"logs":{"logger_names":{"name-site":"log0"}}},"srv0":{"listen":[":443"],"routes":[{"handle":[{"handler":"subroute","routes":[{"handle":[{"handler":"vars","root":"/var/www/site"}]},{"handle":[{"handler":"reverse_proxy","upstreams":[{"dial":"localhost:8765"}]}],"match":[{"path":["/api/*"]}]},{"handle":[{"handler":"file_server","hide":["/etc/caddy/Caddyfile"]}]}]}],"terminal":true}],"tls_connection_policies":[{}],"automatic_https":{},"logs":{"logger_names":{"name-site":"log0"}}}}}}
{"level":"debug","ts":"2024/02/22 18:57:38","name":"http","msg":"starting server loop","address":"[::]:80","tls":false,"http3":false}
{"level":"info","ts":"2024/02/22 18:57:38","name":"http.log","msg":"server running","name":"remaining_auto_https_redirects","protocols":["h1","h2","h3"]}
{"level":"info","ts":"2024/02/22 18:57:38","name":"http","msg":"enabling HTTP/3 listener","addr":":443"}
{"level":"debug","ts":"2024/02/22 18:57:38","name":"http","msg":"starting server loop","address":"[::]:443","tls":true,"http3":true}
{"level":"info","ts":"2024/02/22 18:57:38","name":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2","h3"]}
{"level":"info","ts":"2024/02/22 18:57:38","name":"http","msg":"enabling automatic TLS certificate management","domains":["name-site"]}
{"level":"info","ts":"2024/02/22 18:57:38","name":"http","msg":"servers shutting down with eternal grace period"}
{"level":"info","ts":"2024/02/22 18:57:38","msg":"autosaved config (load with --resume flag)","file":"/var/lib/caddy/.config/caddy/autosave.json"}
{"level":"info","ts":"2024/02/22 18:57:38","name":"admin.api","msg":"load complete"}
{"level":"info","ts":"2024/02/22 18:57:38","name":"admin","msg":"stopped previous server","address":"localhost:2019"}
{"level":"info","ts":"2024/02/22 18:57:38","name":"tls","msg":"cleaning storage unit","storage":"FileStorage:/var/lib/caddy/.local/share/caddy"}
{"level":"info","ts":"2024/02/22 18:57:38","name":"tls","msg":"finished cleaning storage units"}
{"level":"debug","ts":"2024/02/22 18:58:11","name":"http.handlers.file_server","msg":"sanitized path join","site_root":"/var/www/site","request_path":"/assets/news/cover/талисман/0-microsoftteams-image_3_.png","result":"/var/www/site/assets/news/cover/талисман/0-microsoftteams-image_3_.png"}
{"level":"debug","ts":"2024/02/22 18:58:11","name":"http.log.error.log0","msg":"{id=e990rk2mh} fileserver.(*FileServer).notFound (staticfiles.go:629): HTTP 404","request":{"remote_ip":"000.000.00.00","remote_port":"64300","client_ip":"000.000.00.00","proto":"HTTP/3.0","method":"GET","host":"name-site","uri":"/assets/news/cover/%D1%82%D0%B0%D0%BB%D0%B8%D1%81%D0%BC%D0%B0%D0%BD/0-microsoftteams-image_3_.png","headers":{"Sec-Ch-Ua":["\"Not A(Brand\";v=\"99\", \"Google Chrome\";v=\"121\", \"Chromium\";v=\"121\""],"Sec-Fetch-Dest":["document"],"Cookie":[],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7"],"Sec-Fetch-Site":["none"],"Sec-Fetch-User":["?1"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36"],"Upgrade-Insecure-Requests":["1"],"Accept-Encoding":["gzip, deflate, br"],"Sec-Ch-Ua-Mobile":["?0"],"Sec-Ch-Ua-Platform":["\"Windows\""],"Sec-Fetch-Mode":["navigate"],"Dnt":["1"],"Accept-Language":["ru,en;q=0.9,ru-RU;q=0.8,en-US;q=0.7,jv;q=0.6,ko;q=0.5,zh-TW;q=0.4,zh-CN;q=0.3,zh;q=0.2"]},"tls":{"resumed":true,"version":772,"cipher_suite":4865,"proto":"h3","server_name":"name-site"}},"duration":0.000158836,"status":404,"err_id":"e990rk2mh","err_trace":"fileserver.(*FileServer).notFound (staticfiles.go:629)"}
{"level":"debug","ts":"2024/02/22 18:58:18","name":"http.handlers.file_server","msg":"sanitized path join","site_root":"/var/www/site","request_path":"/assets/news/cover/талисман/0-microsoftteams-image_3_.png","result":"/var/www/site/assets/news/cover/талисман/0-microsoftteams-image_3_.png"}
{"level":"debug","ts":"2024/02/22 18:58:18","name":"http.log.error.log0","msg":"{id=a56igc2es} fileserver.(*FileServer).notFound (staticfiles.go:629): HTTP 404","request":{"remote_ip":"000.000.00.00","remote_port":"64300","client_ip":"000.000.00.00","proto":"HTTP/3.0","method":"GET","host":"name-site","uri":"/assets/news/cover/%D1%82%D0%B0%D0%BB%D0%B8%D1%81%D0%BC%D0%B0%D0%BD/0-microsoftteams-image_3_.png","headers":{"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36"],"Cookie":[],"Sec-Fetch-Site":["none"],"Sec-Fetch-Dest":["document"],"Upgrade-Insecure-Requests":["1"],"Sec-Fetch-User":["?1"],"Sec-Ch-Ua-Mobile":["?0"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7"],"Sec-Fetch-Mode":["navigate"],"Accept-Encoding":["gzip, deflate, br"],"Dnt":["1"],"Cache-Control":["max-age=0"],"Sec-Ch-Ua":["\"Not A(Brand\";v=\"99\", \"Google Chrome\";v=\"121\", \"Chromium\";v=\"121\""],"Sec-Ch-Ua-Platform":["\"Windows\""],"Accept-Language":["ru,en;q=0.9,ru-RU;q=0.8,en-US;q=0.7,jv;q=0.6,ko;q=0.5,zh-TW;q=0.4,zh-CN;q=0.3,zh;q=0.2"]},"tls":{"resumed":true,"version":772,"cipher_suite":4865,"proto":"h3","server_name":"name-site"}},"duration":0.000157534,"status":404,"err_id":"a56igc2es","err_trace":"fileserver.(*FileServer).notFound (staticfiles.go:629)"}
{"level":"debug","ts":"2024/02/22 19:20:42","name":"events","msg":"event","name":"tls_get_certificate","id":"ffbfb346-1168-455e-bad6-ec6c6932d34f","origin":"tls","data":{"client_hello":{"CipherSuites":[49196,49195,49200,49199,159,158,49188,49187,49192,49191,49162,49161,49172,49171,157,156,61,60,53,47,10],"ServerName":"name-site","SupportedCurves":[29,23,24],"SupportedPoints":"AA==","SignatureSchemes":[2052,2053,2054,1025,1281,513,1027,1283,515,514,1537,1539],"SupportedProtos":["http/1.1"],"SupportedVersions":[771,770,769],"RemoteAddr":{"IP":"000.000.00.00","Port":50461,"Zone":""},"LocalAddr":{"IP":"147.45.111.155","Port":443,"Zone":""}}}}
{"level":"debug","ts":"2024/02/22 19:20:42","name":"tls.handshake","msg":"choosing certificate","identifier":"name-site","num_choices":1}
{"level":"debug","ts":"2024/02/22 19:20:42","name":"tls.handshake","msg":"default certificate selection results","identifier":"name-site","subjects":["name-site"],"managed":true,"issuer_key":"acme-v02.api.letsencrypt.org-directory","hash":"0659568ab53c8bfbb823435f3a57d86603526901db173bb51af83f6b0876591f"}
{"level":"debug","ts":"2024/02/22 19:20:42","name":"tls.handshake","msg":"matched certificate in cache","remote_ip":"000.000.00.00","remote_port":"50461","subjects":["name-site"],"managed":true,"expiration":"2024/05/15 18:24:49","hash":"0659568ab53c8bfbb823435f3a57d86603526901db173bb51af83f6b0876591f"}
{"level":"debug","ts":"2024/02/22 19:20:42","name":"http.handlers.file_server","msg":"sanitized path join","site_root":"/var/www/site","request_path":"/assets/news/cover/талисман/0-microsoftteams-image_3_.png","result":"/var/www/site/assets/news/cover/талисман/0-microsoftteams-image_3_.png"}
{"level":"debug","ts":"2024/02/22 19:20:42","name":"http.log.error.log0","msg":"{id=qk5fnt6gi} fileserver.(*FileServer).notFound (staticfiles.go:629): HTTP 404","request":{"remote_ip":"000.000.00.00","remote_port":"50461","client_ip":"000.000.00.00","proto":"HTTP/1.1","method":"GET","host":"name-site","uri":"/assets/news/cover/%D1%82%D0%B0%D0%BB%D0%B8%D1%81%D0%BC%D0%B0%D0%BD/0-microsoftteams-image_3_.png","headers":{"User-Agent":["curl/8.4.0"],"Accept":["*/*"]},"tls":{"resumed":false,"version":771,"cipher_suite":49195,"proto":"http/1.1","server_name":"name-site"}},"duration":0.000199996,"status":404,"err_id":"qk5fnt6gi","err_trace":"fileserver.(*FileServer).notFound (staticfiles.go:629)"}

3. Caddy version:

v2.7.6 h1:w0NymbG2m9PcvKWsrXO6EEkY9Ru4FJK8uQbYcev1p3A=

4. How I installed and ran Caddy:

a. System environment:

Ubuntu 22.04.3 LTS (GNU/Linux 5.15.0-94-generic x86_64)
not Docker, systemctl start caddy

Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Address sizes: 40 bits physical, 48 bits virtual
Byte Order: Little Endian
CPU(s): 2
On-line CPU(s) list: 0,1
Vendor ID: AuthenticAMD
Model name: AMD EPYC Processor

b. Command:

Install: Install — Caddy Documentation
Stable releases:

sudo apt install -y debian-keyring debian-archive-keyring apt-transport-https curl
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | sudo gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | sudo tee /etc/apt/sources.list.d/caddy-stable.list
sudo apt update
sudo apt install caddy

c. Service/unit/compose file:

no

d. My complete Caddy config:

{
	log {
		output file /var/log/caddy/caddy.log {
			roll_size 12mb
			roll_keep 10
			roll_keep_for 2160h
		}
		format json {
			message_key msg
			level_key level
			time_key ts
			name_key name
			time_format wall
		}
	}
}
name-site {
	log {
		output file /var/log/access.log
	}


	# Set this path to your site's directory.
	root * /var/www/site

	# Enable the static file server.
	file_server

	# Another common task is to set up a reverse proxy:
	reverse_proxy /api/* localhost:8765

}

5. Links to relevant resources:

If necessary, I will provide the missing information. Thank you for your help!

Are you sure that path is correct? You said the directory has public, but I’m not seeing that in the joined path here.

Yes, I’m sure, since in this directory there are many other files that have only Latin characters and they load without problems.
So far I see a solution only in excluding unicode characters from the paths, but I want to understand what the problem might be.

If I can help verify this, please write how to do it.

You wrote that the path to the file is:

But the logs show:

So which one is right? Is there public or not?

1 Like

Sorry for misleading you. public/ are the project’s source files before the build. (Pictures)
Everything is correct in the logs.

I tested this locally like this, and it worked fine for me (only the relevant lines of output shown, curl run in a separate terminal window while the server is running):

$ mkdir талисман                                                                             
$ touch талисман/0-microsoftteams-image_3_.png

$ caddy file-server --listen :8882 --debug
2024/02/24 17:07:22.121	DEBUG	http.handlers.file_server	sanitized path join	{"site_root": ".", "request_path": "/талисман/0-microsoftteams-image_3_.png", "result": "талисман/0-microsoftteams-image_3_.png"}
2024/02/24 17:07:22.121	DEBUG	http.handlers.file_server	opening file	{"filename": "талисман/0-microsoftteams-image_3_.png"}

$ curl -v http://localhost:8882/талисман/0-microsoftteams-image_3_.png
< HTTP/1.1 200 OK

$ curl -v http://localhost:8882/%D1%82%D0%B0%D0%BB%D0%B8%D1%81%D0%BC%D0%B0%D0%BD/0-microsoftteams-image_3_.png
< HTTP/1.1 200 OK

I’m not seeing any problem with Caddy in particular. It works when the request is URL encoded, or not URL encoded. Requests with a different path (e.g. removing any single character from the path) results in a 404.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.