Rewrite directive with chinese characters respond 404

jun_dai · February 8, 2023, 4:37am

1. Caddy version:

v2.6.2 h1:wKoFIxpmOJLGl3QXoo6PNbYvGW4xLEgo32GPBEjWL8o=

2. How I installed, and run Caddy:

apt install, systemctl start caddy

a. System environment:

ubuntu 22.04

b. Command:

Paste command here.

c. Service/unit/compose file:

Paste full file contents here.
Make sure backticks stay on their own lines,
and the post looks nice in the preview pane.

d. My complete Caddy config:

:80 {
      handle_path /Upload/* {
            rewrite * /Upload{uri}
            reverse_proxy http://cgs-server.mcs:3030
    }
   handle_path /api/cloud/* {
               reverse_proxy http://cgs-server.mcs:3030

   }
}

3. The problem I’m having:

There is a file named 中文.txt , and another file test.txt,

For URL: http://127.0.0.1/Upload/Model/2023020308171328d48e/中文.txt
It’s result an 404.
But use http://127.0.0.1/api/cloud/Upload/Model/2023020308171328d48e/中文.txt
It’s responed with correct file content.

And if I use the en named file, both returned correct.

http://127.0.0.1/Upload/Model/2023020308171328d48e/test.txt
http://127.0.0.1/api/cloud/Upload/Model/2023020308171328d48e/test.txt

So, the problem is does rewirte directive support chinese characters ?

4. Error messages and/or full log output:

Paste logs/commands/output here.
USE THE PREVIEW PANE TO MAKE SURE IT LOOKS NICELY FORMATTED.

5. What I already tried:

6. Links to relevant resources:

francislavoie · February 8, 2023, 4:40am

Please enable the debug global option, and show what’s in your logs. It’ll show what’s happening with the rewrite.

Are you sure the problem is with Caddy? You’re proxying to another server. Is it a problem with that server?

jun_dai · February 8, 2023, 5:31am

Here is the log:

{
	"level": "debug",
	"ts": 1675833788.326499,
	"logger": "http.handlers.rewrite",
	"msg": "rewrote request",
	"request": {
		"remote_addr": "10.233.88.0:57577",
		"proto": "HTTP/1.1",
		"method": "GET",
		"host": "117.78.5.138:15389",
		"uri": "/Upload/Model/2023020308171328d48e/%E5%B7%A5%E4%BD%9C%E9%9D%A2.glb",
		"headers": {
			"Connection": ["keep-alive"],
			"Upgrade-Insecure-Requests": ["1"],
			"User-Agent": ["Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36"],
			"Accept": ["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"],
			"Accept-Encoding": ["gzip, deflate"],
			"Accept-Language": ["zh-CN,zh;q=0.9"],
			"Cookie": ["saber-user-headPic=undefined; token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2NzgzNDgxMDIsImlhdCI6MTY3NTc1NjEwMiwidXNlcklEIjoiNjM4ODQyNzE2Y2FiNDljODkyOTIyMmNhIn0.t6JOAZNxK3KvQDhzt5cUgtOwDUrZ9ZHN0S4yStGePrk; Token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2NzgzNDgxMDIsImlhdCI6MTY3NTc1NjEwMiwidXNlcklEIjoiNjM4ODQyNzE2Y2FiNDljODkyOTIyMmNhIn0.t6JOAZNxK3KvQDhzt5cUgtOwDUrZ9ZHN0S4yStGePrk; UserInfo={%22Name%22:%22%E7%8E%8B%E7%8E%89%22%2C%22Token%22:%22eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2NzgzNDgxMDIsImlhdCI6MTY3NTc1NjEwMiwidXNlcklEIjoiNjM4ODQyNzE2Y2FiNDljODkyOTIyMmNhIn0.t6JOAZNxK3KvQDhzt5cUgtOwDUrZ9ZHN0S4yStGePrk%22%2C%22Username%22:%22123456%22}"]
		}
	},
	"method": "GET",
	"uri": "/Model/2023020308171328d48e/%E5%B7%A5%E4%BD%9C%E9%9D%A2.glb"
}
{
	"level": "debug",
	"ts": 1675833788.326546,
	"logger": "http.handlers.rewrite",
	"msg": "rewrote request",
	"request": {
		"remote_addr": "10.233.88.0:57577",
		"proto": "HTTP/1.1",
		"method": "GET",
		"host": "117.78.5.138:15389",
		"uri": "/Model/2023020308171328d48e/%E5%B7%A5%E4%BD%9C%E9%9D%A2.glb",
		"headers": {
			"Accept-Encoding": ["gzip, deflate"],
			"Accept-Language": ["zh-CN,zh;q=0.9"],
			"Cookie": ["saber-user-headPic=undefined; token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2NzgzNDgxMDIsImlhdCI6MTY3NTc1NjEwMiwidXNlcklEIjoiNjM4ODQyNzE2Y2FiNDljODkyOTIyMmNhIn0.t6JOAZNxK3KvQDhzt5cUgtOwDUrZ9ZHN0S4yStGePrk; Token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2NzgzNDgxMDIsImlhdCI6MTY3NTc1NjEwMiwidXNlcklEIjoiNjM4ODQyNzE2Y2FiNDljODkyOTIyMmNhIn0.t6JOAZNxK3KvQDhzt5cUgtOwDUrZ9ZHN0S4yStGePrk; UserInfo={%22Name%22:%22%E7%8E%8B%E7%8E%89%22%2C%22Token%22:%22eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2NzgzNDgxMDIsImlhdCI6MTY3NTc1NjEwMiwidXNlcklEIjoiNjM4ODQyNzE2Y2FiNDljODkyOTIyMmNhIn0.t6JOAZNxK3KvQDhzt5cUgtOwDUrZ9ZHN0S4yStGePrk%22%2C%22Username%22:%22123456%22}"],
			"Connection": ["keep-alive"],
			"Upgrade-Insecure-Requests": ["1"],
			"User-Agent": ["Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36"],
			"Accept": ["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"]
		}
	},
	"method": "GET",
	"uri": "/Upload/Model/2023020308171328d48e/%25E5%25B7%25A5%25E4%25BD%259C%25E9%259D%25A2.glb"
}
{
	"level": "debug",
	"ts": 1675833788.3301532,
	"logger": "http.handlers.reverse_proxy",
	"msg": "upstream roundtrip",
	"upstream": "cgs-server.mcs:3030",
	"duration": 0.003567194,
	"request": {
		"remote_addr": "10.233.88.0:57577",
		"proto": "HTTP/1.1",
		"method": "GET",
		"host": "117.78.5.138:15389",
		"uri": "/Upload/Model/2023020308171328d48e/%25E5%25B7%25A5%25E4%25BD%259C%25E9%259D%25A2.glb",
		"headers": {
			"Cookie": ["saber-user-headPic=undefined; token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2NzgzNDgxMDIsImlhdCI6MTY3NTc1NjEwMiwidXNlcklEIjoiNjM4ODQyNzE2Y2FiNDljODkyOTIyMmNhIn0.t6JOAZNxK3KvQDhzt5cUgtOwDUrZ9ZHN0S4yStGePrk; Token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2NzgzNDgxMDIsImlhdCI6MTY3NTc1NjEwMiwidXNlcklEIjoiNjM4ODQyNzE2Y2FiNDljODkyOTIyMmNhIn0.t6JOAZNxK3KvQDhzt5cUgtOwDUrZ9ZHN0S4yStGePrk; UserInfo={%22Name%22:%22%E7%8E%8B%E7%8E%89%22%2C%22Token%22:%22eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2NzgzNDgxMDIsImlhdCI6MTY3NTc1NjEwMiwidXNlcklEIjoiNjM4ODQyNzE2Y2FiNDljODkyOTIyMmNhIn0.t6JOAZNxK3KvQDhzt5cUgtOwDUrZ9ZHN0S4yStGePrk%22%2C%22Username%22:%22123456%22}"],
			"X-Forwarded-For": ["10.233.88.0"],
			"X-Forwarded-Proto": ["http"],
			"Upgrade-Insecure-Requests": ["1"],
			"User-Agent": ["Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36"],
			"Accept": ["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"],
			"Accept-Encoding": ["gzip, deflate"],
			"Accept-Language": ["zh-CN,zh;q=0.9"]
		}
	},
	"headers": {
		"Access-Control-Allow-Origin": ["*"],
		"Content-Type": ["application/json; charset=utf-8"],
		"Last-Modified": ["Wed, 08 Feb 2023 05:23:08 GMT"],
		"X-Content-Type-Options": ["nosniff"],
		"X-Frame-Options": ["DENY"],
		"X-Request-Id": ["e9b43697-362b-4b7e-b639-73e81ae7be5e"],
		"Content-Length": ["42"],
		"Cache-Control": ["no-cache, no-store, max-age=0, must-revalidate, value"],
		"Expires": ["Thu, 01 Jan 1970 00:00:00 GMT"],
		"X-Xss-Protection": ["1; mode=block"],
		"Date": ["Wed, 08 Feb 2023 05:23:08 GMT"]
	},
	"status": 404
}

And I found the URL was replaced from %E5%B7%A5%E4%BD%9C%E9%9D%A2.glb to %25E5%25B7%25A5%25E4%25BD%259C%25E9%259D%25A2.glb

jun_dai · February 8, 2023, 5:44am

For correct, The caddy version was v2.4.6 h1:HGkGICFGvyrodcqOOclHKfvJC0qTU7vny/7FhYp9hNw=
Not 2.6.2

francislavoie · February 8, 2023, 6:06am

Ah, I see. So the problem is that the {uri} placeholder returns the URI in its encoded form. So when the rewrite happens, encodes it a second time.

URI handling and encoding is very very tricky stuff.

In your case, I’m not convinced you even need the rewrite though.

You’re using handle_path, which strips the /Upload part of the URL. If you instead use handle, then it doesn’t strip it.

	handle /Upload/* {
		reverse_proxy http://cgs-server.mcs:3030
   	}

An alternate approach to doing the rewrite which I think won’t double-encode is using uri replace like this:

uri replace / /prefix 1

This should replace the first / in the URI with /prefix, limited to 1 match so it doesn’t replace every / in the URI.

jun_dai · February 8, 2023, 6:24am

Yes, In this case rewrite is not needed if use handle directive.
And after testing, The uri replace directive solved this problem.
Thanks a lot.

system · March 10, 2023, 6:24am

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.