Translate nginx rewrite rule for Nextcloud

Help
1

I see a similar topic posted here, but it’s closed without an answer:

tl;dr: I need to translate this rewrite rule from Nginx to a Caddyfile:

        # Required for legacy support
        rewrite ^/(?!index|remote|public|cron|core\/ajax\/update|status|ocs\/v[12]|updater\/.+|ocs-provider\/.+|.+\/richdocumentscode\/proxy) /index.php$request_uri;

I maintain a script to install Nextcloud in a TrueNAS CORE jail using Caddy as the webserver. It’s been working well for some time, but with the release of Nextcloud 28, multiple file uploads and downloads have been failing. Discussion on the Nextcloud forum has pointed to the absence of the above rewrite rule.

If I’m reading that rule correctly (in which I have very little confidence), it says to rewrite anything that does not begin with index or remote or public or… to /index.php{uri}.

So under that understanding, and in hopes of getting rid of the regex (both because I don’t know RE2, and to make the Caddyfile more readable), I inserted this block in the Caddyfile:

	# Required for legacy
	@legacy {
		path /index*
		path /remote*
		path /public*
		path /cron*
		path /core/ajax/update*
		path /status*
		path /ocs/v1/updater/*
		path /ocs/v2/updater/*
		path /ocs-provider/*
		path */richdocumentscode/proxy
	}
	rewrite !@legacy /index.php{uri}

But Caddy doesn’t want to start with this in there, giving the following error:
Error: adapting config using caddyfile: parsing caddyfile tokens for 'rewrite': wrong argument count or unexpected line ending after '/index.php{http.request.uri}', at /usr/local/www/Caddyfile:48

My guess is that I’m using incorrect syntax to invert the named matcher, but I haven’t been able to find what the correct syntax is–or if it’s even possible. Or should I just try to translate the regex to RE2?

2

That’s not valid syntax. Matcher tokens can only be one of three things. See Request matchers (Caddyfile) — Caddy Documentation. You can’t use ! in front of a named matcher like that.

If you want to negate a matcher, you need to use the not matcher which takes another matcher as an argument.

If you have a regexp, you should use the path_regexp matcher. You could negate the regexp match with not if that helps you.

For the path matcher, you can shorten it by putting all the paths on one line

3

I got Caddy to start using

        @notlegacy {
                not path /index*
                not path /remote*
                not path /public*
                not path /cron*
                not path /core/ajax/update*
                not path /status*
                not path /ocs/v1/updater/*
                not path /ocs/v2/updater/*
                not path /ocs-provider/*
                not path */richdocumentscode/proxy
        }
        rewrite @notlegacy /index.php{uri}

But the issue of downloading and uploading remains.

4

I was saying you can simplify it to this:

@notlegacy not path /index* /remote* /public* /cron* /core/ajax/update* /status* /ocs/v1/updater/* /ocs/v2/updater/* /ocs-provider/* */richdocumentscode/proxy
rewrite @notlegacy /index.php{uri}

Please elaborate. Show an example request with curl -v. Show your logs.

5 
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 192.168.1.133:443...
* Connected to nextcloud.local (192.168.1.133) port 443
* schannel: disabled automatic use of client certificate
* ALPN: curl offers http/1.1
* ALPN: server accepted http/1.1
* using HTTP/1.1
* Server auth using Basic with user 'user1'
> GET /apps/files/Translating/* HTTP/1.1
> Host: nextcloud.local
> Authorization: Basic dHNjaGV0dGVyLnZpY3RvckBnbWFpbC5jb206TW9uZ28xOTk3IT8=
> User-Agent: curl/8.4.0
> Accept: */*
>
  0     0    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0* schannel: remote party requests renegotiation
* schannel: renegotiating SSL/TLS connection
* schannel: SSL/TLS connection renegotiated
* schannel: failed to decrypt data, need more data
< HTTP/1.1 200 OK
< Alt-Svc: h3=":443"; ma=2592000
< Cache-Control: no-cache, no-store, must-revalidate
< Content-Length: 42672
< Content-Security-Policy: default-src 'none';base-uri 'none';manifest-src 'self';script-src 'self';style-src 'self' 'unsafe-inline';img-src 'self' data: blob: * https://*.tile.openstreetmap.org;font-src 'self' data:;connect-src 'self';media-src 'self';frame-src 'self' prezi.com player.vimeo.com vine.co www.youtube.com;frame-ancestors 'self';worker-src 'self';form-action 'self'
< Content-Type: text/html; charset=UTF-8
< Expires: Thu, 19 Nov 1981 08:52:00 GMT
< Feature-Policy: autoplay 'self';camera 'none';fullscreen 'self';geolocation 'none';microphone 'none';payment 'none'
< Pragma: no-cache
< Referrer-Policy: no-referrer
< Server: Caddy
< Set-Cookie: oc_sessionPassphrase=yWSuQapu5arypIow7spQaX6rtHAIrVnh7X5HQ7SVYQi9VJETXGysHCsSD7y7e6NJVK7Ay6HdVz%2FBl%2B0jFrro0SamUzHnzng%2F2QONAV2CDXgFhe2CnYBuSaOP5mH1b%2FoB; path=/; secure; HttpOnly; SameSite=Lax
< Set-Cookie: __Host-nc_sameSiteCookielax=true; path=/; httponly;secure; expires=Fri, 31-Dec-2100 23:59:59 GMT; SameSite=lax
< Set-Cookie: __Host-nc_sameSiteCookiestrict=true; path=/; httponly;secure; expires=Fri, 31-Dec-2100 23:59:59 GMT; SameSite=strict
< Set-Cookie: ocn3w8j5jl8q=fe911ctrmbsmrl566fn6rcb1pu; path=/; secure; HttpOnly; SameSite=Lax
< Set-Cookie: cookie_test=test; expires=Sat, 02 Mar 2024 20:15:38 GMT; Max-Age=3600
< X-Content-Type-Options: nosniff
< X-Frame-Options: SAMEORIGIN
< X-Permitted-Cross-Domain-Policies: none
< X-Powered-By: PHP/8.2.14
< X-Request-Id: t5VeraDZR7enrVSCB6iZ
< X-Robots-Tag: noindex, nofollow
< X-Xss-Protection: 1; mode=block
< Date: Sat, 02 Mar 2024 19:15:38 GMT
<
{ [7165 bytes data]
100 42672  100 42672    0     0  29043      0  0:00:01  0:00:01 --:--:-- 29087
* Connection #0 to host nextcloud.local left intact

Command used was curl -v -u user1:password -O https://nextcloud.local/apps/files/Translating/*

6

And the downloaded result was download.htm

7

Looks like the response was successful. What’s the problem exactly?

Show evidence of the problem. Explain what’s not working. Show your Caddy logs.

8

Sorry. The issue is that when uploading multiple files, only one file will be uploaded. When downloading multiple files, a “download.htm” file will be download.

This is the only log I’m getting.

{"level":"info","ts":1709408123.041174,"logger":"http.log.access.log0","msg":"handled request","request":{"remote_ip":"192.168.1.113","remote_port":"59444","client_ip":"192.168.1.113","proto":"HTTP/1.1","method":"GET","host":"nextcloud.local","uri":"/apps/files/Translating/*","headers":{"Authorization":[],"User-Agent":["curl/8.4.0"],"Accept":["*/*"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"http/1.1","server_name":"nextcloud.local"}},"bytes_read":0,"user_id":"","duration":0.993976669,"size":42668,"status":200,"resp_headers":{"Feature-Policy":["autoplay 'self';camera 'none';fullscreen 'self';geolocation 'none';microphone 'none';payment 'none'"],"X-Content-Type-Options":["nosniff"],"Expires":["Thu, 19 Nov 1981 08:52:00 GMT"],"Content-Security-Policy":["default-src 'none';base-uri 'none';manifest-src 'self';script-src 'self';style-src 'self' 'unsafe-inline';img-src 'self' data: blob: * https://*.tile.openstreetmap.org;font-src 'self' data:;connect-src 'self';media-src 'self';frame-src 'self' prezi.com player.vimeo.com vine.co www.youtube.com;frame-ancestors 'self';worker-src 'self';form-action 'self'"],"Set-Cookie":[],"X-Permitted-Cross-Domain-Policies":["none"],"X-Powered-By":["PHP/8.2.14"],"Cache-Control":["no-cache, no-store, must-revalidate"],"X-Request-Id":["bcP1b1ciNd7dPqz3i9lf"],"Content-Length":["42668"],"X-Xss-Protection":["1; mode=block"],"X-Robots-Tag":["noindex, nofollow"],"Content-Type":["text/html; charset=UTF-8"],"Alt-Svc":["h3=\":443\"; ma=2592000"],"Referrer-Policy":["no-referrer"],"X-Frame-Options":["SAMEORIGIN"],"Server":["Caddy"],"Pragma":["no-cache"]}}
9

I wouldn’t rule out a Nextcloud issue as at least part of what’s going on. But am I correctly understanding that both:

@notlegacy {
                not path /index*
                not path /remote*
                not path /public*
                not path /cron*
                not path /core/ajax/update*
                not path /status*
                not path /ocs/v1/updater/*
                not path /ocs/v2/updater/*
                not path /ocs-provider/*
                not path */richdocumentscode/proxy
        }
        rewrite @notlegacy /index.php{uri}

…and:

@notlegacy not path /index* /remote* /public* /cron* /core/ajax/update* /status* /ocs/v1/updater/* /ocs/v2/updater/* /ocs-provider/* */richdocumentscode/proxy
rewrite @notlegacy /index.php{uri}

…should be equivalent to this from Nginx:

# Required for legacy support
        rewrite ^/(?!index|remote|public|cron|core\/ajax\/update|status|ocs\/v[12]|updater\/.+|ocs-provider\/.+|.+\/richdocumentscode\/proxy) /index.php$request_uri;

?

10

And this might be a better error here.

{"level":"error","ts":1709418579.408178,"logger":"http.log.access.log0","msg":"handled request","request":{"remote_ip":"192.168.1.113","remote_port":"61692","client_ip":"192.168.1.113","proto":"HTTP/2.0","method":"GET","host":"nextcloud.local","uri":"/apps/files/ajax/download.php?dir=%2FAudio%20Bibles&files=%5B%22Blank%20Audio%20Bible%20Folders%22%5D&downloadStartSecret=ue04g5838yl","headers":{"Sec-Fetch-Mode":["navigate"],"Sec-Fetch-Dest":["empty"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36"],"Accept-Encoding":["gzip, deflate, br"],"Accept-Language":["en-US,en;q=0.9"],"Cookie":[],"Sec-Gpc":["1"],"Sec-Fetch-Site":["same-origin"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","server_name":"nextcloud.local"}},"bytes_read":0,"user_id":"","duration":0.001383421,"size":0,"status":500,"resp_headers":{"Server":["Caddy"],"Alt-Svc":["h3=\":443\"; ma=2592000"],"Status":["500 Internal Server Error"],"X-Powered-By":["PHP/8.2.14"],"Content-Type":["text/html; charset=UTF-8"]}}

It looks like an internal server error. Would that be Caddy or Nextcloud?

11

That error is coming from NextCloud.

13

So upon further testing, this block allows proper downloads.

@notlegacy {
                not path /index*
                not path /remote*
                not path /public*
                not path /cron*
                not path /core/ajax/update*
                not path /status*
                not path /ocs/v1*
                not path /ocs/v2*
                not path /ocs-provider/*
                not path /updater/*
                not path */richdocumentscode/proxy*
        }
        rewrite @notlegacy /index.php{uri}

The issue is now that some things like the login page are not rendering properly. When I add path /apps* which this rule should be rewriting on its own, then the login page renders. Any ideas? Would a debug help?

14

Below are three ways to implement this rewrite rule
NGINX

 location ~ \.php(?:$|/) {
        # Required for legacy support
        rewrite ^/(?!index|remote|public|cron|core\/ajax\/update|status|ocs\/v[12]|updater\/.+|ocs-provider\/.+|.+\/richdocumentscode(_arm64)?\/proxy) /index.php$request_uri;

CADDY
This is the organized version and easy to read

# required for legacy support
@notlegacy {
                path *.php *.php/
                not path /index*
                not path /remote*
                not path /public*
                not path /cron*
                not path /core/ajax/update*
                not path /status*
                not path /ocs/v1*
                not path /ocs/v2*
                not path /ocs-provider/*
                not path /updater/*
                not path */richdocumentscode/proxy*
        }
        rewrite @notlegacy /index.php{uri}

This is the shortened version of the above

# required for legacy support
@notlegacy {
    path *.php *.php/
    not path /index* /remote* /public* /cron* /core/ajax/update* /status* /ocs/v1* /ocs/v2* /updater* /ocs-provider/* */richdocumentscode/proxy*
}
rewrite @notlegacy /index.php{uri}

This is the path_regexp that is the most similar to the nginx block

# required for legacy support
        @notlegacy {
                path *.php *.php/
                not path_regexp ^/(index|remote|public|cron|core/ajax/update|status|ocs/v[12]|updater/.+|ocs-provider/.+|.+/richdocumentscode(_arm64)?/proxy)
        }
        rewrite @notlegacy /index.php{uri}
2 Likes
15

I played around with the path_regexp, but can’t get it working like the above example. If someone wants to shorten their Caddyfile, the can simply do this as well.

@notlegacy {
    path *.php *.php/
    not path /index* /remote* /public* /cron* /core/ajax/update* /status* /ocs/v1* /ocs/v2* /updater* /ocs-provider/* */richdocumentscode/proxy*
}
rewrite @notlegacy /index.php{uri}

This works exactly the same, but with less lines.

16

I don’t think the $ here works as you expect, path isn’t regexp. Unless you’re trying to match an actual $ in the URL which isn’t really a thing.

You can use https://regex101.com/ to test your regexp. Choose the golang mode on the left.

17

Ooops, forgot about that one…

18

I guess a relevant question would be if there’s any benefit to reducing the number of lines in the Caddyfile. I expect any difference would be marginal at best, but if we have options of:

  • Several lines of path/not path statements
  • A couple of lines of path/not path statements
  • A single path_regexp statement

Is there going to be any difference in how Caddy processes those? That is, is Caddy going to perform better with one vs. the others, or is one going to be notably worse? Because, presuming they all accomplish the same thing, if there isn’t a significant performance difference, I’d favor what Victor posted a few days back over putting everything on one or two lines, just for ease of reading. Yeah, it’ll take a few more bytes of disk space, and maybe a few more bytes of RAM, but hardly enough to be noticeable.

19

There can be a performance difference between path_regexp and path because it’s running totally different code – you’ll need to do some benchmarking if you care, I doubt you do cause it’s on the order of nanoseconds of difference.

There’s no performance difference between single line vs multi line, it should produce the same JSON config (you can verify by running caddy adapt -p on your config to see the adapted JSON). It’s just a matter of being nicer to read (less visual noise) if that’s important to you. If you prefer the long form, then by all means use that. Just wanted to point out a shorter way because that’s something some people value.

20

Kind of what I thought. In a much larger deployment than I have in mind this might be significant, but I’d be very surprised if the installation resulting from my script would serve as many as a few dozen users per instance.

It’s always good to know of alternatives.

21

Finally got the not path_regexp working properly. It needs to be paired with a path like this though

        # required for legacy support
        @notlegacy {
                path *.php *.php/
                not path_regexp ^/(index|remote|public|cron|core/ajax/update|status|ocs/v[12]|updater/.+|ocs-provider/.+|.+/richdocumentscode(_arm64)?/proxy)
        }
        rewrite @notlegacy /index.php{uri}

Downloads work, login page works, can’t see anything NOT working at this point.
Its exactly the same as the nginx rule right now with two exceptions.

  1. We use a path *.php *.php/ to substitue the location ~ \.php(?:$|/) of nginx
  2. Instead of the negative lookahead ?! we use the not matcher for path_regexp

See above post for all solutions

2 Likes