Rewrite first then evaluate php_fastcgi

Help
1

This does not seem to work as expected, the re-written request is not processed by php_fastcgi. Is there a way to fix it without using redir instead of rewrite?

        @nextcloud host nc.example.com
        handle @nextcloud {
                root * /var/www/html
                file_server
                php_fastcgi unix//run/php/php7.4-fpm.sock {
                        env front_controller_active true
                        header_up X-Real-IP {remote_host}
                }
                rewrite /.well-known/carddav /remote.php/dav
                rewrite /.well-known/caldav /remote.php/dav
                ....
       }

I also tried moving the rewrites before php_fastcgi thinking that maybe the order matters, but it does not seem to.

2

The order you put lines in your Caddyfile does not matter because Caddy sorts directives according to a predetermined order. See here:

Turn on the debug global option.

What do you see in your logs?

Please fill out the help topic template when asking questions on the forums. It’s hard to help with limited information, because it only adds more variables to why things may not be working.

3

OK, but this is supposed to work, so I don’t chase things that are not intended to work, correct? I will look at debug next.

4

I got a log, the rewrite does happen before php_fastcgi. Relevant lines (with a replaced actual domain):

{"level":"debug","ts":1645289708.9771225,"logger":"http.handlers.rewrite","msg":"rewrote request","request":{"remote_addr":"192.168.1.2:43136","proto":"HTTP/2.0","method":"GET","host":"nc.example.com","uri":"/.well-known/caldav","headers":{"User-Agent":["curl/7.68.0"],"Accept":["*/*"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","proto_mutual":true,"server_name":"nc.example.com"}},"method":"GET","uri":"/remote.php/dav"}

{"level":"debug","ts":1645289708.9771683,"logger":"http.handlers.rewrite","msg":"rewrote request","request":{"remote_addr":"192.168.1.2:43136","proto":"HTTP/2.0","method":"GET","host":"nc.example.com","uri":"/remote.php/dav","headers":{"User-Agent":["curl/7.68.0"],"Accept":["*/*"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","proto_mutual":true,"server_name":"nc.example.com"}},"method":"GET","uri":"/remote.php"}

{"level":"debug","ts":1645289708.977239,"logger":"http.reverse_proxy.transport.fastcgi","msg":"roundtrip","request":{"remote_addr":"192.168.1.2:43136","proto":"HTTP/2.0","method":"GET","host":"nc.example.com","uri":"/remote.php","headers":{"User-Agent":["curl/7.68.0"],"Accept":["*/*"],"X-Real-Ip":["192.168.1.2"],"X-Forwarded-For":["192.168.1.2"],"X-Forwarded-Proto":["https"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","proto_mutual":true,"server_name":"nc.example.com"}},"dial":"127.0.0.1:9000","env":{"AUTH_TYPE":"","CONTENT_LENGTH":"","CONTENT_TYPE":"","DOCUMENT_ROOT":"/var/www/html","DOCUMENT_URI":"/remote.php","GATEWAY_INTERFACE":"CGI/1.1","HTTPS":"on","HTTP_ACCEPT":"*/*","HTTP_HOST":"nc.example.com","HTTP_USER_AGENT":"curl/7.68.0","HTTP_X_FORWARDED_FOR":"192.168.1.2","HTTP_X_FORWARDED_PROTO":"https","HTTP_X_REAL_IP":"192.168.1.2","PATH_INFO":"/dav","PATH_TRANSLATED":"/var/www/html/dav","QUERY_STRING":"","REMOTE_ADDR":"192.168.1.2","REMOTE_HOST":"192.168.1.2","REMOTE_IDENT":"","REMOTE_PORT":"43136","REMOTE_USER":"","REQUEST_METHOD":"GET","REQUEST_SCHEME":"https","REQUEST_URI":"/.well-known/caldav","SCRIPT_FILENAME":"/var/www/html/remote.php","SCRIPT_NAME":"/remote.php","SERVER_NAME":"nc.example.com","SERVER_PROTOCOL":"HTTP/2.0","SERVER_SOFTWARE":"Caddy/v2.4.6","SSL_CIPHER":"TLS_AES_128_GCM_SHA256","SSL_PROTOCOL":"TLSv1.3","front_controller_active":"true"}}

In the request to php-fpm 127.0.0.1:9000 these look possibly wrong:

"DOCUMENT_URI": "/remote.php",
"PATH_INFO": "/dav",
"PATH_TRANSLATED": "/var/www/html/dav",

php-fpm logs show "GET /remote.php" 404 but if I directly request /remote.php/dav it works. It appears there’s some confusion in the path/doc root when rewriting a path and passing it to php-fpm.

Does this look like a Caddy bug?

5

Here’s the log line for a direct request (no rewrite) to /remote.php/dav that works:

{"level":"debug","ts":1645290598.4459014,"logger":"http.reverse_proxy.transport.fastcgi","msg":"roundtrip","request":{"remote_addr":"192.168.1.2:43138","proto":"HTTP/2.0","method":"GET","host":"nc.example.com","uri":"/remote.php","headers":{"Accept":["*/*"],"X-Forwarded-For":["192.168.1.2"],"X-Forwarded-Proto":["https"],"X-Real-Ip":["192.168.1.2"],"User-Agent":["curl/7.68.0"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","proto_mutual":true,"server_name":"nc.example.com"}},"dial":"127.0.0.1:9000","env":{"AUTH_TYPE":"","CONTENT_LENGTH":"","CONTENT_TYPE":"","DOCUMENT_ROOT":"/var/www/html","DOCUMENT_URI":"/remote.php","GATEWAY_INTERFACE":"CGI/1.1","HTTPS":"on","HTTP_ACCEPT":"*/*","HTTP_HOST":"nc.example.com","HTTP_USER_AGENT":"curl/7.68.0","HTTP_X_FORWARDED_FOR":"192.168.1.2","HTTP_X_FORWARDED_PROTO":"https","HTTP_X_REAL_IP":"192.168.1.2","PATH_INFO":"/dav","PATH_TRANSLATED":"/var/www/html/dav","QUERY_STRING":"","REMOTE_ADDR":"192.168.1.2","REMOTE_HOST":"192.168.1.2","REMOTE_IDENT":"","REMOTE_PORT":"43138","REMOTE_USER":"","REQUEST_METHOD":"GET","REQUEST_SCHEME":"https","REQUEST_URI":"/remote.php/dav","SCRIPT_FILENAME":"/var/www/html/remote.php","SCRIPT_NAME":"/remote.php","SERVER_NAME":"nc.example.com","SERVER_PROTOCOL":"HTTP/2.0","SERVER_SOFTWARE":"Caddy/v2.4.6","SSL_CIPHER":"TLS_AES_128_GCM_SHA256","SSL_PROTOCOL":"TLSv1.3","front_controller_active":"true"}}

The only differences between the two:

<     "REMOTE_PORT": "43136",
---
>     "REMOTE_PORT": "43138",
25c25
<     "REQUEST_URI": "/.well-known/caldav",
---
>     "REQUEST_URI": "/remote.php/dav",

The port diff doesn’t matter of course, so is it all about the REQUEST_URI then?

6

Looks very similar to this Nginx issue:

Perhaps Caddy needs to pass the re-written path as REQUEST_URI.

7

That’s correct. That’s how the FastCGI spec requires those environment variables to be set.

PATH_TRANSLATED is by definition the webroot + PATH_INFO. Which is super weird and arguably useless. But that’s what the spec says to do. See RFC 3875 - The Common Gateway Interface (CGI) Version 1.1. I don’t think many apps actually use it.

I think what you actually need is redir, not rewrite, because the upstream app may be looking at REQUEST_URI which will always contain the original URI of the request as Caddy received it from the client (again, this is as required by the FastCGI spec).

This isn’t a problem with Caddy, this is a problem with the upstream app not handling a rewritten request properly.

8

Is it possible to force Caddy to pass the re-written path in REQUEST_URI via php_fastcgi env settings?

9

Yes, but I don’t recommend it.

Why not just use redir?

Changing the behaviour of REQUEST_URI is breaking the FastCGI spec.

10

Some (broken) DAV clients don’t seem to want to follow redirects for these special discovery URLs. If it’s not one problem it’s another …

Changing the behaviour of REQUEST_URI is breaking the FastCGI spec.

I’m not worried about breaking the spec. This is fine for 1 subdomain that only serves Nextcloud via php-fpm and for only these 2 re-writes. I’m more interested in making it work well rather than following the spec.

Could you please post the magic env directive to try it? Thank you.

11

You could try env REQUEST_URI {uri}

Consider reporting this issue to Nextcloud for them to fix it.

12

It doesn’t appear to work, it gets set to /remote.php, it’s missing the trailing /dav. It also breaks everything else because all paths end up trimmed now. Is there another variable that also includes the path_info?

Do you think it’s a Nextcloud issue or rather a general php-fpm bug? I’m not sure whether it’s the apps that handle REQUEST_URI themselves or they are at the mercy of the php-fpm framework.

13

Ah, right. I think you need this:

env REQUEST_URI {path}{http.matchers.file.remainder}
14

Unfortunately it still doesn’t seem to work. It fixes the rewrites but breaks everything else, not sure if it somehow interacts with env front_controller_active true

15

Is the {rewrite_uri} placeholder no longer available in v2? It sounds like that’s what I would need as for v1 it was documented to include path, query string, and fragment.
Is there a list of all available placeholders in v2?

16

Not really, because placeholders are added per module. They’re not always all set. There is a list for the HTTP app here though:

That’s what {uri} is in Caddy v2.

But not fragment, because the fragment is never sent to the server by the browser, that’s a client-side thing.

I really don’t know what to suggest because you haven’t given any feedback as to what effect it has. Check your logs. What does a good request look like without the env change, and what does a bad one look like?

What you could do is have two php_fastcgi directives, one with a matcher for the well-known paths you need which overrides env, and the other without a matcher and no overrides.

1 Like
17

I don’t know how to log just placeholder values so instead I’m just sending DEBUG env vars and looking at the log:

        php_fastcgi 127.0.0.1:9000 {
			root /var/www/html
			env front_controller_active true
			header_up X-Real-IP {remote_host}

			env DEBUG_uri {uri}
			env DEBUG_path {path}
			env DEBUG_remainder {http.matchers.file.remainder}
		}

A request to https://nc.example.com/settings/apps/enabled that works without any re-writes logs this:

    "DEBUG_path": "index.php",
    "DEBUG_remainder": "",
    "DEBUG_uri": "index.php",
    "REQUEST_URI": "/settings/apps/enabled",

If I want to always override REQUEST_URI, when there is no re-write it’s not possible to get `“REQUEST_URI”: “/settings/apps/enabled” from the DEBUG_ values above.

As you suggested, there may need to be two php_fastcgi directives, unless there is a conditional way to set REQUEST_URI only when there was a rewrite applied or more specific only if REQUEST_URI == /.well-known/ca*dav*

18

Well clearly in that case, the original URI is necessary. There is {http.request.orig_uri} but I don’t see a way for you to have your cake and eat it too.

There isn’t. You’ll definitely need two separate handlers for that. Request matchers are how you set up conditions in Caddy, conditions on requests don’t happen within the handlers (usually).

2 Likes
19

Thank you very much for your help and patience, I’ve learned a lot from this.

20

This topic was automatically closed after 30 days. New replies are no longer allowed.