V2 Help with path_regexp and rewrite

Help
1

1. Caddy version (caddy version):

v2.1.1

2. How I run Caddy:

/usr/bin/caddy run --config /etc/caddy/Caddyfile --adapter caddyfile

a. System environment:

Docker, alpine 3.12, interactive session.

b. Command:

/usr/bin/caddy run --config /etc/caddy/Caddyfile --adapter caddyfile

c. Service/unit/compose file:

N/A

d. My complete Caddyfile or JSON config:

sub.domain.com {
    tls /etc/ssl/cert.pem /etc/ssl/key.pem
    encode gzip

    root * /www

    @ipfilter {
        not remote_ip 192.168.100.0/24
    }
    respond @ipfilter 403

    @zxp {
        path_regexp myregex ^/([zx]p?)/(.*)/(.+\.(?:gif|jpe?g|png|txt|html?|css|js))$
    }
    rewrite @zxp /mnt/{http.matchers.path_regexp.myregex.1}/{http.matchers.path_regexp.myregex.2}/img/{http.matchers.path_regexp.myregex.3}
    file_server

    log {
        output stdout
        format single_field common_log
    }

    header Strict-Transport-Security max-age=31536000;
}

3. The problem I’m having:

Idea is, that when user goes to https://sub.domain.com/z/blah/image.png, image is sourced from the /mnt/z/blah/img/image.png directory on the server. The file is present, but every time I get 404 error.

As a troubleshooting step, I replaced rewrite detective with the

respond @zxp "FOUND" 200

and when making queries to the expected urls, I get FOUND and code 200 in the browser. So perhaps I’m somewhere near, but not getting it…

Basically, I’m looking for something like Nginx’s alias detective.

2

This isn’t exactly right. The placeholder is {http.regexp.myregex.1}, but there’s a shortcut available in the Caddyfile since v2.1, {re.myregex.1}.

3

@francislavoie Thanks for pointing this out. Perhaps, I was using outdated thread on this forum to find this information and didn’t look up updated syntax. Unfortunately, it still does not work for me, after I replaced the line in question with the:

    rewrite @zxp /mnt/{re.myregex.1}/{re.myregex.2}/img/{re.myregex.3}

Is there a way to run Caddy in more verbose mode, to point out what’s going on internally?

Thanks!

4

Yeah, you can add the following at the top of your Caddyfile to see a bit more information in the logs:

{
	debug
}
5

You can also play around with https://regex101.com/ (pick the Golang mode on the left) to see if your regexp is correct. You’ll notice your capture groups on the right.

6

Oh, wait - you’re expecting the files to come from /mnt on your server, I see.

A rewrite will change the request path, but Caddy takes that request path and appends it to the configured root. In your case, what will happen is Caddy will be looking in /www/mnt/...

What you’ll need to do is change the root at the same time if the matcher matches:

	@zxp path_regexp myregex ^/([zx]p?)/(.*)/(.+\.(?:gif|jpe?g|png|txt|html?|css|js))$

	root @zxp /mnt
	rewrite @zxp /{re.myregex.1}/{re.myregex.2}/img/{re.myregex.3}

I think this should do.

7

@francislavoie – thanks. unfortunately, using root @zxp /mnt did not help, despite I clearly see in the logs, that rewrite happens. As a test, I mounted file systems under /www and it made things work.
Any way to debug why changing root for the matcher? I didn’t find any useful information with enabled debug :frowning:

8

What’s your full Caddyfile at this point? Did you remove /mnt from the rewrite? My guess is that you didn’t do that, which means Caddy would look at /mnt/mnt on disk.

9

I did remove, otherwise it wouldn’t work under /www either.

{
    debug
}

sub.domain.com {
    tls /etc/ssl/cert.pem /etc/ssl/key.pem
    encode gzip

    root * /www

    @ipfilter {
        not remote_ip 192.168.100.0/24
    }
    respond @ipfilter 403

	@zxp path_regexp myregex ^/([zx]p?)/(.*)/(.+\.(?:gif|jpe?g|png|txt|html?|css|js))$
	root @zxp /mnt
	rewrite @zxp /{re.myregex.1}/{re.myregex.2}/img/{re.myregex.3}
    # respond @zxp "FOUND" 200
    file_server *

    log {
        output stdout
        format single_field common_log
    }

    header Strict-Transport-Security max-age=31536000;
}
10

Oh dear… I have no idea how I came up with this, but this is the config which works.

{
    debug
}

sub.domain.com {
    tls /etc/ssl/cert.pem /etc/ssl/key.pem
    encode gzip

    root * /www

    @ipfilter {
        not remote_ip 192.168.100.0/24
    }
    respond @ipfilter 403

    @zxp path_regexp myregex ^/([zx]p?)/(.*)/(.+\.(?:gif|jpe?g|png|txt|html?|css|js))$
    rewrite @zxp /{re.myregex.1}/{re.myregex.2}/img/{re.myregex.3}
    file_server @zxp {
        root @zxp /mnt
    }
    file_server *

    log {
        output stdout
        format single_field common_log
    }

    header Strict-Transport-Security max-age=31536000;
}

If someone could explain my why, I’d appreciate.

11

Ah, I see what’s going on now.

This is a quirk of the Caddyfile that I wasn’t considering when I answered earlier. The behaviour is clearer if you adapt your config to JSON. As a simplified example, take a look at this:

:80

root /* /foo
root /z* /bar

file_server

This adapts to this JSON:

{
  "apps": {
    "http": {
      "servers": {
        "srv0": {
          "listen": [
            ":80"
          ],
          "routes": [
            {
              "group": "group1",
              "match": [
                {
                  "path": [
                    "/z*"
                  ]
                }
              ],
              "handle": [
                {
                  "handler": "vars",
                  "root": "/bar"
                }
              ]
            },
            {
              "group": "group1",
              "match": [
                {

                  "path": [
                    "/*"
                  ]
                }
              ],
              "handle": [
                {
                  "handler": "vars",
                  "root": "/foo"
                }
              ]
            },
            {
              "handle": [
                {
                  "handler": "file_server",
                  "hide": [
                    "Caddyfile-f9442"
                  ]
                }
              ]
            }
          ]
        }
      }
    }
  }
}

Notice that the root with the /z* matcher appears before the one with the * matcher (i.e. no matcher in JSON because * means all requests).

This is because when adapting the Caddyfile, the handlers of the same type are sorted according to the length of their path matchers. This is typically the right thing to do for most matchers, because you want the most specific matcher to be handled first.

But in the case of root, this is actually a variable that gets set, so you actually want them to be set in the opposite order, such that the least specific is first, and the most specific is last, so that the most specific value applies.

I’m not sure this is really a “bug” per se, but it’s definitely a non-obvious behaviour that I think should be fixed (because it’s pretty clear which one is typically intended here, not much ambiguity).

So the reason it works with your version (although I don’t think it’s exactly correct, having @zxp inside of file_server isn’t right, it should look more like the below) is because file_server can accept a root subdirective to override the root that’s set “globally” on the request.

    file_server @zxp {
        root /mnt
    }

Another possible way to write it is like this:

	route {
		root * /www
		root @zxp /mnt
	}

The route directive will override the default sorting behaviour and make sure the root directives will appear in that order in the final JSON config which Caddy actually uses to run your server. It also lets you avoid having two file_server directives.

Anyways, glad you found a fix, I’ll try to see if it’s viable to change the default ordering behaviour for root so that this sort of issue doesn’t come up again. :+1:

12

Thanks for the explanation!

I will try route option, as it feels more sane, but so far I’m happy that it was possible to resolve :slight_smile:

13

FYI I opened a PR to fix this behaviour, hopefully it’ll be in the next release (but no promises).

https://github.com/caddyserver/caddy/pull/3658

2 Likes
14

FYI the fix was merged (but ended up being different than originally proposed) and will be in the final v2.2 release.

15

This topic was automatically closed after 30 days. New replies are no longer allowed.