Can't get simple alias to work

[piranha (Alexander Solovyov)]

So I’m moving to Caddy 2 from Nginx and I want to have a setup like this:

solovyov.net {
  root * /opt/solovyov.net/www
  root /q/* /mnt/share
  file_server *
  file_server /q/* browse
  encode zstd gzip
}

But no luck: /q/ says 404. /mnt/share exists for sure (this works in Nginx). I tried configuring root inside of file_server, but the problem persists: site itself works, but my little alias does not.

Any pointers please? Also how do I debug stuff like this, since logs just say 404 (nginx usually gives out file name it tries to find in error log).

[Whitestrake (Matthew Fay)]

Hey @piranha, welcome to the Caddy community!

On its face, this can look a bit complicated because of how you’re setting up two file servers. On top of that, remember that just because you’re filtering for /q/ doesn’t mean it gets stripped out of the request when looking for a file!

For example:

If you’ve got a file at /mnt/share/foo/bar.txt, and you make a request to /q/foo/bar/txt, you’ll get a 404. Why? Because Caddy’s trying to find /mnt/share/q/foo/bar.txt (root + URI path = served file). So if you don’t have your files actually located inside /mnt/share/q/, you’re going to need to strip that part of the URI first.

So lets try splitting this up a little bit. Conceptually and functionally, this is probably going to serve you a lot better:

example.com {
  # First, handle requests for /q/
  handle /q/* {
    root /mnt/share
    uri strip_prefix /q
    file_server browse
  }

  # Otherwise, handle the rest
  handle {
    root /opt/solovyov.net/www
    file_server
  }

  encode zstd gzip
}

handle groups are mutually exclusive - only one ever gets executed for a given request. It also encapsulates your handling a bit, making things easier to read and understand at a glance. For any given request, only the first handle block that matches is executed. This makes it impossible for the “wrong” root to be set or the “wrong” file server to act. Finally I throw the encoder at the bottom, it executes on all requests regardless of route.

[piranha (Alexander Solovyov)]

I see! That makes sense, just as root/alias thingie in nginx. I somehow didn’t notice handle, is not mentioned in tutorials so I skimmed over it on directive page.

Thanks!

EDIT: I ended up with config like that:

solovyov.net {
  file_server * { root /opt/solovyov.net/www }
  handle /q/* {
    uri strip_prefix /q
    file_server * browse { root /mnt/share }
  }
  encode zstd gzip
}

[piranha (Alexander Solovyov)]

Also interesting that I had to write my cache headers like that:

  handle {
    header Cache-Control max-age=3600
  }

  handle /static/* {
    header Cache-Control max-age=31536000
  }
  handle /favicon.ico {
    header Cache-Control max-age=31536000
  }

instead of much simpler:

    header * Cache-Control max-age=3600
    header /static/* Cache-Control max-age=31536000
    header /favicon.ico Cache-Control max-age=31536000

since in this case main header overrides everything. Not sure if that’s expected?

[piranha (Alexander Solovyov)]

@Whitestrake interesting, but after I add that handle { header ... } (the one for all site) my handle /q/* .. stops working. Any ideas?

[Whitestrake (Matthew Fay)]

Nope, only one handle gets executed, ever.

You can put your header directives outside the handle block, that shouldn’t be an issue.

The “much simpler” variant you posted should be fully functional.

[piranha (Alexander Solovyov)]

@Whitestrake

Nope, only one handle gets executed, ever.

Hmm, this is weird then, why does my setup with few handle blocks with headers work? Also, in simpler setup all my responses get max-age=3600 header.

EDIT: okay, in this case everything works:

  header * Cache-Control max-age=3600

  handle /static/* {
    header Cache-Control max-age=31536000
  }

  handle /favicon.ico {
    header Cache-Control max-age=31536000
  }

  handle /q/* {
    uri strip_prefix /q
    root * /mnt/share
    file_server * browse
  }

but this looks like some bug or oversight or underdocumentation. I’d really like to remove handle around my headers plus to understand how handle is chosen.

[Whitestrake (Matthew Fay)]

Evaluates a group of directives mutually exclusively from other handle blocks at the same level of nesting.

The handle directive is kind of similar to the location directive from nginx config: the first matching handle block will be evaluated. Handle blocks can be nested if needed. Only HTTP handler directives can be used inside handle blocks.
—handle (Caddyfile directive) — Caddy Documentation

So, at each level, only one handle directive is evaluated, and it’s the first one that matches.

I did note that when setting the same header like that multiple times, the generic one was always selected (header *), with no regard to either specificity or the order in which they’re set, so the longer-duration ones never seemed to apply.

Putting the longer-duration ones in a handle block while leaving the generic one on the top level seemed to fix that. This must be something to do with the fact they’re put in a subroute?

Wouldn’t mind @matt’s eyes and input on this one. At the very least, there should be some clear method of exercising control over which header is executed, so that cases of specific-over-general can be configured. Certainly, having to use handle groups in this manner to enable specificity of header directives isn’t ergonomic, nor is it particularly discoverable from documentation.

Anyway, I discovered through testing that the headers don’t need to be handled separately, they just need to be put in any subroute to “beat” the generic header. So here’s the neatest Caddyfile I could come up with that seems to work:

solovyov.net {
  encode zstd gzip
  header Cache-Control max-age=3600

  handle /q/* {
    uri strip_prefix /q
    root * /mnt/share
    file_server browse
  }

  handle {
    header /static/* Cache-Control max-age=31536000
    header /favicon.ico Cache-Control max-age=31536000
    root * /opt/solovyov.net/www
    file_server
  }
}

[matt (Matt Holt)]

We’re looking into why this is the case.

[Whitestrake (Matthew Fay)]

Just had a chat with @matt. Apart from discovering what looks like a weird bug (which is adjacent to your issue), to explain what’s going on, allow me to elaborate on the intended behaviour:

When all three are lined up like this:

header * Cache-Control max-age=3600
header /static/* Cache-Control max-age=31536000
header /favicon.ico Cache-Control max-age=31536000

The Caddyfile adapter interprets it like this:

          "routes": [
            {
              "match": [
                {
                  "path": [
                    "/static/*"
                  ]
                }
              ],
              "handle": [
                {
                  "handler": "headers",
                  "response": {
                    "set": {
                      "Cache-Control": [
                        "max-age=31536000"
                      ]
                    }
                  }
                }
              ]
            },
            {
              "match": [
                {
                  "path": [
                    "/favicon.ico"
                  ]
                }
              ],
              "handle": [
                {
                  "handler": "headers",
                  "response": {
                    "set": {
                      "Cache-Control": [
                        "max-age=31536000"
                      ]
                    }
                  }
                }
              ]
            },
            {
              "handle": [
                {
                  "handler": "headers",
                  "response": {
                    "set": {
                      "Cache-Control": [
                        "max-age=3600"
                      ]
                    }
                  }
                }
              ]
            }
          ]

Note that the smallest Cache-Control setting - the generic one (i.e. header *) - comes last. This is because Caddy is designed to order adjacent routes, whenever they are based on path matchers, by the specificity of the path matched.

In plain English: longest matching path executes first. Side effect - shortest matching path executes last. When dealing with headers, when the headers are written, the most recent call to set a header is the only one that matters. So if we execute the most generic header last, the most generic header is the one that will override the others. Obviously this isn’t ideal behaviour.

(You may or may not have noticed how in the above JSON, /static/* comes before /favicon.ico - despite how I just said Caddy’s designed to sort longest path routes first. Yeah, that’s the bug I mentioned earlier. It’s doing some sorting - * always comes last - but something’s whacky there. Strictly speaking, though, this bug isn’t what’s causing your issue, the actual intended design of Caddy is doing that. Matt also found the cause of that bug and will fix it…)

Anyway, putting the header directives in a handle block forces the Caddyfile adapter not to put it before the generic one.

Putting them in a route block - which is designed to give you explicit control over the execution of the directives - will also allow you to get the desired behaviour, by manually ordering the headers from generic to specific.

So here’s the latest I believe should work:

solovyov.net {
  route {
    header Cache-Control max-age=3600
    header /static/* Cache-Control max-age=31536000
    header /favicon.ico Cache-Control max-age=31536000
  }

  handle /q/* {
    uri strip_prefix /q
    root * /mnt/share
    file_server browse
  }

  encode zstd gzip
  root * /opt/solovyov.net/www
  file_server
}

---

Edit: @matt moves fast - above mentioned bug fixed in httpcaddyfile: Fix route ordering bug · caddyserver/caddy@cd9317e · GitHub, in case you’re curious.

[matt (Matt Holt)]

My bad.

Anyway, @Whitestrake’s explanation is correct. You want route (or handle, but route is more concise) in your case.

[piranha (Alexander Solovyov)]

Cool! This works. I think this needs to be mentioned in header docs somehow? That with overlapping matches more generic wins rather than more specific. Or maybe there is a way to change to a more expected behavior?

[Whitestrake (Matthew Fay)]

I think that question bears a little more looking into.

At the moment, I’m thinking it might be best just to include an example like this in the header documentation:

route {
  header Cache-Control max-age=3600
  header /static/* Cache-Control max-age=31536000
  header /favicon.ico Cache-Control max-age=31536000
}

And explain what the use case is for it.

This is an interesting case where the behaviour we’d usually expect to be ideal - that is, evaluating the more specific route first - is actually counterproductive for headers!

But it’s not feasible, I don’t think, to change Caddy to sort routes the opposite way around when path matchers that only have header handlers are in play. I can just imagine how hacky that’d be, and I can see edge cases galore…

So yeah, probably an update to the docs is at hand.