Need help migrating to Caddyfile v2 about url rewrite rules

Help
1

I am trying to upgrade my Caddyfile from v1 to v2, and I have completed most content unless some url rewrite rules for Nextcloud, cause they are too difficult for me to convert so far. The related codes in the current Caddyfile v1 are listed below:

  # PHP
  fastcgi / 127.0.0.1:9000 php {
    env PATH /bin
    env front_controller_active true
    env modHeadersAvailable true
  }

  # Functional rewrite
  rewrite {
    r ^/\.well-known/host-meta$
    to /public.php?service=host-meta&{query}
  }
  rewrite {
    r ^/\.well-known/host-meta\.json$
    to /public.php?service=host-meta-json&{query}
  }
  rewrite {
    r ^/\.well-known/webfinger$
    to /public.php?service=webfinger&{query}
  }
 
  # Check for IMAGES and CSS JS
  rewrite {
    if {method} is GET
    ext .js .svg .gif .png .html .ttf .woff .ico .jpg .jpeg .css
    r ^/(.+)$
    to /{1} /index.php?{1}
  }

  # For direct PHP using case
  rewrite {
    if_op or
    if {path} starts_with /updater
    if {path} starts_with /ocm-provider
    if {path} starts_with /ocs-provider
    r /.*
    to {path}/
  }
 
  # Route the views to index.php
  rewrite {
    ext /
    if_op and
    if {path} not_starts_with /index.php
    if {path} not_starts_with /remote.php
    if {path} not_starts_with /public.php
    if {path} not_starts_with /corn.php
    if {path} not_starts_with /core/ajax/update.php
    if {path} not_starts_with /status.php
    if {path} not_starts_with /updater
    if {path} not_starts_with /ocs
    if {path} not_starts_with /.well-known/
    r /(.*)
    to /index.php?{query}
  }
 
  # Remove trailing as it causes errors with php-fpm
  rewrite {
    r ^/remote.php/(webdav|caldav|carddav|dav)(\/?)(\/?)$
    to /remote.php/{1}
  }
 
  rewrite {
    r ^/remote.php/(webdav|caldav|carddav|dav)/(.+?)(\/?)(\/?)$
    to /remote.php/{1}/{2}
  }
 
  rewrite {
    r ^/public.php/(dav|webdav|caldav|carddav)(\/?)(\/?)$
    to /public.php/{1}
  }
 
  rewrite {
    r ^/public.php/(dav|webdav|caldav|carddav)/(.+)(\/?)(\/?)$
    to /public.php/{1}/{2}
  }

It is hoped that someone can help me modify it to the right format for Caddyfile v2, any advice would be highly appreciated. Thanks!

2

What have you tried so far?

3

I have rewritten the PHP part as below:

php_fastcgi localhost:9000 {
    env PATH /bin
    env front_controller_active true
    env modHeadersAvailable true
  }

However when I run caddy validate it noticed “Error during parsing: unrecognized subdirective env”.

And I have tried some rewrite lines directly as:

  # Functional rewrite
  rewrite ^/\.well-known/host-meta$ /public.php?service=host-meta&{query}
  
  rewrite ^/\.well-known/host-meta\.json$ /public.php?service=host-meta-json&{query}
  
  rewrite ^/\.well-known/webfinger$ /public.php?service=webfinger&{query}

  # Remove trailing as it causes errors with php-fpm
  rewrite ^/remote.php/(webdav|caldav|carddav|dav)(\/?)(\/?)$ /remote.php/{1}
 
  rewrite ^/remote.php/(webdav|caldav|carddav|dav)/(.+?)(\/?)(\/?)$ /remote.php/{1}/{2}
 
  rewrite ^/public.php/(dav|webdav|caldav|carddav)(\/?)(\/?)$ /public.php/{1}
 
  rewrite ^/public.php/(dav|webdav|caldav|carddav)/(.+)(\/?)(\/?)$ /public.php/{1}/{2}

and without accident it also cannot be validated with errors of “Wrong argument count or unexpected line ending”.

1 Like
4

Ah, yeah, the php_fastcgi one might be a little tricky because it’s just a shortcut for a more complex configuration, and you have an advanced use case here.

This guide should help: Upgrading to Caddy 2 — Caddy Documentation

For the php stuff, I’d recommend using the expanded form so you can customize the environment variables in the fastcgi transport yourself; this is something we could improve on but it’s not slated for the 2.0 release. I bet @francislavoie would have some tips here.

5

Unfortunately, the php_fastcgi shortcut doesn’t allow to configure additional env. For that, you’ll need to use the expanded form seen here:

This’ll look something like this:

# Add trailing slash for directory requests
@canonicalPath {
    file {
        try_files {path}/index.php
    }
    not {
        path */
    }
}
redir @canonicalPath {path}/ 308

# If the requested file does not exist, try index files
try_files {path} {path}/index.php index.php

# Proxy PHP files to the FastCGI responder
@phpFiles {
    path *.php
}
reverse_proxy @phpFiles <php-fpm_gateway> {
    transport fastcgi {
        split .php
        env PATH /bin
        env front_controller_active true
        env modHeadersAvailable true
    }
}

We’ll look into making it possible to do this with the php_fastcgi directive. The problem is that currently, it passes through all options to the reverse_proxy directive, and we don’t have a clean way to intercept any other directives that aren’t supported by reverse_proxy but that are supported by php_fastcgi. I’ll try to look into improving this soon.

That said, are you sure you need to pass those env variables through Caddy? They aren’t dynamic, so you can probably set them set them in your php-fpm's .conf file instead and it would simplify your Caddy config significantly.

As for your rewrite rules, you’ll need to use a matcher block for each one because you’re using regexp. Please read through the matcher docs to understand how they work:

For example:

@hostMeta {
    path_regexp ^/\.well-known/host-meta$
}
rewrite @hostMeta /public.php?service=host-meta&{query}

@remoteDav1 {
    path_regexp dav1 ^/remote.php/(webdav|caldav|carddav|dav)(\/?)(\/?)$
}
rewrite @remoteDav1 /remote.php/{http.regexp.dav1.1}

Note that you need to use a named regexp (i.e. dav1) if you need to reference any capture groups in your result by name or number. See here for the syntax:

Note that since your dav rewrites seems to just strip a final /, you can do this much more easily with the uri directive. It might be something as simple as:

@remoteOrPublic {
    path_regexp ^/(remote|public).php/
}
uri @remoteOrPublic strip_suffix /

You’ll need to test this out to make sure it does what you expect.

1 Like
6

@francislavoie

Wouldn’t simply this work? (No need to use the fully expanded form I think. Just customize the transport.)

php_fastcgi 127.0.0.1:9000 {
    transport fastcgi {
        split .php
        env PATH /bin
        env front_controller_active true
        env modHeadersAvailable true
    }
}
1 Like
7

Oh, yeah that’s true, that should work too!

8

No idea what really is going on, just trying to get nextcloud going without warnings and by some googling and trying and combining shit without understanding it… I got the warnings gone:

Your web server is not properly set up to resolve “/.well-known/caldav”.
Your web server is not properly set up to resolve “/.well-known/carddav”.

using this:

@hostMeta {
    path_regexp ^/.well-known/(card|cal)dav
}
rewrite @hostMeta /remote.php/dav/

for complain about HTTP Strict Transport Security and max age this solved it

header {
    Strict-Transport-Security max-age=31536000;
}

/edit so that I dont polute the thread needlessly

So clean looking and working reverse proxy for nextcloud:

nextcloud.{$MY_DOMAIN} {
    reverse_proxy nextcloud:80
    header Strict-Transport-Security max-age=15552000;
    redir /.well-known/carddav /remote.php/carddav 301
    redir /.well-known/caldav /remote.php/caldav 301
}
9

That pretty closely matches the recommended nginx config from here Nginx configuration — Nextcloud 15 Administration Manual 15 documentation, but they use a redirect instead of a rewrite. I’m not sure if that’ll make much of a different in this case, but just thought I’d note that.

Also, you should probably rename your matcher to something like @wellKnownDav, more descriptive than @hostMeta which is the name I picked for a different rewrite path.

If this is the only header you’re setting, you can shorten that to:

header Strict-Transport-Security max-age=31536000;
1 Like
10

Thanks very much for your guidance , it is really helpful! Now I have modified other rewrite rules in my old Caddyfile to the version below, however there are still some syntax confusing me such as ext and if_op, which I don’t know how to convert them properly, and please point out if any syntax error exists, cause I am really not sure whether they are correct. :joy:

  # Check for IMAGES and CSS JS (OLD)
  rewrite {
    if {method} is GET
    ext .js .svg .gif .png .html .ttf .woff .ico .jpg .jpeg .css
    r ^/(.+)$
    to /{1} /index.php?{1}
  }

 # Check for IMAGES and CSS JS (NEW)
  @staticFile {
    method GET
    path *.js|*.svg|*.gif|*.png|*.html|*.ttf|*.woff|*.ico|*.jpg|*.jpeg|*.css
    path_regexp static ^/(.+)$
  }
  rewrite @staticFile /{http.regexp.static.1} /index.php?{http.regexp.static.1}
----------------------------------------------------------
  # For direct PHP using case (OLD)
  rewrite {
    if_op or
    if {path} starts_with /updater
    if {path} starts_with /ocm-provider
    if {path} starts_with /ocs-provider
    r /.*
    to {path}/
  }

  # For direct PHP using case (NEW)
  @phpCase {
    ???
    path /updater/*
    path /ocm-provider/*
    path /ocs-provider/*
    regexp /.*
  }
  rewrite @phpCase {path}/
----------------------------------------------------------
  # Route the views to index.php (OLD)
  rewrite {
    ext /
    if_op and
    if {path} not_starts_with /index.php
    if {path} not_starts_with /remote.php
    if {path} not_starts_with /public.php
    if {path} not_starts_with /corn.php
    if {path} not_starts_with /core/ajax/update.php
    if {path} not_starts_with /status.php
    if {path} not_starts_with /updater
    if {path} not_starts_with /ocs
    if {path} not_starts_with /.well-known/
    r /(.*)
    to /index.php?{query}
  }

  # Route the views to index.php (NEW)
  @indexView {
    path */
    ???
    not {
      path /index.php/*
      path /remote.php/*
      path /public.php/*
      path /corn.php/*
      path /core/ajax/update.php/*
      path /status.php/*
      path /updater/*
      path /ocs/*
      path /.well-known/*
    }
    path_regexp /(.*)
  }
  rewrite @indexView /index.php?{query}
11

Perhaps you can try

  # Client support
  redir /.well-known/carddav /remote.php/carddav 301
  redir /.well-known/caldav /remote.php/caldav 301
12

Alright, there’s a whole lot going on here. Instead of just fixing the entire thing for you, I’ll just tell you what I’m spotting as incorrect syntax or what you should look into.

The syntax for path matchers is path <paths>. In general, Caddyfile arguments are separated by spaces, not by |.

That said, I think that in this case when whitelisting extensions, it would be faster to use a regexp matcher instead like \.(js|svg)$ etc.

This isn’t necessary. You’re using it to get a match on the path, but Caddy actually provides you with a {path} placeholder which does this already. You can see here a list of the shortened placeholders that the Caddyfile provides (there are others, but they’re scattered throughout the docs depending on the module that provides them).

Instead of rewrite, you should use the try_files directive to check if the requested path exists on disk, otherwise fall back to another path. This is what the to /{1} /index.php?{1} actually did in v1, it first checked if /{1} existed on disk, otherwise rewrote the path to /index.php?{1}. Be sure to read the try_files docs!

Also to note, the syntax of your rewrite is incorrect, because you’re giving two arguments after the matcher. rewrite only supports one to argument.

I honestly don’t understand what the goal of this is. It feels like an incomplete rule and I’m probably lacking context as to what your app is doing. If I’m reading it correctly, it’s just suffixing the path with a /. I feel that it’s probably unnecessary and it’s probably already covered by the try_files that’s built into php_fastcgi, i.e. trying {path}/index.php, and also by the path canocalization redirect (see the expanded form).

Also FYI, path_regexp /.* is unnecessary, that would always be true if any other the other conditions pass.

I think most of this block is also unnecessary, because of the try_files in php_fastcgi as well. It’ll try the file as-is (like remote.php) then do the fallback to index.php if the file doesn’t exist on disk. The fastcgi transport uses a split .php argument (see the expanded form) which basically says “take anything after .php and make that the request path, but use that PHP file to handle it”.

I don’t know what the /updater, /ocs and /.well-known paths have in them, but if they just include files on disk, then it should already be just fine (because of try_files).

1 Like
13

I tried your format and an error occurred:

2020/04/11 10:07:00.812 INFO using adjacent Caddyfile
validate: adapting config using caddyfile: parsing caddyfile tokens for 'php_fastcgi': Caddyfile:16 - Error during parsing: transport already specified

However the fully expanded form which @francislavoie suggested can be validated normally.

14

Thanks so much for your patient reply. I have modified the static file rewrite rules as below, is there still anything wrong and can it achieve the desired effect? (At least the ./caddy validate doesn’t report an error now.)

  # Check for IMAGES and CSS JS
  @staticFile {
    method GET
    path_regexp static \.(js|svg|gif|png|html|ttf|woff|ico|jpg|jpeg|css)$
  }
  try_files @staticFile /{http.regexp.static.1} /index.php?{http.regexp.static.1}
15

All the rewrite rules are copied from examples/nextcloud/Caddyfile at master ¡ caddyserver/examples ¡ GitHub

Maybe someone could create a new examples repository for Caddyfile v2 if possible someday :stuck_out_tongue_closed_eyes:

16

Ah, that’s unfortunate. Good news is that yes, the expanded form should work just fine. Even better news, I spent some time last night working on a PR to make it possible to pass some options given to php_fastcgi down to the underlying transport! It might take some time to make it in though.

I still highly recommend you look into setting those environment variables in your php-fpm config instead of through Caddy. You may also not even need those environment variables, they might only be required to work around some issues with Nginx that might not apply to Caddy.

That looks better, but using {http.regexp.static.1} doesn’t make sense here. That would rewrite to /js literally because the first capturing group is just the file extension, not the requested path. Instead, you’d want something like this:

try_files @staticFile {path} /index.php?{path}

What would actually be easier is to base your Caddy v2 config on the official Nginx config for Nextcloud. Caddy v2 is closer in capabilities for request matching to Nginx than to Caddy v1, so it would be easier to translate that way. See

https://docs.nextcloud.com/server/15/admin_manual/installation/nginx.html#nextcloud-in-the-webroot-of-nginx

1 Like
17

Thanks again for your help. Currently the PHP part can work almost, however there are still several problems remaining on the loading of static files, and I’ve decided to give up for now, it seems that I should continue with Caddy v1 temporarily till more users have stepped over the pit, cause my knowledge in this area is rather lacking. :joy:

18

A post was split to a new topic: PHP Nextcloud - PROPFIND method not allowed error (v2)

19

Older topic. Looking at redirect (using a 301) v. rewrite: Rewrite would allow you to redirect if you wanted to move some functionality to a different server (dav, card serving). Also, wouldn’t a handful of matched redirects be more efficient than a path_regexp rewrite rule? And redirect happens once… that’s the point of a 301 is the client will now look at the correct path, as opposed to a rewrite happening on every call.

20

@lexpierce Hmm, almost!

  • Rewrites are internal redirects. The Caddyfile tries to make sure that only one of them happens exclusively of others (though that is not an absolute requirement, but it is the default behavior). Rewrites do not change the server or host of the request; they happen purely internally. The rewrite directive means “I’m going to accept this request, but only with changes first.”

  • Redirects are external and evaluated by the client. Since redirects are the response to a request, necessarily only one of them happens. Redirects can change anything about the request, including its scheme, host, port, path, etc. The redir directive means “I’m going to reject this request. The client should use another location instead.”

2 Likes