Converting rewrite Rules for Woltab Suite

Help
1

1. My Caddy version (caddy version):

v2.0.0-rc.3 h1:z2H/QnaRscip6aZJxwTbghu3zhC88Vo8l/K57WUce4Q=

2. How I run Caddy:

Caddy has been installed on Ubuntu using the repo’s:

echo "deb [trusted=yes] https://apt.fury.io/caddy/ /" \
    | sudo tee -a /etc/apt/sources.list.d/caddy-fury.list
sudo apt update
sudo apt install caddy

a. System environment:

  • Ubuntu 20.04
  • Caddy is installed as a service using the repo
  • PHP-FPM 7.4.5

b. Command:

systemctl start caddy

c. Service/unit/compose file:

/etc/caddy/Caddyfile

test.ke.vinpet.it, www.test.ke.vinpet.it {
        root * /var/www/test.ke.vinpet.it/website/public
        encode zstd gzip
        file_server
}
staging.guildwarslegacy.com, www.staging.guildwarslegacy.com {
        root * /var/www/staging.guildwarslegacy.com/public_html
        php_fastcgi unix//run/php/php7.4-fpm-gwl_staging.sock

        @woltlab {
            path_regexp woltlab ^/index.php/(forum/|cms/|wcf/|calendar/|filebase/|blog/|gallery/)?([^.]+)$
        }
        rewrite @woltlab /index.php/{http.regexp.woltlab.1}


        encode zstd gzip
        file_server
        log {
           output file /var/log/caddy/access.log
       }
}

3. The problem I’m having:

I’m trying to rewrite the Woltlab nginx rewrite rules to something that Caddy understands.

These rewrite rules look like:

location / {
    index index.php;
    
    try_files $uri $uri/ @rewrite;
}
location @rewrite {
    rewrite ^/(forum/|cms/|wcf/|calendar/|filebase/|blog/|gallery/)?([^.]+)$ /$1index.php?$2 last;
}

If I am correct, Caddy handles the try files with php_fastcgi.

Currently, it just loads the homepage over and over again, not a single link works.

4. Error messages and/or full log output:

I don’t get any errors and all log output simply gives the homapage.

5. What I already tried:

At first, I tried this approach:

#        rewrite /forum/* forum/index.php?{query}{path}
#       rewrite /cms/* cms/index.php?{path}
#       rewrite /wcf/* wcf/index.php?{path}
#       rewrite /calender/* calendar/index.php?{path}
#       rewrite /filebase/* filebase/index.php?{path}
#       rewrite /blog/* blog/index.php?{path}
#       rewrite /gallery/* gallery/index.php?{path}

I switched out path and query at one point, that also did not work.

After some research, I tried this:

        @woltlab {
            path_regexp woltlab ^/index.php/(forum/|cms/|wcf/|calendar/|filebase/|blog/|gallery/)?([^.]+)$
        }
        rewrite @woltlab /index.php/{http.regexp.woltlab.1}

However, it still didn’t work. I have no further idea how to fix this, though I’m still searching for more information online.

6. Links to relevant resources:

The website with issues is located at https://staging.guildwarslegacy.com

Thanks for taking a look at this!

Kevin

2

Awesome! I’m glad to see you’re using the apt repo we set up, you’re the first I’ve seen that’s using it. That was my pet project in the past couple weeks :grin:

I think this is the equivalent of your nginx rewrite:

rewrite @woltlab /{http.regexp.woltlab.1}index.php?{http.regexp.woltlab.2}
1 Like
3

Thanks very much for the repo, it is so easy to get things set up and it works like a charm!

I tried that config, but unfortunately still the same: it just loads the main page and nothing else.

1 Like
4

Hmm.

I’m on mobile right now so I’m not able to fully test things - try the site https://regex101.com and choose the golang mode on the left pane. You should be able to test that the regex you’re using matches correctly for the paths you try. I’m thinking the ? may be the issue, it may need to be escaped with a \ before it.

5

I got it to work partially with some edits, This is my current setup:

        @woltlab {
            path_regexp woltlab ^/(forum/|cms/|wcf/|calendar/|filebase/|blog/|gallery/)?([^.]+)$
        }
        rewrite @woltlab /{http.regexp.woltlab.1}index.php?{http.regexp.woltlab.2}

Things that aren’t working: the main forum does not load. It simply gives an error page now that it can’t find the page. Most other pages are working.
The admin part (located at https://staging.guildwarslegacy.com/acp/index.php?login/&url=https%3A%2F%2Fguildwarslegacy.com%2Facp%2Findex.php%3Findex%2F) is still broken, it gives a login loop.

I’m still trying to find out what’s going wrong here, but thanks for all your help so far francislavoie! :smiley:

6

You don’t seem to have acp/ in your regexp. You probably should add that in the first match group.

I’m not certain about the forum though, I don’t have enough information about that one.

7

acp/ is a special case. It’s not mentioned in the official Woltlab rewrite rules either, it doesn’t use pretty URL’s to work right. It does give a redirect loop now, it mentions the session has timed out.
Adding it to the regex doesn’t work, either.

For now, some things seem to work correctly:

Main issue here seems to be that the ?pageNo=6 isn’t recognized :confused:

I’m not so good with regexes… do you have any idea?

8

Okay, next day, fresh mind - I have found something that is odd.

When I try to open up a forum thread, for example https://staging.guildwarslegacy.com/forum/thread/1607-minions-of-the-rabbit-legion-pupu/ it works without an issue.
However, when something is added behind this, it becomes an issue, for example https://staging.guildwarslegacy.com/forum/thread/1607-minions-of-the-rabbit-legion-pupu/?action=firstNew gets rewritten to https://staging.guildwarslegacy.com/forum/?action=firstNew which leads to the error. It looks like part of the path gets removed and that is causing the issue.
Not certain if Caddy is doing this or Woltlab, but I’m suspecting that this is the issue.

9

Yeah I think the ? in the regexp is the issue. Depending on the regexp flavor it might be interpreted in different ways. It can mean “zero or one of the preceding element” or it can match for the character ? literally. I’m not certain which is happening but I think it might be the later in this situation meaning that the rest of the request path is discarded.

Sorry I’m still on mobile this weekend and the regex101.com site didn’t seem to work for me on mobile, so I can’t test it out to verify :disappointed_relieved:

10

Haven’t looked in much detail, but yes ? is a special character in regex that means to make the previous token optional. \? matches a literal question mark.

If you’re trying to match query string, you’ll need a different matcher, since path_regexp matches on path only (obviously).

11

Okay I’m finally back at my PC.

Here’s what the regexp and the replacements look like with all the URLs you provided: regex101: build, test, and debug regex

Note that I had to add \n as an excluded character from the second matcher otherwise it was greedily reading past the newline when matching. That isn’t necessary when using it in Caddy, it just makes it easier to test in regex101.com

You can add more example paths to the top textbox (one per line) and you’ll see how the rewrite will look in the bottom. I’m still not entirely clear on what the ultimate endgoal is for each path, what should be the correct rewrite for each path.

One thing I’m noticing is that the second capture group seems to take precedence over the first for paths like /forum/ because the second capture group doesn’t have a “quantifier” that tells it that it can appear zero times. I think this regexp might be more correct (note the ? as the 2nd last character):

^/(forum/|cms/|wcf/|calendar/|filebase/|blog/|gallery/)?([^.\n]+)?$
1 Like
12

Okay, a few days have gone past.
I have just reinstalled my server as there were several strange things going on on the server.

This is my current setup:

test.guildwarslegacy.com  {
        root * /var/www/test.guildwarslegacy.com/public_html
        encode zstd gzip
        log {
                output file /var/log/access-test.log {
            roll_size 100mb
            roll_keep 10
            roll_keep_for 720h
        }
        format single_field common_log
        level INFO
        }
        php_fastcgi unix//run/php/php7.4-fpm-gwl_test.sock

        @woltlab {
             path_regexp woltlab ^/(forum/|cms/|wcf/|calendar/|filebase/|blog/|gallery/)?([^.\n]+)?$
       }
       rewrite @woltlab /{http.regexp.woltlab.1}index.php?{http.regexp.woltlab.2}

     file_server

}

I’ve now gotten the control panel to work, which allows me to run a test to see if the URL rewriting is working properly. Under the hood, it requests these pages:

188.188.113.52 - - [13/May/2020:19:46:38 +0000] "GET /core-rewrite-test/?uuidHash=e6d1665ee15da70311b50f1cf76b11649212b9efc02d2c97619b6ea3be91199a HTTP/2.0" 404 9987
188.188.113.52 - - [13/May/2020:19:46:38 +0000] "GET /forum/core-rewrite-test/?uuidHash=e6d1665ee15da70311b50f1cf76b11649212b9efc02d2c97619b6ea3be91199a HTTP/2.0" 404 9996

Since the test installation is on https://test.guildwarslegacy.com, that means the URLs are:

These currently both return a 404 instead of what they are supposed to show:

{"core_rewrite_test":"passed"}

Currently I get this in my Caddy logs:

188.188.113.52 - - [13/May/2020:20:57:19 +0000] "GET /forum/core-rewrite-test/?uuidHash=e6d1665ee15da70311b50f1cf76b11649212b9efc02d2c97619b6ea3be91199a HTTP/2.0" 404 10001
188.188.113.52 - - [13/May/2020:20:57:19 +0000] "GET /core-rewrite-test/?uuidHash=e6d1665ee15da70311b50f1cf76b11649212b9efc02d2c97619b6ea3be91199a HTTP/2.0" 404 9991

This is identical to what I get on my Nginx setup, so I am a bit clueless to what the issue can be :confused:
I’m looking at this with the help of Woltlab as well, hopefully I can find a solution to this.

  • Kevin
13

If you add this at the top of your Caddyfile, it should reveal some more logs that might help to see what the rewritten path looks like:

{
    debug
}
14

Okay, that was a great tip francislavoie,

I have found out that the path that it’s currently being rewritten to is the following:

forum/index.php?core-rewrite-test%2F

%2F is the backslash character, but for the rest it’s not filling in anything from the URL. It simply stops there.

From the debugging logs, this is the main output:

May 14 19:52:09 devona caddy[622]: {"level":"debug","ts":1589485929.9476984,"logger":"http.handlers.rewrite","msg":"rewrote request","request":{"method":"GET","uri":"/core-rewrite-test/?uuidHash=e6d1665ee15da70311b50f1cf76b11649212b9efc02d2c97619b6ea3be91199a",">
May 14 19:52:09 devona caddy[622]: {"level":"debug","ts":1589485929.948156,"logger":"http.handlers.rewrite","msg":"rewrote request","request":{"method":"GET","uri":"/forum/core-rewrite-test/?uuidHash=e6d1665ee15da70311b50f1cf76b11649212b9efc02d2c97619b6ea3be9119>
May 14 19:52:09 devona caddy[622]: {"level":"debug","ts":1589485929.9481826,"logger":"http.reverse_proxy.transport.fastcgi","msg":"roundtrip","request":{"method":"GET","uri":"index.php?core-rewrite-test%2F}","proto":"HTTP/2.0","remote_addr":"188.189.112.68:14675>
May 14 19:52:09 devona caddy[622]: {"level":"debug","ts":1589485929.948522,"logger":"http.reverse_proxy.transport.fastcgi","msg":"roundtrip","request":{"method":"GET","uri":"forum/index.php?core-rewrite-test%2F}","proto":"HTTP/2.0","remote_addr":"188.189.112.68:>
May 14 19:52:10 devona caddy[622]: {"level":"debug","ts":1589485930.017703,"logger":"http.handlers.reverse_proxy","msg":"upstream roundtrip","upstream":"unix//run/php/php7.4-fpm-gwl_test.sock","request":{"method":"GET","uri":"index.php?core-rewrite-test%2F}","pr>
May 14 19:52:10 devona caddy[622]: {"level":"error","ts":1589485930.0184588,"logger":"http.log.access.log0","msg":"handled request","request":{"method":"GET","uri":"/core-rewrite-test/?uuidHash=e6d1665ee15da70311b50f1cf76b11649212b9efc02d2c97619b6ea3be91199a","p>
May 14 19:52:10 devona caddy[622]: {"level":"debug","ts":1589485930.0584798,"logger":"http.handlers.reverse_proxy","msg":"upstream roundtrip","upstream":"unix//run/php/php7.4-fpm-gwl_test.sock","request":{"method":"GET","uri":"forum/index.php?core-rewrite-test%2>
May 14 19:52:10 devona caddy[622]: {"level":"error","ts":1589485930.0593143,"logger":"http.log.access.log0","msg":"handled request","request":{"method":"GET","uri":"/forum/core-rewrite-test/?uuidHash=e6d1665ee15da70311b50f1cf76b11649212b9efc02d2c97619b6ea3be9119>

I suspect the regex itself is working correct, but my rewriting bit is off:

       rewrite @woltlab {http.regexp.woltlab.1}index.php?{http.regexp.woltlab.2}

I have already removed the / between the {http.regexp.woltlab.1} and index.php since it introduced a double backslash. I think I’m looking over an obvious solution here :confused:

15

That’s forward slash actually! / is forward slash (or simply “slash”), \ is backslash

Your logs seem truncated, I think you copied them from some CLI program that doesn’t wrap lines. Could you post the full logs?

16

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.

17

FYI you may want to follow the discussion here, which is an attempt to configure the same app: