V2: redirect /path to /path/ with assets

Help
1

1. My Caddy version (caddy version):

v2.0.0-beta.14 h1:QX1hRMfTA5sel53o5SuON1ys50at6yuSAnPr56sLeK8=

2. How I run Caddy:

a. System environment:

  • Ubuntu 16.04
  • systemd 219
  • php 7.0

b. Command:

systemctl restart caddy

c. Service/unit/compose file:

[Unit]
Description=Caddy HTTP/2 web server
Documentation=https://caddyserver.com/docs
After=network-online.target
Wants=network-online.target systemd-networkd-wait-online.service

; Do not allow the process to be restarted in a tight loop. If the
; process fails to start, something critical needs to be fixed.
StartLimitIntervalSec=14400
StartLimitBurst=10

[Service]
Restart=on-abnormal

; User and group the process will run as.
User=www-data
Group=www-data

; Letsencrypt-issued certificates will be written to this directory.
Environment=CADDYPATH=/etc/ssl/caddy

; Always set "-root" to something safe in case it gets forgotten in the Caddyfile.
ExecStart=/usr/local/bin/caddy2 run --config=/etc/caddy/Caddyfile > /var/log/caddy.log
ExecReload=/bin/kill -USR1 $MAINPID

; Use graceful shutdown with a reasonable timeout
KillMode=mixed
KillSignal=SIGQUIT
TimeoutStopSec=5s

; Limit the number of file descriptors; see `man systemd.exec` for more limit settings.
LimitNOFILE=1048576
; Unmodified caddy is not expected to use more than that.
LimitNPROC=512

; Use private /tmp and /var/tmp, which are discarded after caddy stops.
PrivateTmp=true
; Use a minimal /dev (May bring additional security if switched to 'true', but it may not work on Raspberry Pi's or other devices, so it has been disabled in this dist.)
PrivateDevices=false
; Hide /home, /root, and /run/user. Nobody will steal your SSH-keys.
ProtectHome=true
; Make /usr, /boot, /etc and possibly some more folders read-only.
ProtectSystem=full
; … except /etc/ssl/caddy, because we want Letsencrypt-certificates there.
;   This merely retains r/w access rights, it does not add any new. Must still be writable on the host!
ReadWritePaths=/etc/ssl/caddy
ReadWriteDirectories=/etc/ssl/caddy

; The following additional security directives only work with systemd v229 or later.
; They further restrict privileges that can be gained by caddy. Uncomment if you like.
; Note that you may have to add capabilities required by any plugins in use.
;CapabilityBoundingSet=CAP_NET_BIND_SERVICE
;AmbientCapabilities=CAP_NET_BIND_SERVICE
;NoNewPrivileges=true

[Install]
WantedBy=multi-user.target

d. My complete Caddyfile or JSON config:

www.zhangshenjia.com {
redir https://zhangshenjia.com{uri}
}

zhangshenjia.com
{
root * /www/zhangshenjia.com
encode zstd gzip
reverse_proxy /gg localhost:10000
php_fastcgi 127.0.0.1:9000
file_server
}

3. The problem I’m having:

I have a directory name [next] in my webroot, and there is a [index.php] in it. When I visit [/next/] , everything looks right. But when I visit [/next] , the page shows but assets (js and imgs) are failed to load, since the css path become [/css/style.css], but [/next/css/style.css] is expected.

Caddy V2 document said that php_fastcgi will add trailing slash for directory requests ([php_fastcgi (Caddyfile directive) — Caddy Documentation]). But obviously, it doesn’t work in my case.

4. What I already tried:

I search a lot on google and caddy community for hours, but all solutions is for V1 but not V2.

I found this issue [V2: redirect /path to /path/index.php with assets],It seems that this problem had been fixed severals month ago, and merged in 2.0beta2 [v2: php_fastcgi: 301 redirect if path is a directory (add trailing slash at the end) · Issue #2752 · caddyserver/caddy · GitHub]; but why it doesn’t work in 2.0beta14?

2

That’s strange. I tried to replicate your issue, and it’s working as expected for me:

$ tree
.
├── Caddyfile
├── caddy2_beta14_linux_amd64
└── site
    ├── index.php
    └── next
        └── index.php

Caddyfile:

:8111 {
        root * /home/francis/caddy/site

        encode gzip

        php_fastcgi unix//var/run/php/php7.4-fpm.sock

        file_server
}

I run caddy with: ./caddy2_beta14_linux_amd64 start

Then I make a request:

$ curl -v localhost:8111/next
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 8111 (#0)
> GET /next HTTP/1.1
> Host: localhost:8111
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 308 Permanent Redirect
< Location: /next/
< Server: Caddy
< Date: Sat, 22 Feb 2020 19:38:16 GMT
< Content-Length: 0
<
* Connection #0 to host localhost left intact

As you can see, it triggers a redirect to /next/.

1 Like
3

I found it works after I remove this line below from Caddyfile, and visit (/next) by Chrome Incognito mode:

try_files {path} {path}/index.php /index.php?{query}

While I add this line back to Caddyfile, then redirect keep failed.

I thought I had tried to removed this line which came from V1 config, maybe I’m exhausted last night.

Sorry for this post.

2 Likes
4

Glad you found the solution! But I don’t see that line in your Caddyfile? Where did it come from? I just want to make sure I understand.

5

Review what I actually done that night:

  1. Deploy Wordpress & v2ray on caddy V1, I bind two domains on one site. Then I decided to upgrade to caddy V2.

  2. Deploy on caddyV2 with a brand new Caddyfile, I test the redirection on domain A, everything works.

  3. Try to deploy v2ray & websocket, because there is no any example for V2, I use V1 config and fix all config parse errors by remove lines. At that time, Caddyfile should be:

(php) {
encode zstd gzip
php_fastcgi 127.0.0.1:9000
try_files {path} {path}/index.php /index.php?{query}
file_server
}

zhangshenjia.com
{
root * /www/zhanshenjia.com
reverse_proxy /gg localhost:10000
import php
}

shenjia.de
{
root * /www/zhangshenjia.com
reverse_proxy /gg localhost:10000
import php
}

  1. I found the direction works on domain A (zhangshenjia.com), but not work on domain B (shenjia.de). I checked Caddyfile and make sure the part of them were exactly same!

  2. I tried to debug Caddyfile line by line, but domain A keep working, domain B keep failing. I tried Chrome Incognito mode and found out that direction failed on both domain A & B. I got completely confused.

My browser cached the 301 redirection on domain A I visited before I migrated config from V1, that’s what I figured out next day.

  1. A hour later, I’m so exhausted that I wrote this post at 3:00 AM, then go to sleep. I remove this line below because it looks useless in test, that’s my fault.
try_files {path} {path}/index.php /index.php?{query}
  1. Next morning, I saw your reply, and tried to deploy a brand new site on another VPS, and found out that direction works with basic config. Inspire of you, I use curl -v to test direction instead of browser, and finally figure out everything works without that single line and browser cache.

And the lesson is : NEVER test redirection with browser cache. BTW, I think the feature (automatically add trailing slash on directory) should be mentioned in documents.

THX for your time and inspiration!

2 Likes
6

Technically, it is mentioned on php_fastcgi (Caddyfile directive) — Caddy Documentation if you read the “Expanded form” section. The first line there has a comment:

# Add trailing slash for directory requests

Thanks for the writeup on what happened! It’s very much appreciated!

7

@shenjia Wow, that is an excellent writeup. Thank you for documenting what you did! I bet it will help future searchers find their answers. In fact, I think experiences like yours are common enough (based on interactions here through the years) that I may link to yours in the future to illustrate the kind of effort it takes to solve problems like this.

I really appreciate you writing this up, it’s a great slice of knowledge!

8

What you said is truth, but I can hardly understand why it doesn’t work with php_fastcgi and this line below, since the php_fastcgi include this line as the documents show:

try_files {path} {path}/index.php /index.php?{query}

If I put two same lines in a nginx config, it will be fine. But in caddy V2, [ABC] + [B] doesn’t work, that’s strange.

It looks like that php_fastcgi include so many black magic, make it complex to understand. I prefer the V1 config, because I know what will make out by my config. But in V2? god bless me…

1 Like
9

We can do an experiment, that will help understand why that is the case, and demonstrate why the v1 config is broken: https://www.diffchecker.com/D4mygNbW (link expires in 1 month)

On the left is the working config without the extra try_files. On the right is the same config with extra try_files. (I’ve used only one site definition from your Caddyfile, to simplify things and reduce duplication for this demo.)

You will see that with the extra try_files, it will attempt to perform rewrites before any logic concerning reverse proxying / PHP is even considered. This can be extremely useful if you need to change a request before we start processing it as a PHP request.

The v1 Caddyfile is broken because it cannot express this logic without hacky hacks.

In the v2 Caddyfile, this is a piece of cake.

Yeah, there’s some “magic” under there, but that’s the point of the Caddyfile. That’s why people love it. If you don’t like what it hides away, you can use the “expanded form” of that directive as @francislavoie pointed out – we do document it – and even change it to your liking. We just made the php_fastcgi directive work for most modern PHP apps for convenience, which is the point of the Caddyfile.

You can see that the php_fastcgi directive does not correlate to a single handler in the route! It cannot be expressed in a single handler, because frankly, PHP is a needy, high-maintenance lover. :stuck_out_tongue: The web server has to do a lot of work and do it carefully to satisfy the expectations of modern PHP apps if the relationship is going to work out.

Significantly, the several handlers injected by the single php_fastcgi directive are kept together as if they were wrapped in a route directive (I should probably add that to the docs… the route directive just didn’t exist yet when I wrote the docs page for php_fastcgi). Handlers outside an explicit route will be sorted into a consistent, deterministic order. This means you can define rewrites that take place before proxying and that won’t interfere with PHP logic. But you can’t really do that easily in v1, since all the rewrites get mushed together.

So, we’ve solved two problems in Caddy 2: 1) made PHP even easier to proxy (don’t think too hard, the Caddyfile is really simple), and 2) made certain (common) routing logic possible without hacks.

2 Likes
10

Actually, I tried “expanded form” which you guys pointed out. But it doesn’t work as the documents said : " The php_fastcgi directive is the same as the following configuration". It’s different!

You can try this config below (copy from V2 document, only replaced <php-fpm_gateway>). /next -> /next/ redirection doesn’t work:

zhangshenjia.com
{
root * /www/zhanshenjia.com
# Add trailing slash for directory requests
@canonicalPath {
    file {
        try_files {path}/index.php
    }
    not {
        path */
    }
}
redir @canonicalPath {path}/ 308

# If the requested file does not exist, try index files
try_files {path} {path}/index.php index.php

# Proxy PHP files to the FastCGI responder
@phpFiles {
    path *.php
}
reverse_proxy @phpFiles 127.0.0.1:9000 {
    transport fastcgi {
        split .php
    }
}
file_server
}

I thought the example in documents maybe not up to date? Otherwise, there’re more logic behind php_fastcgi?

11

Try running caddy adapt --pretty on both versions of your config to compare what they’re doing.

Like @matt said, I think you might need to wrap the entire PHP stuff inside of a route { } block

12

Yeah, gotta update that. Wrap the “expanded form” in route { ... } and see what happens.

13

this way?

zhangshenjia.com
{
    root * /var/www
    route {
        @canonicalPath {
            file {
                try_files {path}/index.php
            }
            not {
                path */
            }
        }

        @phpFiles {
            path *.php
        }
        redir @canonicalPath {path}/ 308
        try_files {path} {path}/index.php index.php
        reverse_proxy @phpFiles unix//run/php/php7.0-fpm.sock {
            transport fastcgi {
                split .php
            }
        }
    }
    file_server
}

It doesn’t work:

caddy2 adapt --pretty --config=Caddyfile
adapt: parsing caddyfile tokens for 'route': Caddyfile:5 - Error during parsing: unrecognized directive: @canonicalPath
14

I think canonicalPath might need to be defined outside of the route?

15

Yeah unfortunately matchers are special and need to be top-level in a site block.

16

unfortunately matchers are special and need to be top-level in a site block.

I can’t find those notice in documents… so, like this?

@canonicalPath {
    file {
        try_files {path}/index.php
    }
    not {
        path */
    }
}

@phpFiles {
    path *.php
}

zhangshenjia.com
{
    root * /var/www
    route {
        redir @canonicalPath {path}/ 308
        try_files {path} {path}/index.php index.php
        reverse_proxy @phpFiles unix//run/php/php7.0-fpm.sock {
            transport fastcgi {
                split .php
            }
        }
    }
    file_server
}

It doesn’t work, again.

caddy2 adapt --pretty --config=Caddyfile
adapt: Caddyfile:3: unrecognized directive: file
17

No, inside the site block, after your domain name

18

https://www.diffchecker.com/HlkHF2Df

the left side config is :

 zhangshenjia.com
{
    root * /var/www
    php_fastcgi unix//run/php/php7.0-fpm.sock
    file_server
}

the right side config is:

zhangshenjia.com
{
    @canonicalPath {
        file {
            try_files {path}/index.php
        }
        not {
            path */
        }
    }

    @phpFiles {
        path *.php
    }
    root * /var/www
    route {
        redir @canonicalPath {path}/ 308
        try_files {path} {path}/index.php index.php
        reverse_proxy @phpFiles unix//run/php/php7.0-fpm.sock {
            transport fastcgi {
                split .php
            }
        }
    }
    file_server
}

There’s still a little different, but anyway it works. THX you guys!

BTW, I hope you could replace TAB with one or two spaces in caddy adapt --pretty to make the JSON readable.

19

I think you can probably just pipe the output to sed:

$ caddy adapt --pretty | sed 's/\t/  /g'

Typing from a phone so your mileage may vary

3 Likes
20

I noticed that this page hasn’t been updated yet: php_fastcgi (Caddyfile directive) — Caddy Documentation

You should put a route {} in the “Expanded form” part to make sure users can understand how php_fastcgi exactly work and make it possible to customize.

1 Like