Rewrite path to index.php if no file or directory exists

[senshi]

1. The problem I’m having:

I’m currently trying to migrate an existing, pretty convoluted and very old (2005) webserver ecosystem away from apache to caddy. Caddy will have to serve a single site with many subdomains, with different php-fpms per site as well as static files mixed in.

I’m currently stumped by a probably simple issue: The homepage uses a very old (almost) SPA layout, relying on a central index.php at root level, and only serving anything “as-is” that exists as either files (in that case, serve statically, unless hidden/forbidden) or, if a folder, serve its own index.html/index.php file (if exists).
To address specific subcontent templates, the index.php needs to conditionally rewrite paths as a query params (with exceptions):
E.g. a request to
realitymod.com/contact
should be redirected to
realitymod.com/index.php?page2=contact
Because there is no file or folder called “contact”.

A request to
realitymod.com/prspy or realitymod.com/mapgallery
should NOT be redirected - or rather, follow standard index rules by serving their index.html - , because those folders exist and have their own index.html.

Later, there will be a realitymod.com/forum subfolder that will serve a php-based forum, so will need to serve its own index.php without interference from the homepage’s index.php.

The equivalent .htaccess rules that made this particular part work on the previous Apache (I know, yuck) look like this:

RewriteCond %{REQUEST_FILENAME} !-f # If URI doesn't lead to a file
RewriteCond %{REQUEST_FILENAME} !-d # If URI doesn't lead to a dir
RewriteRule ^media/screenshots-([0-9]+)/?$  index.php?page2=screenshots&p=$1 [L] # special treatment of media/screenshots
RewriteRule ^([a-zA-Z0-9_-]+)/?$ index.php?page2=$1 [END,QSA] # Perform the actual rewrite

I tried around a bunch with rewrite and try_files (see my Caddyfile below for some attempts), but couldn’t get close to what I actually want to happen and usually just made it worse. I’m afraid I fail to understand some fundamentals regarding how rewrite/redirects are handled in Caddy, despite searching the web, this forum and reading the docs over and over again.

As a newbie, I also appreciate any other recommendations or suggestions on unrelated issues with my configs that you spot that could be improved.

2. Error messages and/or full log output:

I don’t have an explicit “error message” to show for this scenario, sadly. My configs just don’t work as expected.

{"level":"info","ts":"2024-11-07T11:17:09.181Z","logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc000a2b380"}
{"level":"info","ts":"2024-11-07T11:17:09.182Z","logger":"http.auto_https","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443}
{"level":"info","ts":"2024-11-07T11:17:09.182Z","logger":"http.auto_https","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
{"level":"info","ts":"2024-11-07T11:17:09.186Z","logger":"tls.cache.maintenance","msg":"stopped background certificate maintenance","cache":"0xc000a2b380"}
{"level":"info","ts":"2024-11-07T11:18:43.269Z","logger":"http.auto_https","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443}
{"level":"info","ts":"2024-11-07T11:18:43.270Z","logger":"http.auto_https","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
{"level":"debug","ts":"2024-11-07T11:18:43.270Z","logger":"http.auto_https","msg":"adjusted config","tls":{"automation":{"policies":[{"subjects":["redmine.realitymod.com","realitymod.com"]},{"subjects":["ingame.realitymod.com","www.realitymod.com"]},{}]}},"http":{"servers":{"remaining_auto_https_redirects":{"listen":[":80"],"routes":[{},{}],"logs":{"logger_names":{"ingame.realitymod.com":["log1"],"redmine.realitymod.com":["log2"],"www.realitymod.com":["log0"]},"skip_hosts":["realitymod.com"]}},"srv0":{"listen":[":443"],"routes":[{"handle":[{"handler":"subroute","routes":[{"handle":[{"handler":"reverse_proxy","upstreams":[{"dial":"localhost:9292"}]}]}]}],"terminal":true},{"handle":[{"handler":"subroute","routes":[{"handle":[{"handler":"vars","root":"/opt/ingame/"},{"handler":"file_server","hide":["/etc/caddy/sites/ingame.conf"]}]}]}],"terminal":true},{"handle":[{"handler":"subroute","routes":[{"handle":[{"handler":"vars","root":"/var/www/vhosts/realitymod.com/html"}]},{"handle":[{"close":true,"handler":"static_response","status_code":403}],"match":[{"path":["/config.php","/common.php","*.log","*.htaccess"]}]},{"handle":[{"handler":"static_response","headers":{"Location":["{http.request.orig_uri.path}/"]},"status_code":308}],"match":[{"file":{"try_files":["{http.request.uri.path}/index.php"]},"not":[{"path":["*/"]}]}]},{"handle":[{"handler":"rewrite","uri":"{http.matchers.file.relative}"}],"match":[{"file":{"split_path":[".php"],"try_files":["{http.request.uri.path}","{http.request.uri.path}/index.php","index.php"]}}]},{"handle":[{"handler":"reverse_proxy","transport":{"protocol":"fastcgi","split_path":[".php"]},"upstreams":[{"dial":"unix//run/php/php8.3-fpm-homepage.sock"}]}],"match":[{"path":["*.php"]}]},{"handle":[{"handler":"file_server","hide":["forum","lcp",".git",".gitignore","*.log","./","/etc/caddy/sites/homepage.conf"]}]}]}],"terminal":true},{"handle":[{"handler":"subroute","routes":[{"handle":[{"handler":"static_response","headers":{"Location":["https://www.{http.request.host}{http.request.uri}"]},"status_code":302}]}]}],"terminal":true}],"tls_connection_policies":[{}],"automatic_https":{},"logs":{"logger_names":{"ingame.realitymod.com":["log1"],"redmine.realitymod.com":["log2"],"www.realitymod.com":["log0"]},"skip_hosts":["realitymod.com"]}}}}}
{"level":"info","ts":"2024-11-07T11:18:43.269Z","logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc00053e700"}
{"level":"info","ts":"2024-11-07T11:18:43.271Z","logger":"tls.cache.maintenance","msg":"stopped background certificate maintenance","cache":"0xc00053e700"}

3. Caddy version:

v2.8.4 h1:q3pe0wpBj1OcHFZ3n/1nl4V4bxBrYoSoab7rL9BMYNk=

4. How I installed and ran Caddy:

Following this guide:

Build from source - Caddy Documentation

Installed the ubuntu caddy package, then set up the alternative for a custom binary that includes the dns-cloudflare module.

a. System environment:

Ubuntu 20.04 adm64, systemd, “bare metal”

b. Command:

Does not apply

c. Service/unit/compose file:

# caddy.service
#
# For using Caddy with a config file.
#
# Make sure the ExecStart and ExecReload commands are correct
# for your installation.
#
# See https://caddyserver.com/docs/install for instructions.
#
# WARNING: This service does not use the --resume flag, so if you
# use the API to make changes, they will be overwritten by the
# Caddyfile next time the service is restarted. If you intend to
# use Caddy's API to configure it, add the --resume flag to the
# `caddy run` command or use the caddy-api.service file instead.

[Unit]
Description=Caddy
Documentation=https://caddyserver.com/docs/
After=network.target network-online.target
Requires=network-online.target

[Service]
Type=notify
User=caddy
Group=caddy
ExecStart=/usr/bin/caddy run --environ --config /etc/caddy/Caddyfile
ExecReload=/usr/bin/caddy reload --config /etc/caddy/Caddyfile --force
TimeoutStopSec=5s
LimitNOFILE=1048576
PrivateTmp=true
ProtectSystem=full
AmbientCapabilities=CAP_NET_ADMIN CAP_NET_BIND_SERVICE

[Install]
WantedBy=multi-user.target

d. My complete Caddy config:

{
        debug
        log default {
                output file /var/log/caddy/caddy.log
                format json {
                        time_format iso8601
                }
        }
}

import sites/*.conf

sites/homepage.conf

realitymod.com {
	redir https://www.{host}{uri}
}

www.realitymod.com {
	tls {
		dns cloudflare {env.cloudflare_api_token}
	}
	root * /var/www/vhosts/realitymod.com/html

	@redirects {
		not file
		# not path /prspy /mapgallery
	}

	php_fastcgi unix//run/php/php8.3-fpm-homepage.sock {
		# try_files @redirects /index.php?page2={file}
		# try_files {path} {path}/index.php?page2={file}
	}
	file_server {
		hide forum lcp .git .gitignore *.log ./
	}

	log {
		output file /var/www/vhosts/realitymod.com/html/caddy.log
		format json {
                        time_format iso8601
                }
	}

	@blocked {
		path /config.php /common.php *.log *.htaccess
	}
	respond @blocked 403 {
		close
	}

	# try_files * /index.php?page2={query}
}

sites/ingame.conf (not relevant to the problem)

ingame.realitymod.com {
	tls {
		dns cloudflare {env.cloudflare_api_token}
	}

	root * /opt/ingame/
	file_server

	log {
		output file /var/log/caddy/ingame-access.log
	}
}

sites/redmine.conf (not relevant to the problem)

redmine.realitymod.com {
	tls {
		dns cloudflare {env.cloudflare_api_token}
	}

	reverse_proxy http://localhost:9292

  log {
    output file /var/log/caddy/redmine.realitymod.com-access.log
  }
}

5. Links to relevant resources:

[senshi]

The closest I can get to the desired behavior is by using

@redirects {
	not file
}
rewrite @redirects /index.php?page2={file}

But this leads to the subfolders not working properly and instead bouncing back to the homepage. I would need something like “not dir”, probably?

[francislavoie]

@senshi

I’m currently stumped by a probably simple issue: The homepage uses a very old (almost) SPA layout, relying on a central index.php at root level

Actually, that’s how modern PHP apps are meant to work. That’s how php_fastcgi works by default. Read here:

php_fastcgi (Caddyfile directive) - Caddy Documentation

What I recommend is doing the routing in your PHP app. You can make changes to your index.php file. Check the original request URI in $_SERVER['REQUEST_URI'] and do what you need from there.

@senshi

Later, there will be a realitymod.com/forum subfolder

I strongly recommend serving that from a subdomain instead.

The "subfolder problem", OR, "why can't I reverse proxy my app into a subfolder?"

Have you ever tried to reverse proxy an app into its own little subfolder of your domain name? Makes things neat, doesn’t it? Using example.com/foo/ for one app, example.com/bar/ for another. If you’re coming here from one of the selfhosted communities, you might be thinking along the lines of example.com/sonarr/, example.com/radarr/ etc. Chances are, you’ve tried some configuration along these lines: example.com { redir /sonarr /sonarr/ handle_path /sonarr/* { reverse_proxy localhost…

[senshi]

Thanks for the answer!
I understand that’s how modern php apps work as well, it’s just that this one is not that, which is why it was set up with these various subfolders among other, less important quirks. This also makes it very fragile and I try to mess with it as little as possible. We’re working on a proper, full replacement so we can get rid of this, it’s just not there yet.
This old homepage also has some intermingling (requiring files from etc.) with the “forum” folder, which is why I cannot easily migrate it to a subdomain (believe me, I wish I could). The forum subfolder/path already exists, I just do not want to enable this big part before I have sorted the current issues with the other subfolders such as prspy and mapgallery.

If I can’t get a nice solution going inside Caddy for this routing issue, I’ll dive into the index.php, but I’d really rather not. I also know there’s more routing and rewrite issues waiting for me once I migrate more services with arcane .htaccess rules, so I am hoping to improve my apparently lacking knowledge in that area beforehand.

[senshi]

After a good night’s sleep and a revisit to the docs, I found the answer, which was as simple as I have come to expect from Caddy.
The easiest solution was to add the directory exemptions maually:

	@redirects {
		not file
		not path /prspy* /mapgallery* /manual*
	}

	php_fastcgi unix//run/php/php8.3-fpm-homepage.sock
	rewrite @redirects /index.php?page2={file}

But in another scenario where I could reliably exclude directories, changing the matcher to not files {path} {path}/ worked perfectly. I just had to also add an exemption for an empty path so a request to the root/empty domain would properly call the index.php.