Caddyfile help - Removing subdirectory / subfolder from url

[misterdinus]

1. Caddy version (caddy version):

Caddy 2.4.3 w/ caddy-dns cloudflare

2. How I run Caddy:

a. System environment:

Dietpi 7.4 on Raspberry Pi 4B, systemd running Caddy as a service (using package alongside cloudflare custom package following documentation guide).

b. Command:

sudo systemctl start caddy

c. Service/unit/compose file:

N/A - Caddy runs fine through systemd

d. My complete Caddyfile or JSON config:

# Using 01DNS method for TLS as I am behind a CGNAT. 
{
	acme_dns cloudflare APIKEY
}

# Proxy raspberry pi DNS blocker webui to 'adguard.postulat.es'. Domain mapped to 10.0.0.2 Caddy server for local access. 
adguard.postulat.es {
# Adguard is running on same raspberry pi server as Caddy.
	reverse_proxy http://localhost:8083
}

# Proxy UDM router webui to 'unifi.postulat.es'. Domain mapped to 10.0.0.2 Caddy server for local access. 
unifi.postulat.es {
	reverse_proxy https://10.0.0.1 {
		transport http {
			tls_insecure_skip_verify
		}
	}
}

# Proxy QNAP NAS admin webui to 'qnap.postulat.es'. Domain mapped to 10.0.0.2 Caddy server for local access.  
qnap.postulat.es {

#QNAP NAS is on 10.0.0.222 and is running several docker containers. Port 8080 is webui for system. 
	reverse_proxy http://10.0.0.222:8080
}

# Attempt to proxy various QNAP containers to subdirectories of root domain rather than subdomains. Domain is mapped to 10.0.0.2 Caddy server for local access.
postulat.es {
# Method for radarr and sonarr taken from Caddyfile suggestion found on Google.  Appears to work using 'base url' change.  
	redir /radarr /radarr/
	redir /sonarr /sonarr/
# I redir-ed qnap to copy the approach for radarr and sonarr.
	redir /qnap /qnap/

	reverse_proxy /radarr/* {
		to http://10.0.0.222:7878
	}

	reverse_proxy /sonarr/* {
		to http://10.0.0.222:8989
	}

	***[QNAP Reverse Proxy - see below for the two attempts I have used to do this]***
}

3. The problem I’m having:

A. Using Caddy to access various home server applications

I own a domain which I have mostly mapped to the local (10.0.0.2) IP address of my Raspberry Pi running Caddy. I am running a reverse proxy and can currently access several applications through subdomains, listed above.

I am separately considering exposing some Caddy services through a password-protected Cloudflare/Argo tunnel (because we have CGNAT I can’t use Caddy directly with port forwarding), but that is secondary to just having my various services available under a domain mapped to local IPs.

B All working fine for services that let me modify base urls

I would like to expose the various applications running in docker on my QNAP NAS on the root postulat.es domain rather than subdomains. I found a few examples of how to do this online and have got Sonarr and Radarr working by copying these examples and making the limited necessary adjustments.

I also do not anticipate any issues with a few other containers, as their webuis or configuration files all let me adjust the ‘base url’ for those services. All of these services originally run at the domain level
(example. com), but can be altered so they run at [serverip:port]/service/. As I understand it, this means that there are no issues with me using postulat.es/service, because they map across properly (sorry that my terminology is terrible).

C QNAP webui does not work and I can’t change the base url

My QNAP NAS has a web ui that lets me manage updates and generally administer the server. The ui’s login page and operations load at 10.0.0.222:8080/cgi-bin/, and navigating to 10.0.0.222:8080 automatically redirects to 10.0.0.222:8080/cgi-bin. There does not seem to be any way to change the base url for this service as can be done with Sonarr and Radarr. As explained in part 5 below, I have tried two attempts to forward the QNAP web ui, but neither work.

I would like to know if it is possible to get this working so that navigating to postulat.es/qnap/ takes me to the QNAP web ui. I am currently using qnap.postulat.es/ to get to the webui, but I would prefer to have all my docker-related services exposed at the top-level of the domain.

4. Error messages and/or full log output:

N/A I have not received any error messages.

5. What I already tried:

I originally tried to just copy the approach for Sonarr and Radarr that I used, which I found from Google searches.

	reverse_proxy /qnap/* {
		to http://10.0.0.222:8080
	}

This produces an ‘page not found’ error message (see bottom of post) that I think comes from the QNAP webui trying to load its interface but (I assume) not being able to find files where it expects them. The error message looks like this:

From looking online, this seems to have been a known issue. One poster on this forum dubbed this the ‘subfolder problem’ and noted there are three ways to get around this (their post is from June 2020):

1. Fix ‘upstream’ by using base urls. This isn’t an option as the QNAP webui and OS doesn’t allow you to change the base url, from the searches I’ve done on Google.

2. Use a subdomain instead of subfolders. I am doing this now, but am posting to see if a subdirectory approach is possible as I would prefer that.

3. Using HTML filtering or rewriting. I am struggling to understand how this works and do not fully understand exactly what to put in my Caddyfile. It’s also not clear to me if there was a fix at the time they wrote this, or if it was under development, as their post notes that there is a Caddy module being developed, but I do not see how it would remove the /qnap/ from my url but rather just replace it.

I saw several references online to settings in Caddy v1 that seem to suggest the “without” directive, but read that these were considered ‘hacks’ and are ‘completely unnecessary’ as of Caddy v2. I did find a separate blog post that provides tips on upgrading from Caddy v1 to Caddy v2, which seems to suggest some directives that can replace the “without” directive (my emphasis):

Requests to https://example.com/api/foo/bar/ is redirected to https://backend.com/api/foo/bar/.
To remove the path prefix:

#v1
proxy /api https://backend.com {
  without /api
}

Requests to https://example.com/api/foo/bar/ is redirected to https://backend.com/foo/bar/.
v2 doesn’t have without directive, instead you need to use route the request and remove the prefix using uri strip_prefix:

#v2
route /api/* {
  uri strip_prefix /api
  reverse_proxy https://backend.com
}

v2.1 adds handle_path directive which integrates prefix stripping:

#v2.1
handle_path /api/* {
  reverse_proxy https://backend.com
}

I tried a version of the v2 approach (the 2.1 also doesn’t work):

	route /qnap/* {
		uri strip_prefix /qnap
		reverse_proxy 10.0.0.222:8080
	}

But this just gives me a blank page at postulat.es/redirect.html?count=0.13713111451312998.

Is there any hope for me?

This seems to have been an issue for other people, and is at least a significant enough happening with reverse proxy use that several web self-hosted web applications let you specify custom base urls. I have tried my best, but my lack of technical knowledge along with a lot of essentially legacy references to (possible?) solutions using Caddy v1 which are no longer working means I am at a loss as to how to get the QNAP webui to show up in a subfolder.

I would be extremely grateful if the community could let me know if there’s something I’m missing here, or if this is just not feasible at the moment.

6. Links to relevant resources:

I’ve posted links to the blogposts I read above. There were a good dozen and more of posts on this forum and other blogs online, but none of them were of help to me (this is probably also due to my inability to make sense of them; I am trying to learn.

The picture of the error message I get when running a ‘vanilla’ reverse proxy configuration mentioned above is:

Thanks

Thanks a lot for taking the time to read, and also in advance in case anyone is able to help with this. Thanks to the development community for making Caddy: I am not very technical at all but have been blown away by how easy to use Caddy is otherwise from this admittedly fairly minor issue.

[francislavoie]

Yeah, this is a common issue.

The "subfolder problem", OR, "why can't I reverse proxy my app into a subfolder?"

Have you ever tried to reverse proxy an app into its own little subfolder of your domain name? Makes things neat, doesn’t it? Using example.com/foo/ for one app, example.com/bar/ for another. If you’re coming here from one of the selfhosted communities, you might be thinking along the lines of example.com/sonarr/, example.com/radarr/ etc. Chances are, you’ve tried some configuration along these lines: example.com { redir /sonarr /sonarr/ handle_path /sonarr/* { reverse_proxy localhost…

The best solution is to just use subdomains for each service.

If you must, and you can get it to work with the service via base path config etc, the handle_path directive is the preferred one to use (shortest config).

[misterdinus]

Thank you Francis!

Do you have any suggestions on what may be going wrong with the handle_path directive approach? When I try the following handle_path directive:

route /qnap/* {
  uri strip_prefix /qnap
  reverse_proxy http://10.0.0.222:8080
}

I end up with http://postulat.es/redirect?count=0.19290717531145218 in my address bar. This is a step further than the normal reverse_proxy method where I get what looks like a 404 error from the QNAP server. On the upstream server itself, this server first loads a page contains the following javascript:

<html style="background:#007cef">
<head>
<meta http-equiv="expires" content="0">
<script type='text/javascript'>
pr=(document.location.protocol == 'https:') ? 'https' : 'http';
pt=(location.port == '') ? '' : ':' + location.port;
redirect_suffix = "/redirect.html?count="+Math.random();
if(location.hostname.indexOf(':') == -1)
{
location.href=pr+"://"+location.hostname+pt+redirect_suffix;
}
else	//could be ipv6 addr
{
var url = "";
url=pr+"://["+ location.hostname.replace(/[\[\]]/g, '') +"]"+pt+redirect_suffix;
location.href = url;
}
</script>
</head>
<body>
</body>
</html>

The redirect.html mentioned above contains:

<!DOCTYPE html>
<html>
<head>
<meta http-equiv="expires" content="0">
</head>
<body style='display:none;'>
</body>
<script type='text/javascript'>
function goMainPage(pass){
try{
if (document.getElementById("remindValue").checked){
var expires = new Date();
expires.setTime(expires.getTime() + 365 * 24 * 60 * 60 * 1000);
document.cookie="skip_IE_detect=1; expires="+expires.toGMTString();
}
}catch(e){

}

pr=(document.location.protocol == 'https:') ? 'https' : 'http';
pt=(location.port == '') ? '' : ':' + location.port;
redirect_suffix = "/cgi-bin/QTS.cgi?count="+Math.floor(Math.random()*1000000);
if(location.hostname.indexOf(':') == -1)
{
location.href=pr+"://"+location.hostname+pt+redirect_suffix;
}
else	//could be ipv6 addr
{
var url = "";
url=pr+"://["+ location.hostname.replace(/[\[\]]/g, '') +"]"+pt+redirect_suffix;
location.href = url;
}
}
goMainPage();
</script>
</html>

It seems that the QNAP webui does all the rest of its ‘work’ from the cgi-bin folder once redirected there, so I am hopeful that once the redirect is solved the page will work properly. The ultimate page is either http://10.0.0.222:8080/cgi-bin/QTS.cgi or http://10.0.0.222:8080/cgi-bin/login.html. I wonder is there a way to tell Caddy to go to one of those pages immediately when navigating to postulat.es/qnap?

Thanks in advance for any insights you may have.