Reverse Proxy, rewriting

Help
1

Hello, my caddy version is 0.10.4.

I am trying to reverse proxy uTorrent

	proxy /atorrent 192.168.1.4:5011/gui {
		without /atorrent
		transparent
		header_upstream X-Forwarded-Host {host}´
	}

Putting it under its own subdomain works and putting it as /gui for the proxy works, but now when I do this uTorrent responds with 404 as not found.

I had the same problem with another application that uses absolute path and can not change base url. It works on subdomain as it expects / as location, I was wondering if I could use the rewrite rules to rewrite these applications.

proxy /fusion 192.168.1.4:20604 {
		without /fusion
		transparent
		header_upstream X-Forwarded-Host {host}´
	}

The app redirects to /login when used without reverse proxy and when used with reverse proxy it goes to domain.com/login instead of domain.com/fusion/login as it is not proxy aware.

I tried to use the glances rule but they made no difference.

2

There is, as far as I’m aware, no solution publically available for this purpose.

Such a solution would need to skim all content transmitted back through the proxy, look for relevant links, and rewrite them to include your chosen subfolder.

The only other options are to give it the site root of a subdomain, or find another app that can be made proxy-aware with a base URL.

3

I was considering that was the case, I was planning to make all apps into subdomains if that is the case, but how would I make the config as easy as it is now?

As I got the login and jwt directive and all proxy apps like

domain.com {
jwt {
path /
}
login {
}
	proxy {
		transparent
	}
	proxy {
		header_upstream X-Forwarded-Host {host}
		transparent
	}

This is just example but with this I can restrict all apps under one file and password which is the reason I did subfolder or one of the reasons.

Which means I got it all under one domain if I do subdomains I would have to split up the logins? Is there a way to do it easily under same directive? Can I put all subdomains in same directive then do proxy under same or would the path / not work and then break each app?

1 Like
4

There’s one way you could simplify it a little bit.

# login.conf
jwt {
	path /
}
login {
	...
}

# Caddyfile
example.com {
	import login.conf
	...
}

https://caddyserver.com/docs/import

The server uses the same JWT_SECRET and the same credentials, so the same login would work across all sites importing login.conf.

5

That is exactly the use case for https://caddyserver.com/docs/http.filter

2 Likes
6

Ahh… Looks like I’ve given some bad advice. Looks like that could very neatly handle prepending the base URL to non-proxy-aware backends.

7

Interesting, I was not aware http.filter had such functionality.

Regarding the subdomains I encountered some issues where I can not get each subdomain to go to a global pc.domain.com/login but rather it goes to /login and breaks the apps /login as they share the same path, I was also wondering if there is a way to send the user back after success similar to

success_url /private

Is there some uri that referers to path?

I am gonna give http.filter a go instead as that seems better for what I want to do, but how would it work practically?
Can I make it so psuedo code example

IF refer /fusion
rewrite /* > /fusion/*

and make it not break root?

8

I haven’t downloaded it and given it a shot yet, but it looks from the docs that you’d want something like:

proxy /atorrent ... {
  ...
}
filter rule {
  path /atorrent
  content_type text/html
  search_pattern example.com
  replacment example.com/atorrent
}

With maybe some tweaks if the CSS or JS contains links also.

9

Alright it rewrote it successfully when I appended /Fusion to the url, but it seems as I got another issue as none of the links are actually clickable. They do not work with the proxy. Clicking the link takes me somewhere as it does not actually exist on the server?

Will I have to use rewrite as well as this just replaced the link and not rewrote it?

I tried the same on Tixati now

          <li><a href="/home">Home</a></li>
          <li><a href="/transfers">Transfers</a></li>
          <li><a href="/log">Log</a></li>
          <li><a href="/settings">Settings</a></li>
          <li><a href="/help">Help</a></li>

Is the tixati code and I replace it using

	filter rule {
		path /tixati
		content_type text/html
		search_pattern ="/
		replacement ="/tixati/
	}
	proxy /tixati 192.168.1.4:5002/ {
		without /tixati
		transparent
		header_upstream X-Forwarded-Host {host}´
	}

But it seems to break as it leads to 404 not found. So how would I proceed to rewrite the URL as well before being sent to the proxy?

10

Huh? You lost me here… Clicking which link takes you where as what does not actually exist, on which server, exactly? I’m not looking at the page you’re looking at, so I can’t tell what’s going on without an example. What does the result actually look like, what does your browser do when you follow it?

That search pattern and replacement look good to me though.

The whole page, or just elements?

11

Alright so with the current configuration it does rewrite the rules properly in the html file

        <ul class="right hide-on-med-and-down">
          <li><a href="/tixati/home">Home</a></li>
          <li><a href="/tixati/transfers">Transfers</a></li>
          <li><a href="/tixati/log">Log</a></li>
          <li><a href="/tixati/settings">Settings</a></li>
          <li><a href="/tixati/help">Help</a></li>
        </ul>
        <ul class="side-nav" id="mobile-demo">
          <li><a href="home">Home</a></li>
          <li><a href="/tixati/transfers">Transfers</a></li>
          <li><a href="/tixati/log">Log</a></li>
          <li><a href="/tixati/settings">Settings</a></li>
          <li><a href="/tixati/help">Help</a></li>
        </ul>

But the issue here with http.filter and the proxy, the proxy hits 192.168.1.4:5002/tixati/transfers so it does not exist. As http.filter does not rewrite it just replaces the html in the file correct?

It hits 404 page at Tixatis end.

Tixati does not expect it to get forwarded /tixati/help/ but it needs that link rewritten before being sent back to server, so how would I accomplish this extra step needed for it to resolve?

12

I wasn’t aware there was any rewriting going on, I’d probably avoid/remove rewrites while trying to determine the issue to cut down on complexity.

From the docs I understand that http.filter buffers the entire response in memory for any request that matches the path and/or content_type. Then it operates on the response in memory.

As for getting hits to /tixati/transfers on the backend, that’s what the without subdirective is supposed to prevent… Does this happen if there’s no filter rule? I wouldn’t have expected it to interfere in that way, but from your earlier posts, it seems without was working until we tried filter?

13

This is the current code.

       proxy /tixati 192.168.1.4:5002/home {
		without /tixati
		transparent
		header_upstream X-Forwarded-Host {host}´
	}

Removing the header_upstream seemed to fix some issues but I still got the /home as without home it breaks hard to 404 as it tries to go to /home, it seems Tixati is set to redirect to /home. So I am not sure what the best way here is. It seems it is rewriting correctly in the files but /tixati/transfers seem to be sent to the proxy even though without /tixati is enabled.

I think what I ultimately need is https://github.com/mholt/caddy/issues/649 . Without the proxy to /home it redirects to maindomain/home which is unwanted. Having it like

	filter rule {
		path /tixati
		content_type text/html
		search_pattern ="/
		replacement ="/tixati/
	}
	proxy /tixati 192.168.1.4:5002 {
		without /tixati
		transparent
	}

Changing proxy /tixati 192.168.1.4:5002 to proxy /tixati 192.168.1.4:5002/home fixes the redirect to /home so it goes to /tixati when it is supposed to go to /tixati/home I guess as I wanted to append the url. But doing the /home breaks the other buttons.

Without /home it does seem to work the first time I press on a button that takes me either to transfer or settings but not the second time. The second time I do it gives me different results even though the link in the url is the same as the first one. http://i.imgur.com/qKT30HO.gifv

So I browse to the page and see domain.com/tixati which should of been 192.168.1.4:5002/home when using without reverse proxy but I guess I can not fix that in a proper way.
The second step I do is press on transfers which takes me to domain.com/tixati/transfers which is /transfers without proxy and that does work and all images work.

The third time I press on a different button like log takes me to domain.com/tixati/log but it breaks with 404 from Tixati side so it did not like the output generated by Caddy for some reason.

14

With an upstream of 192.168.1.4:5002/home, a request for example.com with a URI that matches the basepath /tixati will be forwarded to 192.168.1.4:5002/home and the full URI appended, i.e:

example.com/tixati/foo => 192.168.1.4:5002/home/tixati/foo.

So you can see that Caddy treats the /home part as part of the upstream domain, before the URI is appended. This means you will see /home at the start of every request the upstream receives, which seems unlikely to be desirable behaviour.

But with an upstream of just 192.168.1.4:5002, you’ll see:

example.com/tixati/foo => 192.168.1.4:5002/tixati/foo.

Which is what I would normally expect to be the desired behaviour. When you also add without /tixati, Caddy should strip /tixati from the start of the URI, and it should work like this:

example.com/tixati/foo => 192.168.1.4:5002/foo.

Yes, technically, with your proxy to 192.168.1.4:5002/home, but as explained above that proxy will prevent any other requests from behaving as expected through the proxy.

What you probably want to be doing is requesting example.com/tixati/home, which should show you the main page as expected.

The second step sounds like http.filter is working great, but again, because your proxy is to 192.168.1.4:5002/home, your upstream is getting a request for 192.168.1.4:5002/home/transfers, the result of which might look fine but I suspect that its ostensible functionality is coincidental; likewise a request for example.com/tixati/log is being sent to 192.168.1.4:5002/home/log. These don’t look like properly formed requests to me.

Again from what I am seeing you probably want to proxy /tixati 192.168.1.4:5002 (sans /home) and direct your requests to example.com/tixati/home for the homepage. Appending /home to the upstream URL is not a good way to solve the redirect issue.

15

Alright so I removed /home and went to domain.com/tixati/home and it works.

But when going to any of the other tabs I get 404 error. The http filter works and I have tried both with absolute and relative URLs whole domain and subdir only but 404 still.

Alright going to each one in the browser itself
/tixati/home
/tixati/transfers
/tixati/log/
works, but pressing on the buttons which are set to the right link as I just copy pasted that link, so what would cause that?

More testing it seems that even if I press on the links in view-source they do not work, but if I copy paste them into a new tab it works.

16

At a wild guess, the buttons might be having their clicks hijacked by JavaScript?

We’re only modifying text/html content; any links passed through in scripts loaded by src attributes would remain incorrect.

You could test this pretty easily by disabling scripts and reloading /tixati/home, then trying the links with scripts disabled.

17

I cleaned up my caddyfile up a bit as it was messy and I noticed there were some rewrite rules there still left.

#	rewrite {
#        if {>Referer} has /tixati
#        to /tixati/{path}
#    }
#	rewrite {
#        if {>Referer} has /home/
#		if {>Referer} has /transfer/#
#		if {>Referer} has /gui/styles
#		if {>Referer} has /gui/scripts
#        to /tixati/{path}
#    }

I removed these as they were uncommented and can confirm tixati works now when I go to /tixati/home, I guess I could redir /tixati to /tixati/home with 301 and that would work super.

I thank you for your patience but there is a slight problem now, with the http filter rule we have in place I can not use jwt and login plugins as when I go to /login it goes to the page but when I press submit it sends it to tixati

192.168.1.1 - - [25/Jul/2017:07:38:14 +0200] "GET /login HTTP/2.0" 200 2362
192.168.1.1 - - [25/Jul/2017:07:38:16 +0200] "GET /tixati/homeprog.html HTTP/2.0" 200 666
192.168.1.1 - - [25/Jul/2017:07:38:18 +0200] "POST /tixati/login HTTP/2.0" 303 0
192.168.1.1 - - [25/Jul/2017:07:38:18 +0200] "GET /login HTTP/2.0" 200 2362

Why is the path /tixati not just sticking to it’s own path but to /login as well? If I view-source on /login for JWT I see it has gotten replaced with
<form accept-charset="UTF-8" role="form" method="POST" action="https://domain.com/tixati/login">

18

Strange. With path /tixati in the filter rule, it should only be running on responses for requests that initially include /tixati in them. If the request is just for /login, I’d assume the filter wouldn’t operate.

Can you check any other paths in that vhost to see if the filter is running on them when it shouldn’t, too?

I note that it’s rewriting the action attribute; as a workaround you could select against that in your search_pattern regex, with something like (href|src)="/ instead of just ="/.

19

Yeah I only got ipfilter but that is for ips. Disabling that filter which I sent fixes the issue,

	filter rule {
		path /tixati
		content_type text/html
		search_pattern ="/
		replacement ="https://domain.com/tixati/
	}

This one. Can it be that our path is not even being read as it is not regex? When looking at github page for plugin it only seems to accept regex?, caddy-filter/README.md at master · echocat/caddy-filter · GitHub . Yeah it adds /tixati to anything with "/, I put a random file on the root of the domain and it replaced all of it.

Disabling filter fixes it.

20

/tixati is perfectly valid regex, it’s definitely being accepted. It’s equivalent, effectively, to .*/tixati.*. Escaping forward slashes is unnecessary in this context.

I suppose to be unambiguous you might change it to ^/tixati, since we’re testing for the start of the path, but that wouldn’t cause this issue.

Sounds to me like path just isn’t having any effect. Unless you named your file tixati… Maybe open an issue at echocat/caddy-filter.