Reverse Proxy 404 Not Found


(Michael Lewis) #1
  • explain what you are trying to do:
    Setup Caddy as a reverse proxy for my HTPC services

  • show what you have already tried,
    This is my current Caddyfile

www.aomosk.com {
	gzip
    proxy / 172.17.0.2:2368
}

aomosk.com {
	gzip
    proxy / 172.17.0.2:2368
}

htpc.aomosk.com {
	gzip
	proxy /htpc 172.17.0.4:32790 {
	      transparent
	}
}

I’m using Google domains and I have a blank A record pointing to my external IP, a CNAME record for www to my domain (aomosk.com) and a CNAME record for htpc to my domain as well. Currently the root redirect to the Ghost container that I have setup (shown at the top of my Caddyfile) is working.

  • include error messages and log output,
    404 Not Found

  • and link to any relevant resources.
    Not sure, but I can provide them as needed

Thanks


(Matthew Fay) #2

Which site gave you this error?

The top two configurations should not be capable of producing a 404 on their own - so it would have to be from upstream (what’s listening on 172.17.0.2:2368?).

The second one might produce a 404 if you browse directly to htpc.aomosk.com, since only paths starting with /htpc are proxied. Any request other than htpc.aomosk.com/htpc* will be handled by the static file server. Since there’s no root configured, it will try to find files in either the directory provided by the -root CLI flag, or the PWD the binary was run from if no default web root was provided.


(Michael Lewis) #3

The site that gave me the error is htpc.aomosk.com. The Ghost container is configured on port 2368. I set up the webdir for HTPC manager as /htpc but I was just going for a subdomain redirect with htpc.aomosk.com. That make sense though why it would be giving me the Not found error. What would I need to do to get the proper redirect for the subdomain?


(Matthew Fay) #4

What’s the desired behaviour, i.e. what should happen instead when someone browses to htpc.aomosk.com?


(Michael Lewis) #5

It should open up the HTPC Manager web interface, which is running in a docker container at localhost:32790 or the container IP which is what I have in my Caddyfile right now/


(Matthew Fay) #6

You can change /htpc to just / to configure all requests at that subdomain to be proxied to the upstream HTPC Manager.

If the upstream server expects a base URL of /htpc, you could redirect clients to prepend this:

redir {
  if {path} not_starts_with /htpc
  / /htpc{uri}
}

(Michael Lewis) #7

Sorry, I’m still a little new to the networking side of things. When you say that the requests would need to be proxied to the upstream service, what does that entail?

For the redirect example, is the URI the full domain name? and is the path, the web root or should that point somewhere else?


(Matthew Fay) #8

Well, if someone browses to https://htpc.aomosk.com, if you want them to load the HTPC Manager (the upstream server), somehow their request needs to get to 172.17.0.4:32790.

At the moment, with your example Caddyfile, Caddy is only sending requests upstream if their request begins with /htpc. That means, for example, that https://htpc.aomosk.com/htpc/foo would be sent up, but a request for https://htpc.aomosk.com/foo won’t be, so the latter would never load the HTPC Manager.

If you change proxy /htpc 172.17.0.4:32790 to proxy / 172.17.0.4:32790, all requests for the hostname htpc.aomosk.com, including /foo, /bar, or just /, will go upstream and hopefully get a good response.

If the upstream server expects to only receive requests starting with /htpc (some people configure this with some apps so they can have example.com/app1 and example.com/app2, instead of subdomains), instead of proxying / onwards, you want to explicitly tell visiting clients, “Nothing’s here, come back for /htpc instead.” That’s what the redir directive does.

To clarify, {uri} here expands to everything after the hostname. For https://example.com/foo/bar?query=value#anchor, the URI placeholder expands to /foo/bar?query=value#anchor. Essentially we’re saying, “Try again, but put /htpc at the start of the URI,” but only if it didn’t already start with /htpc.


(Michael Lewis) #9

So that change brought about a different error. I’m getting a 502 Bad Gateway now. I tried both with just removing /htpc and also with the redir function in there as well. I removed the /htpc webdir from the HTPC service itself so it’s just / now.

This was the way that I added it:

htpc.aomosk.com {
	gzip
	redir {
          if {path} not_starts_with /htpc
          / /htpc{uri}
    }
	proxy / {container IP}:32790 {
	      transparent
	}
}

The console from Kitematic is giving this error:

15/May/2018:07:35:12 +0000 [ERROR 502 /htpc/] dial tcp 172.17.0.4:32790: connect: connection refused

(Matthew Fay) #10

Is the container up, with the same IP address, and is the service still listening on the same port?


(Michael Lewis) #11

Yes, the IP address is still correct in the Caddyfile and the container is still up and running.


(Matthew Fay) #12

What does curl -IL 172.17.0.4:32790 produce from the Caddy host?


(Michael Lewis) #13

That was the response that I got right here. I’m not sure if this helps at all but I’m running the containers in Kitematic and there is an option for setting up network links between 2 containers. I haven’t messed with that yet but wanted to provide that information if it might be of use.

$ curl -IL 172.17.0.4:32790
curl: (7) Failed to connect to 172.17.0.4 port 32790: Connection refused


(Matthew Fay) #14

Container links should let you refer to the containers by the service name for DNS resolution, e.g. proxy / http://container-name:32790, but if you haven’t set them up and haven’t configured any custom network settings, all the containers should still be able to contact each other by IP address.

Essentially, the 502s are occurring because 172.17.0.4:32790 isn’t playing ball at all, so that (network?) issue needs to be sorted first. If you can get it to respond to curl, you’ll be able to get Caddy to proxy to it.


Forgot to mention, with this you don’t need the redir at all, just proxy / IP:PORT.


(Michael Lewis) #15

Alright good to know, I took that part out of the Caddyfile. Is there anything from the DNS side that could be causing this or do you think it has something to do with the services in the container?


(Matthew Fay) #16

Refused connection generally implies that a reachable host is at that IP address, but it chose not to let you open a connection - which could be because of a firewall, or nothing listening on that port, or whatever is listening on that port actually closed the connection itself. DNS definitely can’t be the issue because you’re not using it to resolve the IP address, you’ve provided the IP directly in the Caddyfile.

Since this kind of issue has so many details specific to how you’ve set up your environment, it’s hard for me to say for sure. We’re venturing beyond the realm of Caddy and into system administration.


(Michael Lewis) #17

I understand, I will have to do some poking around then. Thank you very much for all of your help.


(system) #18

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.