Reverse proxy to specific path

[Duckwingduck (Greg Hao)]

1. The problem I’m having:

I’m attempting to use Caddy docker container to access a DNS over HTTPS docker container via reverse_proxy but reverse_proxy does not allow specific path. Therefore I’ve attempted to use rewrite and route both to no avail.

2. Error messages and/or full log output:

Using reverse_proxy (see below Caddyfile), I get the below error:

2023-05-11T22:18:25.903890677Z {"level":"error","ts":1683843505.9037302,"logger":"http.log.error","msg":"dial tcp 192.168.90.15:80: connect: connection refused","request":{"remote_ip":"162.158.90.202","remote_port":"11342","proto":"HTTP/2.0","method":"GET","host":"doh.greghao.com","uri":"/getnsrecord?name=uci.edu&type=A","headers":{"Cdn-Loop":["cloudflare"],"X-Forwarded-For":["152.70.152.203"],"X-Forwarded-Proto":["https"],"User-Agent":["curl/7.81.0"],"Cf-Connecting-Ip":["152.70.152.203"],"Cf-Ipcountry":["US"],"Accept-Encoding":["gzip"],"Cf-Ray":["7c5dcc378a872b63-LAX"],"Cf-Visitor":["{\"scheme\":\"https\"}"],"Accept":["*/*"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","server_name":"doh.greghao.com"}},"duration":0.000406688,"status":502,"err_id":"ecfmh2pa1","err_trace":"reverseproxy.statusError (reverseproxy.go:1299)"}

Using rewrite, I get the below error:

2023-05-11T22:14:00.367304684Z {"level":"error","ts":1683843240.366963,"logger":"http.log.error","msg":"dial tcp 192.168.90.15:443: connect: connection refused","request":{"remote_ip":"162.158.90.202","remote_port":"15852","proto":"HTTP/2.0","method":"GET","host":"doh.greghao.com","uri":"/getnsrecord?name=uci.edu&type=A","headers":{"Cf-Connecting-Ip":["152.70.152.203"],"Cf-Ipcountry":["US"],"Cdn-Loop":["cloudflare"],"X-Forwarded-For":["152.70.152.203"],"Cf-Ray":["7c5dc5bbfcc92b77-LAX"],"X-Forwarded-Proto":["https"],"Cf-Visitor":["{\"scheme\":\"https\"}"],"User-Agent":["curl/7.81.0"],"Accept-Encoding":["gzip"],"Accept":["*/*"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","server_name":"doh.greghao.com"}},"duration":0.000367245,"status":502,"err_id":"z24k4gg5g","err_trace":"reverseproxy.statusError (reverseproxy.go:1299)"}

Using route, I get the below error:

2023-05-11T22:15:54.154310804Z {"level":"error","ts":1683843354.1540062,"logger":"http.log.error","msg":"dial 192.168.90.15: unknown network 192.168.90.15","request":{"remote_ip":"172.70.207.10","remote_port":"43808","proto":"HTTP/2.0","method":"GET","host":"doh.greghao.com","uri":"/getnsrecord?name=uci.edu&type=A","headers":{"Cf-Ray":["7c5dc8831c1808f2-LAX"],"Cf-Connecting-Ip":["152.70.152.203"],"X-Forwarded-For":["152.70.152.203"],"X-Forwarded-Proto":["https"],"Cf-Visitor":["{\"scheme\":\"https\"}"],"User-Agent":["curl/7.81.0"],"Accept":["*/*"],"Cf-Ipcountry":["US"],"Cdn-Loop":["cloudflare"],"Accept-Encoding":["gzip"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","server_name":"doh.greghao.com"}},"duration":0.000172135,"status":502,"err_id":"74u4cy0xv","err_trace":"reverseproxy.statusError (reverseproxy.go:1299)"}

Using handle_path, I get the below error:

2023-05-11T22:24:57.375965596Z {"level":"error","ts":1683843897.375762,"logger":"http.log.error","msg":"dial 192.168.90.15: unknown network 192.168.90.15","request":{"remote_ip":"172.69.34.76","remote_port":"44032","proto":"HTTP/2.0","method":"GET","host":"doh.greghao.com","uri":"/getnsrecord?name=uci.edu&type=A","headers":{"X-Forwarded-Proto":["https"],"User-Agent":["curl/7.81.0"],"Accept":["*/*"],"Cf-Ipcountry":["US"],"Cdn-Loop":["cloudflare"],"Accept-Encoding":["gzip"],"Cf-Connecting-Ip":["152.70.152.203"],"X-Forwarded-For":["152.70.152.203"],"Cf-Ray":["7c5dd5c63faf520e-LAX"],"Cf-Visitor":["{\"scheme\":\"https\"}"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","server_name":"doh.greghao.com"}},"duration":0.000198686,"status":502,"err_id":"wku7aryza","err_trace":"reverseproxy.statusError (reverseproxy.go:1299)"}

3. Caddy version:

v2.6.4

4. How I installed and ran Caddy:

Pulled from docker hub

a. System environment:

Ubuntu Linux/Docker

b. Command:

docker-compose -f ~/docker/docker_compose.yml up -d

c. Service/unit/compose file:

version: '3.7'
networks:
  caddy:
    external: true
    name: caddy
  default:
    driver: bridge

services:
  caddy:
    container_name: caddy
    image: caddy:latest
    restart: unless-stopped
    networks:
      caddy:
        ipv4_address: 192.168.90.254
    ports:
      - "80:80"
      - "443:443"
      - "443:443/udp"
    volumes:
      - $APPDIR/Caddyfile:/etc/caddy/Caddyfile
      - $APPDIR/caddy_data:/data
      - $APPDIR/caddy_config:/config
  doh-server:
    image: satishweb/doh-server
    container_name: doh
    restart: always
    networks:
      caddy:
        ipv4_address: 192.168.90.15
    environment:
      DEBUG: "0"
      DOH_HTTP_PREFIX: "/getnsrecord"
      DOH_SERVER_LISTEN: ":8053"
      DOH_SERVER_TIMEOUT: "10"
      DOH_SERVER_TRIES: "3"
      DOH_SERVER_VERBOSE: "true"

d. My complete Caddy config:

#doh.greghao.com {
#  route /getnsrecord {
#    reverse_proxy 192.168.90.15/getnsrecord
#  }
#}

#doh.greghao.com {
#     reverse_proxy /getnsrecord 192.168.90.15
#}

#doh.greghao.com {
#  rewrite * /getnsrecord{uri}
#    reverse_proxy https://192.168.90.15
#}

doh.greghao.com {
    handle_path /getnsrecord {
	reverse_proxy 192.168.90.15/getnsrecord
    }
}

5. Links to relevant resources:

Reverse proxy into docker container sub-path

I’m using caddy v2. I want to reverse proxy two different different subdomains to the same docker container, with different sub-paths. For example if my docker container is called my_app, i want to proxy dashboard.example.com to my_app:80/dashboard and api.example.com to my_app:80/api. I’ve tried following caddy configuration: dashboard.example.com { encode gzip reverse_proxy my_app:80/dashboard } api.example.com { encode gzip reverse_proxy my_app:80/api } but it does not see…

Caddy 2 reverse_proxy to path

In Caddy 1, we were able to use proxy / https://host.com/path To reverse to a host+path. I encountered the same error as this issue: https://github.com/caddyserver/caddy/issues/2905 But I couldn’t understand the conclusion of the issue. My use case is the following. I have a an Object Storage/S3 bucket at OVH. It is available at this URL: https://storage.de.cloud.ovh.net/v1/<id>/mstdn-media An object is available at: https://storage.de.cloud.ovh.net/v1/<id>/mstdn-media/file.jpg With t…

Reverse proxy for a specific path

I’m trying to reverse proxy a site to a specific path on the backend. The user should access “http://external.address.com” and on the backend the application is served from “http://server:8083/SpecificPath”. With my actual config “SpecificPath” is carried on to the external address, the response turns into “http://external.address.com/SpecificPath”. my Caddyfile: external.address.com:80 { proxy / server:8083/SpecificPath { proxy_header Host {host} } } It’s like I need the wit…

The "subfolder problem", OR, "why can't I reverse proxy my app into a subfolder?"

Have you ever tried to reverse proxy an app into its own little subfolder of your domain name? Makes things neat, doesn’t it? Using example.com/foo/ for one app, example.com/bar/ for another. If you’re coming here from one of the selfhosted communities, you might be thinking along the lines of example.com/sonarr/, example.com/radarr/ etc. Chances are, you’ve tried some configuration along these lines: example.com { redir /sonarr /sonarr/ handle_path /sonarr/* { reverse_proxy localhost…

rewrite (Caddyfile directive) - Caddy Documentation

[Whitestrake (Matthew Fay)]

Howdy @Duckwingduck,

Just to clarify the use cases for each of the options you’ve tried:

• route (or handle) is ideal for limiting the scope of the reverse proxy, i.e. /getnsrecord → reverse proxy to /getnsrecord, other paths → nothing.
• rewrite is the solution when you want to have the proxy open but the upstream requires a prefix, like: /foo → reverse proxy to /getnsrecord/foo.
• handle_path is for when you want to accept request to /getnsrecord but the upstream does not expect this prefix, e.g. /getnsrecord/foo → reverse proxy to /foo upstream.

However, in all four of your error message outputs, we don’t see any configuration issues, but rather networking issues:

"logger":"http.log.error","msg":"dial tcp 192.168.90.15:80: connect: connection refused"
"logger":"http.log.error","msg":"dial tcp 192.168.90.15:443: connect: connection refused"
"logger":"http.log.error","msg":"dial 192.168.90.15: unknown network 192.168.90.15"
"logger":"http.log.error","msg":"dial 192.168.90.15: unknown network 192.168.90.15"

In the first two cases, the upstream server refused the connection (closed port?) and in the latter two cases we have unknown network (was your server moved to a different LAN subnet, and does it have a valid route to 192.168.90.15?)

[Duckwingduck (Greg Hao)]

Hmm, I see what you mean regarding the network errors, I don’t see anything glaringly wrong with the way the network is setup (I can ping the various containers from one another). But the networking issues aside, in order to accomplish what I’m looking to do, which is to use the GitHub - satishweb/docker-doh: DNS Over HTTP Container Image to resolve DNS lookups, which is the option to use? Route/rewrite/handle_path? Incidentally, the option to test if the DNS over HTTP container is working or not is to use the following command:

curl -w '\n' 'https://dns.example.com/getnsrecord?name=google.com&type=A'

Which means the URL would need to be able to handle the ? query. Thanks!

[francislavoie]

@Whitestrake

"logger":"http.log.error","msg":"dial 192.168.90.15: unknown network 192.168.90.15"

in the latter two cases we have unknown network (was your server moved to a different LAN subnet, and does it have a valid route to 192.168.90.15?)

Sorry I missed this – this is because of a syntax error, i.e. the IP address is being parsed as the network portion of the dial address, which is invalid. The network should be something like tcp or udp or unix.

This was because of 192.168.90.15/getnsrecord which is an invalid dial address. You can’t use a path in a dial address, it’s not a URL.

I don’t think I understand your goal with the rewrites/path stripping.

[Whitestrake (Matthew Fay)]

@Duckwingduck

But the networking issues aside, in order to accomplish what I’m looking to do, which is to use the GitHub - satishweb/docker-doh: DNS Over HTTP Container Image to resolve DNS lookups, which is the option to use? Route/rewrite/handle_path?

I don’t think you need any URI manipulation at the proxy layer? I can’t see why the linked service would need it.

[Duckwingduck (Greg Hao)]

I see, so I should just do

doh.greghao.com {
     reverse_proxy 192.168.90.15
}

And in my browser go to https://doh.greghao.com/getdnsrecord which should work once I resolve my network issue?

[Duckwingduck (Greg Hao)]

Hi @francislavoie, as @Whitestrake mentioned below, maybe I’m just overcomplicating the issue and I should just reverse_proxy to the docker and write the complete URL path.

[francislavoie]

Yeah, the proxy passes through the request as-is, no need to rewrite unless you actually need to change the URL before it’s proxied.

[Duckwingduck (Greg Hao)]

Indeed as both of you correctly surmised, my error was somewhere else, the corrected Caddyfile config below for other who might need it in the future using GitHub - satishweb/docker-doh: DNS Over HTTP Container Image in conjunction with Caddy:

doh.greghao.com {
        reverse_proxy 192.168.90.15:8053
}