Attempting to reverse proxy to a folder w/ Caddy 2

miguelr · May 7, 2020, 6:50am

1. Caddy version (`caddy version`):

v2.0.0 h1:pQSaIJGFluFvu8KDGDODV8u4/QRED/OPyIR+MWYYse8=

2. How I run Caddy:

a. System environment:

Linux rproxy01 4.15.0-55-generic #60-Ubuntu(18.04) SMP Tue Jul 2 18:22:20 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Relevant portion of Caddyfile:

dps.prod.globi.us {
    import dns
    reverse_proxy {
        to https://ruprod.int.globius.org/dps                <---Line 19 is this line
        import transparent
        }
}

Error output:

root@rproxy01:/etc/caddy# /usr/bin/caddy run --config /etc/caddy/Caddyfile

    2020/05/07 06:44:33.309 INFO    using provided configuration    {"config_file": "/etc/caddy/Caddyfile", "config_adapter": ""}
    run: adapting config using caddyfile: parsing caddyfile tokens for 'reverse_proxy': /etc/caddy/Caddyfile:19 - Error during parsing: for now, URLs for proxy upstreams only support scheme, host, and port components

Basically what I’m trying to do is when a user goes to the dps.prod.globi.us website, they reach the page internally /dps

I can remove the https:// and the /dps and it’ll work just fine, but when I include both, it gives that error.

Whitestrake · May 7, 2020, 7:47am

Read the error in detail:

It’s telling you that you can’t have a reverse proxy upstream to a subfolder. You may only specify a scheme (e.g. https://), a host (e.g. example.com), and a port (e.g. :8080). Trying to proxy to a subfolder implies a rewrite, which reverse_proxy doesn’t handle.

Instead of defining an upstream with a URI, manipulate the URI yourself before sending it upstream, e.g.

  # Prepend '/dps'
  rewrite /dps{uri}

  # Send upstream
  reverse_proxy https://upstream

sergipcm · May 7, 2020, 12:29pm

Similar problem here, so I’m replying on this thread

need to reverse proxy to an SSL API subdomain so like: https://example.com/v1/endpointA/

exact same issue than trying to point to a folder
and I get the exact same error

Error during parsing: for now, URLs for proxy upstreams only support scheme, host, and port components

I don’t exactly get how rewrite works in this case
Thanks!

Whitestrake · May 7, 2020, 12:54pm

Just gotta take the URI and prepend it to the request URI before proxying.

rewrite /v1/endpointA{uri}
reverse_proxy https://example.com

sergipcm · May 7, 2020, 1:23pm

thanks @Whitestrake guess it worked because now I’m battling with Cloudflare 403…not sure all the fields are correctly set, because instead of a forbidden I think I should be getting a captcha, which btw still would be an issue…

my.domain.com {
rewrite /v1/inventory https%3A%2F%2Fexample.com
reverse_proxy https://example.com {
        header_up Host {http.request.host}
        header_up X-Real-IP {http.request.remote}
        header_up X-Forwarded-For {http.request.remote}
        header_up X-Forwarded-Port {http.request.port}
        header_up X-Forwarded-Proto {http.request.scheme}
        header_up Authorization Basic mybase64secret:key
        transport http { tls }

        }
}

Help really appreciated!
I started using caddy some weeks ago because I thought it was amazingly straight forward. I think that as documentation with examples on how to “build” with caddy are put on the site, this will really be the most user friendly think out there

Whitestrake · May 7, 2020, 1:27pm

This probably ain’t workin’. Check the syntax:

header_up [+|-]<field> [<value|regexp> [<replacement>]]

—reverse_proxy (Caddyfile directive) — Caddy Documentation

So it’s taking the Authorization request header, looking for instances of Basic and replacing them with mybase64secret:key before passing them forward. This is, predictably, probably what’s breaking your authentication.

Instead, put your header data in quotes so it’s parsed as a single token.

header_up Authorization "Basic mybase64secret:key"

P.S. Since this effectively permanently bypasses a security mechanism, I do hope you’ve got some kind of access control or security on the site you’re serving with Caddy!

P.S.S. There’s also a bunch of headers in there that don’t need to be:

By default, Caddy passes thru incoming headers to the backend—including the Host header—without modifications, with two exceptions:

It adds or augments the X-Forwarded-For header field.

It sets the X-Forwarded-Proto header field.

—reverse_proxy (Caddyfile directive) — Caddy Documentation

And you don’t need transport http { tls }, the whole block is implicit by the https://example.com upstream.

tls uses HTTPS with the backend. This will be enabled automatically if you specify backends using the https:// scheme or port :443 .

—reverse_proxy (Caddyfile directive) — Caddy Documentation

sergipcm · May 7, 2020, 1:51pm

mmm still stuck at the 403- Forbidden screen at cloudflare, so I’m not sure redirect its working fine. Accessing directly to the end api via postman with http auth secret:token works fine.

About security yes, not a big deal, since I’m only pointing to an endpoint, user would not be able to auth to the rest of the api.

Now removed. Btw something a bit surprising:

If I add the transport tls option, with https://example.com, as you comment its not necessary
If I add the transport tls with http://example.com I get this error:

adapt: parsing caddyfile tokens for 'reverse_proxy': Caddyfile:24 - Error during parsing: upstream address scheme is HTTP but transport is configured for HTTP+TLS (HTTPS)

Removed those headers too

Right, has total sense

Still not sure there is a clear way forward

Whitestrake · May 7, 2020, 1:53pm

Naturally - the two conflict. HTTPS is literally HTTP over TLS. Can’t have HTTP (i.e. no TLS) with transport over TLS, that’s just HTTPS.

To explain a little bit - having to specify tls via the transport subdirective used to be necessary for a HTTPS upstream. Earlier on in the piece, Caddy v2 did not allow you to specify the scheme at all in the upstream. It was just a hostname and port. Now it also accepts scheme and from it, implies the transport. The tls option there is… well, not quite vestigial, because you could still specify example.com upstream and then specify HTTP transport over TLS to say it’s HTTPS. But why, when you could just specify https://example.com?

What’s the latest full Caddyfile?

sergipcm · May 7, 2020, 2:04pm

Here it is, getting thinner…

inventory.mydomain.com {
rewrite /v1/inventory https%3A%2F%2Fexample.com
reverse_proxy https://example.com {
        header_up Host {http.request.host}
        header_up X-Real-IP {http.request.remote}
        header_up Authorization Basic mybase64secret:key
        }
}

Guess might have use cases I’m even not aware of. Isn’t for https://host:port already supported?

Whitestrake · May 7, 2020, 2:15pm

Did you try with header_up Authorization "Basic mybase64secret:key" (note the quote marks)?

Yes, exactly. I don’t think I’d ever need to bother to specify TLS transport again, myself!

francislavoie · May 7, 2020, 2:31pm

It’s still needed if you’re proxying to a unix socket, cause you can’t put https:// in front. Probably in some other situations as well.

Whitestrake · May 7, 2020, 2:32pm

Ahh, there it is! Knew there’d be something.

miguelr · May 7, 2020, 4:04pm

Well, I wake up and I see 11 replies in the thread. Well then haha.

I’m going to attempt the rewrite now.

sergipcm · May 7, 2020, 4:22pm

I

I guess reverse proxies are quite popular nowadays

@Whitestrake Yes, ops I see I forgot them when copying many thanks for the help provided. I think I will continue other days to see if I finally come up with a solution. For now I’m a bit stuck…

system · June 6, 2020, 4:22pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.

Attempting to reverse proxy to a folder w/ Caddy 2

1. Caddy version (caddy version):

2. How I run Caddy:

a. System environment:

1. Caddy version (`caddy version`):