V2:rewrite+proxy

1. My Caddy version (caddy -version):

(devel)

2. How I run Caddy:

./caddy start

a. System environment:

CentOs 8

Caddyfile:

server3.zaak-op-orde.nl {
root * /var/www/zoo3
encode gzip
php_fastcgi 127.0.0.1:9000
file_server
matcher a {
path /apv1
}
rewrite match:a /apiv1 /

reverse_proxy match:a {
to 10.13.13.2:8080
}
reverse_proxy / {
to 10.13.13.2:8083
}
}

3. The problem I’m having:

I am trying to run two proxies
The proxy to server3.zaak-op-orde.nl/ works

It seems I dont understand the rewrite syntax.

The proxy to server3.zaak-op-orde.nl/apiv1 works too but still adds the /apiv1 path,
how do I rewrite that to redirect to 10.13.13.2:8083/… instead of 10.13.13.2:8083/apiv1/… ??

Hi @mbetel, welcome to the Caddy community.

First, some clarification on terminology:

A rewrite is invisible to the client (your browser) because the server is changing the request behind the scenes - effectively pretending it’s a different request.

A redirect is visible to the client because the server effectively tells the client, “wrong address, use this URL instead”, and the browser’s address bar will update with this change.

So the question is:

Do you want a rewrite, or do you want a redirect?

The service runs on port 8080, if I do a redir I would have to expose Port 8080 to the outside which I do not want. I am looking for the equivalent of apaches:

ProxyPass /apiv1 http://localhost:8080 connectiontimeout=2400 timeout=2400
ProxyPassReverse /apiv1 http://localhost:8080
ProxyPass /tst http://localhost:3030 connectiontimeout=2400 timeout=2400
ProxyPassReverse /tst http://localhost:3030

Where apache automagically removes the matchstring when proxying

No redirect or rewrite at all, then. You want the browser to access /apiv1, and you want Caddy to translate that to / when proxying upstream.

Quite doable - removing the prefix before proxying can be done with the without /apiv1 subdirective.

I think ProxyPassReverse does some forward magic as well to translate Location and links to the correct URI.

If you have issues with redirects and links from the proxy, fixing Location and and href / src links can be done with the header_downstream regex replacement (undocumented, but explained here: Regex rewriting of upstream/downstream headers in proxy by comp500 · Pull Request #2144 · caddyserver/caddy · GitHub) and by using the http.filter plugin to modify HTML coming back from the proxy.

Matthew, thanks for taking the time to answer.

As far as i can discern Caddy2, with which I am playing, does not support the caddy-v1 ‘without’ subdirective (yet?) Thus I’ll still have to rewrite the url somewhere.

Oh, uhh, that’s 100% my bad. I came back to this thread and completely forgot it’s v2. Let me take another look.

@mbetel

Try:

  matcher a {
    path /apiv1
  }
  rewrite match:a /
  reverse_proxy match:a 10.13.13.2:8080

Changed:

  1. The matcher's path from /apv1 to /apiv1 (assumed typo)
  2. Removed /apiv1 from the rewrite
  3. Consolidated reverse_proxy to a one-liner

According to the v2 rewrite documentation, the syntax is: rewrite [<matcher>] to

So by having rewrite match:a /apiv1 /, Caddy probably dropped the last token and simply rewrote the matched request to /apiv1. Combined with the typo in the matcher means it only rewrote requests for /apv1. Probably not your intended result.

Home · caddyserver/caddy Wiki · GitHub

@Whitestrake Thanks!

rewrite match:a / did the trick!

1 Like

Why not just do rewrite /apiv1 /? You don’t need a complex matcher when it’s just a path.

Because then we can proxy to the upstream as part of the same matcher middleware chain.

Makes things simpler to think about, in my opinion. Because we’re reverse proxying from /apiv1, but only after rewriting.

We’re also reverse proxying from / to a different upstream.

They’re equivalent though. One is just shorthand.

The way the v2 Caddyfile currently is (I say “currently” because I have some last-minute changes planned for the Caddyfile, just need to decide on some things), rewrite is always executed before reverse_proxy unless you set the directive ordering to be something other than the default.

rewrite /apiv1 /
reverse_proxy /apiv1 10.13.13.2:8080

I’d be surprised if that works though because all requests starting with /apiv1 will get rewritten to /, thus reverse_proxy will not match /apiv1

This is a problem I figured using match:a for both would solve. Match the request → rewrite it → reverse proxy it. The request was already “matched” when it was received - so the reverse_proxy can operate on it regardless of the fact the URI has been rewritten before the proxy executes.

If not, how would one proxy only requests to a specific URI but only after rewriting it?

This is what I’m hoping my last-minute changes to the Caddyfile will solve; I think we need to make this easier.

Will be iterating on this before long… stay tuned.

1 Like

@matt I am curious about this use case. Often when reverse proxying we want to trim the start of the request.

Say I have a upstream server with script under the path 10.0.0.1/a/b/script.js

But my reverse proxy will expose this upstream server as /service1 so my full url will be https://mysite.com/service1/a/b/script.js

IMHO the reverse proxy would do the work of stripping /service1 from the proxied request so the request to the upstream server is /a/b/script.js

Thoughts?

Yes, trimming before proxying is a common use case and was supported in v1 via the without subdirective. Right now it can be done in v2 but only with regex and it must be configured in JSON as the Caddyfile can’t quite achieve this complexity and doesn’t have a dedicated without subdirective to handle it.

Along with some potential changes to Caddyfile route matching / directive ordering, here’s a quick proposal Matt and I thought up for some directives that would enable this behaviour:

https://github.com/caddyserver/caddy/issues/2906

1 Like

Hello, I feel the need to do something about this feature now. Tricky to reverse proxy without this at the moment.

Having a little look at the php directive as an example, I should change the reverseProxy httpcaddyfile.RegisterHandlerDirective("reverse_proxy", parseCaddyfile) to implement the httpcaddyfile.RegisterDirective("reverse_proxy", parseCaddyfile) so I can include various rewrite rules after the matcher but before the reverse proxy handler?

To be clear, you can do this today in the JSON config by making a rewrite handler followed by a proxy handler, that simple.

Yep, basically. However, I do not think the reverse proxy handler should be rewriting the request. So before you go down that road, we are looking into a more elegant solution to the problem, here: v2 proposal: New 'route' directive for Caddyfile · Issue #2911 · caddyserver/caddy · GitHub and v2: new directives for fast substring URI manipulation · Issue #2906 · caddyserver/caddy · GitHub

Yes I agree, reading the route proposal makes sense. I was thinking that you could have collected identical matchers and then apply the global ordering within them to collect the handlers together. But having a route looks a lot cleaner and is more explicit.

Hi @matt ,

Can you please help with any tentative release version with this fix, since this is one of the major important requirement for our use case we are trying to build and only the absence of this feature is the blockage.

I went through the example of rewrite and reverse proxy what we are trying to achieve is for example
we have,
/docker/user
/google/user

then ,
/docker/user should redirect to → Log in | Docker
/google/user should redirect to → google.com/user

so if you observe here both the endpoints have URI /user, how do we differentiate request between docker.com and ]google.com as reverse-proxy comes after rewriting.

1 Like

If it helps I added a temporary change to re-add the strip prefix from v1.

https://github.com/sarge/caddy/commit/30d9f1b12f7ba5b3aa1305ba28ce646f6cd5e0d4

1 Like