Help in figuring good symbiosis between Caddy and AGW

[Tomasz_Trebski]

1. The problem I’m having:

Ok, so in very short words I am trying to implement following architecture:

In this particular case what is important is that we may have some custom domains like app-x.com that are quite to the point aliases to something like {appId}.app.com. In the same time clients may also use {appId}.app.com to connect to the system. For those cases I will have wildcard certificate.

LeanReverseProxy is meant to be a Caddy server that has one job maintain list of domains for which certificates are meant to be either generated or externally provided. I was talking about this in Dynamic TLS selection thread.
I really want to keep this quite simple to avoid maintenance costs. Then there’s this AGW (application gateway) that is to be case-specific. For example if I am talking about apps I expect AGW to be fat and be able to understand where to route requests based on the input.

Now my problem is this input. How would you guys say is the best way to handle this on Caddy side so that AGW gets requests that are normalized i.e. request is always like /app/{appId} and then rest of the path and query parameters. Also same logic should apply to any domain we’re discussing which obviously implies some sort of mapping for domains that are not wildcard i.e. have no appId in domain.

PS. Sorry if this is not quite Caddy related but frankly I just need some sort of brainstorming and if you guys are up to the challange, I will appreciate it.

PS2. If AGW sounds too much and we can simply do something like:

/api/* → proxy backend
/files/* → proxy storage
/* (default) → proxy UI

I am all ears.

2. Error messages and/or full log output:

N/A

3. Caddy version:

N/A

4. How I installed and ran Caddy:

N/A

[Tomasz_Trebski]

I would imagine something like this could help. This assumes doing AGW job in Caddy but since we’re talking Caddy, why not.

  @handler1 host test.net
  @handler2 host foo.net
  @handler3 host bar.net

  handle @handler1 {
    reverse_proxy :81
    handle_path /api/* {
      rewrite * /app/1{uri}
      reverse_proxy :82 {
        header_down +X-App-Id 1
      }
    }
  }
  handle @handler2 {
    reverse_proxy :81
    handle_path /api/* {
      rewrite * /app/2{uri}
      reverse_proxy :82 {
        header_down +X-App-Id 2
      }
    }
  }
  handle @handler3 {
    reverse_proxy :81
    handle_path /api/* {
      rewrite * /app/3{uri}
      reverse_proxy :82 {
        header_down +X-App-Id 3
      }
    }
  }

Sounds like a lot of repetition. How can I make it smaller?

[francislavoie]

@Tomasz_Trebski

How would you guys say is the best way to handle this on Caddy side so that AGW gets requests that are normalized i.e. request is always like /app/{appId} and then rest of the path and query parameters.

I’d suggest maybe using HTTP headers instead of rewriting the URL. You could set a X-App-Id: <app-ID> header or something based on however you need to parse it. You can set that in Caddy with header_up in the reverse_proxy.

… oh wait you’re already essentially doing that, read your reply after but you used header_down which writes a response header back to the client, not a request header to the upstream. You probably wanted header_up?

But you could use a @prefixMissing not path_regexp matcher to check if the URL doesn’t start with /app/([a-zA-Z0-9]+)/ then do rewrite @prefixMissing /app/{vars.appId}{uri} to add it.

You can use the map directive to map certain domains to app IDs then use the placeholders for your app ID in your proxy code.

[Tomasz_Trebski]

@francislavoie

But you could use a @prefixMissing not path_regexp matcher to check if the URL doesn’t start with /app/([a-zA-Z0-9]+)/ then do rewrite @prefixMissing /app/{vars.appId}{uri} to add it.

Is @prefixMissing ... something built-in?

@francislavoie

You can use the map directive to map certain domains to app IDs then use the placeholders for your app ID in your proxy code.

That is exactly what I cannot figure out/make work. I am gonna continue trying but if you don’t mind posting some example that later I could just expand, it’d be great. Unless I do it first.

For example I use this:

  map host {appId} {
    test.net 1
    foo.net  1
    bar.net  1
    default  -1
  }

  handle @handler1 {
    reverse_proxy :81
    handle_path /api/* {
      rewrite * /app/{appId}{uri}
      reverse_proxy :82 {
        header_down +X-App-Id {appId}
      }
    }
  }

I always get -1 as appId i.e. http: 80/api/x Host:test.net even though without a map the host seems to be matched perfectly as in Caddy respects the host with @handler1 but does not with map

@francislavoie

I’d suggest maybe using HTTP headers instead of rewriting the URL.

Can’t really do that. This whole appId is builtin into REST-api, it is identifier of quite large object that has quite a lot of sub-resources, so path based solution is mine to live with. I don’t want to maintain two separate entry points to the same API (with paths + headers) because this API can also be used directly by couple of other authorized actors.

@francislavoie

but you used header_down

Actually that is intentional because I will have path to downstream API and I just wanted to reply back to caller with header to have something to trace/debug.

[francislavoie]

@Tomasz_Trebski

Is @prefixMissing ... something built-in?

See Request matchers (Caddyfile) — Caddy Documentation, you define a named matcher.

@Tomasz_Trebski

map host {appId} {

It would be map {host} {appId} { otherwise you’re comparing against the literal word host.

[Tomasz_Trebski]

Ah, It worked quite nice. Thx.

@francislavoie is there any way to replace map with something similar like cert manager (i.e. what we discussed in another thread). Have mapping dynamic based on some endpoint response?

[francislavoie]

You could write your own plugin to do that pretty easily. See Extending Caddy — Caddy Documentation, and read the source for the map handler to see how it sets variables. You can make your plugin make HTTP requests and optionally cache the results for a certain amount of time to avoid pressure etc.

[Tomasz_Trebski]

@francislavoie if you don’t mind.

Is there to store a plugin in corporate bitbucket? I rather think we have quite specialized use case and as such a plugin does not belong to public domain. Can xcaddy download sources to build from in such case?

[francislavoie]

Yes. xcaddy is just a tool to run Go commands to build. See https://www.digitalocean.com/community/tutorials/how-to-use-a-private-go-module-in-your-own-project. You can also use the = syntax to have xcaddy build from a relative path on your local machine (see the README on GitHub).