Reverse proxy parsing difficulties

@francislavoie In the thread Migrate to using a wildcard certificate, specifically post #2 and post #5, you talk about parsing difficulties between the map and reverse_proxy directives. Quoting from @Rob789’s wiki article above:

Is this an instance where we hit a limitation if the map construct is used for managing subdomains?

The issue is that the Caddyfile attempts to provide some additional “magic” when configuring proxy upstreams. For example, specifying a scheme like https://, srv+http:// or h2c:// presets some additional config options (enables the tls transport option for https://, uses the lookup_srv instead of dial for srv+http://, or sets the versions option for h2c://).

Because we use the Go stdlib function url.Parse() to parse the upstream address in that case to split the scheme from the rest, it becomes complicated to handle if there’s a placeholder (i.e. { } inside the upstream address, because that function will throw an error if it sees characters it doesn’t support.

There’s a few ways we “could” resolve it, but none are good; we could reimplement url.Parse ourselves to handle placeholders as a hostname field, but that’s no good because it’s not a simple function and we’d rather not have to reimplement functionality that already exists elsewhere; or we could find any placeholders and replace them with some dummy strings that we could re-replace with the placeholder afterwards, but that also gets complicated code-wise and could open up really dumb edgecases depending on the user inputs, etc.

The other issue is that even if you didn’t use a scheme like https://, the upstream address needs to be both the IP/domain and port to dial. In the Caddyfile, typically if you just specify an IP, it would add :80 to the end if there was no port in the config input (to be helpful to the user). But if there’s a placeholder, we can’t make that assumption of adding :80 because the placeholder might expand to also include a port, which will cause runtime problems where you might end up with two ports in the upstream address which is invalid. So you either need to specify the port yourself if it’s always :80 when using a placeholder, or make sure the placeholder value always includes the port (because the dialer won’t assume :80 itself).

It’s also not possible for Caddy to verify correctness at config adapt/validation time if you use a placeholder, because the actual underlying value is unknown.

Anyways, all this isn’t specific to map, but specifically to placeholders, which map happens to makes available to use (for example for reverse_proxy upstreams).

3 Likes

Hi @francislavoie, thank you for taking the time to take complex subject matter and turn it into language the average layperson, like myself, can begin to appreciate. You would be a terrific educator if you chose this line of work.

I look at my Caddyfile and realise it’s matured a lot, but it took a lot of patience from the community, especially you, to help me bring it to that point. I do also like what @Rob789 has done to encrypt comms between backend services and Caddy in his wiki article Use Caddy for local HTTPS (TLS) between front-end reverse proxy and LAN hosts. On the other hand, I love the elegance and summary perspective that the map directive brings to Caddyfile (it reminds me of the vlookup function in Excel). Where I think it’s important to encrypt backend comms, I’m going to try to find a way to skirt around the map directive to implement this.

1 Like

I don’t think you’ll actually run into any problems along with map as long as you make sure you either always provide a port in the destination values, or you always use the same port and include it as part of the upstream on the directive (like {value}:80).

Good to know. I’ll explore that as I begin to figure out how to implement backend TLS.

FYI the correct term for it is mTLS

1 Like

This topic was automatically closed after 29 days. New replies are no longer allowed.