Caddy V2: Differentiate Full upstreams from Unhealthy upstreams?

Coding
#1

The problem

After setting the upstreams/max_requests, any extra request will get a 502 (Bad Gateway) if the current number of simultaneous requests exceeds the configured one.

This behavior is expected, since in the current implementation, both the Healthy-status and the Full-status are considered in Upstream.Available(). But sometimes this HTTP status code is confusing, since it’s hard to developers to figure out whether the backend is unhealthy or the client is out of control.

The proposed behavior

Maybe 429 (Too Many Requests) is more intuitive for this scenario?

#2

Good point, we could probably see if we could emit helpful logs here.

Maybe 429 (Too Many Requests) is more intuitive for this scenario?

I don’t think so, since that tells the client to enhance their calm, but the problem isn’t that client making too many requests, it’s just that the server has reached its request allocation.

This is probably debateable, but in general I think it’s good to be consistent that we return 502 when the proxy does not have any available backends.

#3

Yes, if max_requests is defined as “the request allocation”, then I think it does make sense to return 502 to keep the semantic consistency.

Currently, I use max_requests as a method of protecting upstream backends from being overloaded (as discussed before here :wink: ) From the perspective of the backend protection, if the client is out of control, I think 429 is more self-explanatory.

So maybe using rate-limiting is a better option here?

#4

Yeah, that’d probably be your fastest solution to get what you want.

#5

Thanks for your advice!

But I’m afraid the caddy-ratelimit plugin is not concurrency-safe. For example, there is a race condition between the write of rl.currentWindow and the read of rl.currentWindow.

#6

You should report it to the developer in an issue then, or submit a PR to fix it. (Sorry, I am not familiar with third party plugins.)

#7

Got it! (That’s OK.)

1 Like