After setting the upstreams/max_requests, any extra request will get a 502 (Bad Gateway) if the current number of simultaneous requests exceeds the configured one.
This behavior is expected, since in the current implementation, both the Healthy-status and the Full-status are considered in Upstream.Available(). But sometimes this HTTP status code is confusing, since it’s hard to developers to figure out whether the backend is unhealthy or the client is out of control.
The proposed behavior
Maybe 429 (Too Many Requests) is more intuitive for this scenario?
Good point, we could probably see if we could emit helpful logs here.
Maybe 429 (Too Many Requests) is more intuitive for this scenario?
I don’t think so, since that tells the client to enhance their calm, but the problem isn’t that client making too many requests, it’s just that the server has reached its request allocation.
This is probably debateable, but in general I think it’s good to be consistent that we return 502 when the proxy does not have any available backends.
Yes, if max_requests is defined as “the request allocation”, then I think it does make sense to return 502 to keep the semantic consistency.
Currently, I use max_requests as a method of protecting upstream backends from being overloaded (as discussed before here ) From the perspective of the backend protection, if the client is out of control, I think 429 is more self-explanatory.
So maybe using rate-limiting is a better option here?