Limit number of concurrent requests to reverse_proxy backend?

Is it possible to limit the number of ongoing requests to a server backend defined in reverse_proxy?

My app server sadly does not include this feature, and can become clobbered / overloaded.

The user experience ends up pretty poor: User is forced to wait a long time (20+ seconds as the app server is clobbered), then finally get a 500 error.

Would rather have a hard limit inside caddy itself to either get a 500 right away when the limit is reached and/or redirect the traffic to another server in the reverse_proxy list.

Any insight, help, recommended patterns appreciated!

1 Like

Of course. You want to set max_requests here: Modules - Caddy Documentation

You could also set a limit for the passive health checker, that’s a more general limit for all backends on the proxy.

1 Like

Wonderful, thank you for the prompt and useful information @matt !!

1 Like

For those following along, here’s also the setting for the passive health checker: Modules - Caddy Documentation (unhealthy_request_count)

Extra note:

You need to specify fail_duration to have unhealthy_request_count work.

Example Caddyfile:

:80 {
	reverse_proxy 127.0.0.1:8000 {
		fail_duration 1s
		unhealthy_request_count 10000
	}
}
1 Like

Yep, sounds right! The fail_duration is needed to enable passive health checks. (I’ll fix the docs, just noticed they’re a bit inconsistent about that.)

1 Like