Load balancing - random initial server selection, then sticky by query param (no cookies)

Help
1

1. The problem I’m having:

I have clients who:

  1. Pick a random UUID
  2. Send HTTP requests via CRUL to the server, with a query param of “client_id” where the uuid is passed. The UUID will be the same for all requests from this client.
    The clients do not support saving cookies, and I have no control over them, i.e. changing them since they will support them.

I want a load balancer strategy that:

  1. randomly chooses a server
  2. Stays with it as long as the client_id parameter is the same

The reason is because the server keeps aggregate data about the client, so I want all requests from a particular client to reach the same instance. IP cannot be used because clients do not have a fixed IP.

I know that the better solution is to move to a session management store, but I would prefer not to spend time on that right now but to find a solution at the load balancer level, so that requests with the same parameter will reach the same instance.
Is this even possible with Caddy?
Thenk you!

3. Caddy version:

v2.8.4 h1:q3pe0wpBj1OcHFZ3n/1nl4V4bxBrYoSoab7rL9BMYNk=

a. System environment:

Ubuntu 20.04.6 LTS (GNU/Linux 5.4.0-204-generic x86_64)

d. My complete Caddy config:

api.domain.com {
  reverse_proxy {
    to localhost:3000 localhost:3001 localhost:3002
    lb_policy query client_id {
      fallback random
    }
    health_uri /health
    health_interval 10s
    health_timeout 5s
    health_status 2xx
  }
}
2

I believe this code you already have should do what you’re looking for. Is that not working for you?

1 Like
3

Yes, It’s sticky but the initial selection is not really random. Just for example, a4aryhgfac1 always goes to the first server, and a4aryhgfac2 always goes to the second server:

➜  Downloads curl 'http://localhost:3000?client_id=a4aryhgfac1'
Hello from server on port 3001!
➜  Downloads curl 'http://localhost:3000?client_id=a4aryhgfac2'
Hello from server on port 3002!
➜  Downloads pgrep caddy
94407
➜  Downloads killall caddy
➜  Downloads pgrep caddy
94628
➜  Downloads curl 'http://localhost:3000?client_id=a4aryhgfac1'
Hello from server on port 3001!
➜  Downloads curl 'http://localhost:3000?client_id=a4aryhgfac2'
Hello from server on port 3002!
➜  Downloads pgrep caddy
94628
➜  Downloads killall caddy
➜  Downloads pgrep caddy
94691
➜  Downloads curl 'http://localhost:3000?client_id=a4aryhgfac1'
Hello from server on port 3001!
➜  Downloads curl 'http://localhost:3000?client_id=a4aryhgfac2'
Hello from server on port 3002!
➜  Downloads
4

I just have an explanation for you, not an alternative solution.

See reverse_proxy (Caddyfile directive) — Caddy Documentation

For policies that involve hashing, the highest-random-weight (HRW) algorithm is used to ensure that a client or request with the same hash key is mapped to the same upstream, even if the list of upstreams change.
…

  • query [key] maps a request query to a sticky upstream, by hashing the query value; if the specified key is not present, the fallback policy will be used to select an upstream (random by default)

The first request (curl ‘http://localhost:3000?client_id=a4aryhgfac1’) already contains the query parameter client_id that you have defined in the load-balancing configuration. Therefore, the fallback policy will not be applied.
Instead the hashing is performed and as outlined in the documentation, linked above, with the effect that “a client or request with the same hash key is mapped to the same upstream”.

Maybe to be imagined like “hashing of a4aryhgfac1 results in upstream 1”, so upstream 1 is used unless upstream 1 is down/unhealthy.

1 Like
5

@stbu Thanks for the explanation! Does this mean that statistically, 10000 requests with random UUID param will be distributed roughly equally across the servers?

6

I don’t know, but why don’t you test it in your environment with curl using something like this:

 curl "http://localhost:3000?client_id=[1-10000]" | grep 'Hello from server on port' | uniq -c
2 Likes
7

OK, as @stbu suggested, I tested this with 50,000 requests with UUIDs and it’s distributed roughly equally between the servers. Thank you!

2 Likes