Load balance docker replicas based on sticky querystring param hash?

1. The problem I’m having:

We’re trying to figure out a simple way to do dynamic sticky sessions <> docker replicas, but based on URL query params, not user IP

  • Imagine something like dynamic chatrooms, where there are more chatrooms than nodes, and they come & go: if a requested room isn’t being served, any node can pick it up, and while it is active, subsequent users of that room should go to the same node. We use websockets, so we’d like to persist those connections while live. It’s fine to retire/reassign a room as folks clear out.

  • Ex: /app1/?room=abc => worker hash('abc', 3) => choose([1,2,3]) => http://app:800<x>/...

  • Our app is a containerized web app, which we have as a docker compose

  • Near term we want N replicas same-node, and not far out, M nodes. We’re thinking of doing via docker swarm (replicas: n) before moving to k8s

We’re looking for something simple for the routing here, so wondering if any pointers, e.g., caddy-docker-proxy plugin and some config for it

2. Error messages and/or full log output:

None yet, we’re looking for architectural advice wrt happy paths in caddy

Please use the preview pane to ensure it looks nice.

3. Caddy version:


4. How I installed and ran Caddy:


a. System environment:

Linux (cloud) → maybe ALB → docker caddy → swarm / k8s docker replicas

b. Command:

c. Service/unit/compose file:

d. My complete Caddy config:

5. Links to relevant resources:

Before addressing your actual question, I’ll say that this is a flawed approach. I would strongly recommend instead moving your chatrooms to using a centralized store so that it’s decoupled from your individual servers.

I recommend using Redis for pub/sub. All servers would subscribe to messages on behalf of each connected user, and each would publish messages to the appropriate channel (chatroom) which would fan-out to each server to send the message to the users in that channel.

Or if that sounds like too much effort, you could use an off-the-shelf solution like https://centrifugal.dev/ instead which does the heavy-lifting for you.

In v2.7.0-beta.1 we added a new query load balancing policy, it should do what you need. The docs update isn’t merged yet, but you can find it here for now Docs for v2.7.0 by francislavoie · Pull Request #322 · caddyserver/website · GitHub The syntax is lb_policy query <key>.

Thanks, will look!

And yes, this is simplified, we have heavy data gravity (potentially GB-TBs) per ‘chatroom’ so centralizing isn’t great

It doesn’t necessarily need to be “centralized”, Redis can be sharded. But either way, it’s just about moving the data pipeline to a service that is specifically designed to handle that kind of workload. It would allow you to scale your applications servers without worrying about specifics of load balancing.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.