The load balancer cookie value is not designed for each client. The value is unique per upstream server. When using round_robin fallback:
Client1 gets cookie value localhost:100.
Client2 gets cookie value localhost:101.
ClientN gets cookie value localhost:99+N.
Client2001 gets cookie value localhost:101. (It wraps around to the first upstream.)
The cookie is not unique per client, Caddy uses it to figure out which backend to route the request to, that’s all. Caddy would need some kind of scheme to produce unique cookie values… Right now they’re literally just a hash of the upstream address. So Caddy would need to figure out what alternative values to use - and then it would need to figure out how to READ them coming back when they’re, by necessity, unique to each and every client. This is straight up a hijack of the purpose of the load balancing cookie which would make its job far more complicated.
This just sounds like you want to use Caddy’s load balancer cookie for authentication (that is, you want to use it to identify your clients individually, separate from each other).
This is not what the load balancer cookie is for.
Just to clarify - Caddy distributes requests to backends following the logic, yes. The files are up to the backend to provide.
I’ve seen a lot of web apps which can serve static assets (JS, CSS, etc) regardless of which backend those requests were routed to. The static assets are simply uniquely named and uniquely referenced in the contexts in which they’re required, and sometimes versioned as well for cacheing purposes. I’m not sure why this strategy isn’t feasible in your case, but that’s okay.
As I see it you’ve described one problem here: any given client must make all its requests to the same backend. The load balancer cookie solves that by giving the client a cookie that tells Caddy which backend to group all its requests to. That is the sole purpose of the load balancer cookie.
Identifying each client is a completely different problem from the “all requests from the same client have to go to the same backend” problem. This problem is called authentication.
And I don’t understand why you can’t just set your own cookie for each new client. It’s very basic logic that needs be implemented:
If no client-identifier-cookie, return header Set-Cookie: client-identifier-cookie=<some SUFFICIENTLY RANDOM hash>.
Then use client-identifier-cookie. This is a very standard solution. Why is this infeasible in your case? Why is it better to change Caddy to do this for you when you can simply do it yourself? Even better - it makes you reverse-proxy-agnostic, which means you’re not REQUIRED to include Caddy in your technology stack for your application.
When there’s no available backends, Caddy loops back to the first backend. That is indeed how the cookie works.
This isn’t a problem unless you’re trying to use the cookie for authentication (i.e. uniquely identifying and differentiating individual clients). The answer to this problem is simple: Don’t rely on the lb cookie for that purpose. Set your own cookie.
Don’t rely on Caddy’s lb cookie to identify clients. It does not identify clients, it identifies backends. The value of the cookie is just the hash of one of the backends. For every client you get, you need to set your own client cookie if it doesn’t already have one. This needs to be part of your authentication process.
Why is your component relying on Caddy’s load balancer cookie to uniquely identify clients? It needs to NOT do this. That cookie can’t be relied on for this purpose whatsoever.
Modifying the load balancer cookie to make it useful for authentication will make it overcomplicated and less useful for its actual intended purpose, which is strictly for load balancing.
If it was Caddy’s job to solve session authentication, it would be better simply to propose an entirely new per-client cookie, completely irrespective of the load balancing process. But it’s not Caddy’s job to solve - it’s your webapp, which needs to authenticate users and can do so easily by setting browser session variables or setting its own cookie instead of trying to read Caddy’s load balancing cookie and relying on that.
Ignore the cookie value when you get it. Pretend the load balancer cookie doesn’t exist. It’s not useful to your webapp.
For every request, if client-identifier-cookie isn’t sent, set it. Then rely on that to identify each session moving forwards.
The way you know it’s a new client is that it doesn’t have client-identifier-cookie (or whatever you want to call your own cookie). So - validate the request, return a Set-Cookie header, and use that. Pretend Caddy isn’t there and rely on no Caddy-specific headers. Just have your webapp talk to the client as though it was talking directly to the client. Need a cookie? Set a cookie you can rely on.