Cache-Key generation or configuration: Trouble understanding the options

[bernd]

Hi ya!

I try to figure out for https://github.com/caddyserver/cache-handler how cache keys are generated or configurable. Use case is to know how to cache a site with some page variations based on cookies (mostly binary combinations like in AB testing or two languages).

I’m not so sure about the difference of the cache_keys and the key directive. The obvious one is that cache_keys allows me to be very specific by matching the URI. But they both share only disable_(body|host|method). And out of a sudden there is talk about GraphQL…

On the other hand headers says in the docs “List of headers to include to the cache” whereas there is a small note in the configuration example at https://github.com/darkweak/souin that says “Default headers concatenated in stored keys”.
So I would “to include in the cache” read as “in the response” and “stored in keys” related to the Cache algo figuring out what variations to return (e.g. three headers, times three variations).

I looked at other caches how cache keys can be computed, for example NGINX has this:
proxy_cache_key $scheme$proxy_host$uri$is_args$args;
I assume this is not how the Cache-handler works, there is no explicit Cache-Key option to be configured - or is it? - but a combination of body-host-method-header(s) distributed over keys/header directives?

[darkweak]

Hello, in the next release I’ll refactor that, te headers will move inside the cache_keys in addition to disable_(body|host|method) and we will store the header value in the key
e.g.

cache_keys {
    /account.php {
        headers Authorization Content-Type
    }
}

And you send a request on dummy.domain/account.php with the headers Authorization: foo Content-Type: text/jsonwill give you the key GET-dummy.domain-/account.php-foo-text/json

With souin you cannot really add a business logic on the cache key generation.

[bernd]

Hi! @darkweak sounds good!

So in order to craft the Cache-key to something specific it could become possible to do it like that?

map {header.Cookie} {a_or_b} {
	~Cookie:.*name=varianta a
	~Cookie:.*name=variantb b
	default ""
}

header X-Variant {a_or_b}

# globally set.
cache_keys {
    /* {
        headers Authorization Content-Type X-Variant
    }
}

That would come close enough to business logic.

[darkweak]

I think that should do the trick. Keep in mind that’s not already available in the latest Souin version (but released in few days) and you’ll have to move the map and header directives outside souin configuration (in your matcher)

[bernd]

Great, i will give it a try then.

Living on the edge
(not related in any way to this thread)

Dec 14 13:26:01 demovps caddy[3772170]: panic: Header called after Handler finished
Dec 14 13:26:01 demovps caddy[3772170]: goroutine 9041 [running]:
Dec 14 13:26:01 demovps caddy[3772170]: net/http.(*http2responseWriter).Header(0x9?)
Dec 14 13:26:01 demovps caddy[3772170]:         net/http/h2_bundle.go:6366 +0x7a
Dec 14 13:26:01 demovps caddy[3772170]: github.com/darkweak/souin/plugins.(*CustomWriter).Header(0xc009907758?)
Dec 14 13:26:01 demovps caddy[3772170]:         github.com/darkweak/souin@v1.6.24/plugins/base.go:55 +0x37

btw. Thanks for the devs and maintainers for Cadd for the add-package and remove-package experimental features. A real timesaver when testing things.

[darkweak]

Which module are you using?

[bernd]

The output was on a testserver. I might have run the following command during last week:

caddy add-package github.com/darkweak/souin/plugins/caddy@latest github.com/caddyserver/transform-encoder

it resulted in v1.6.24. I removed the package today (caddy remove-package github.com/darkweak/souin/plugins/caddy) after the crash.

[darkweak]

That’s weird I thought it was patched since a while.