Note: The module already exists for nearly a year now, but I wanted to open a topic here in addition to the Repository as well.
Repository: oltdaniel/caddy-storage-valkey
This is a native valkey storage backend for caddy. It makes use of the official golang client for valkey, which already includes all bells and whistles that are required to make a caddy storage backend work, resulting in only requiring 3 direct dependencies (caddy, certmagic and valkey-go).
{
storage valkey {
address 127.0.0.1:6379
}
}
mydomain.localhost {
respond "Hello World" 200
}
Configuration
The main thing my module does is, exposing all the various configuration options that are available in the golang client to the Caddyfile.
You can simply use a single node, a full cluster, read-only replicas, authentication, TLS, … Recently I added support for caddy placeholders in specific configuration options to allow for even more flexibility when configuring.
To see all the various options exposed, visit the README. It is quite a long list.
Storage structure
In order to store the files from caddy in valkey, I needed to decide on a data structure. Valkey, as does Redis, offers multiple different variants. As I need to store the metadata of each file, like size and last modified timestamp.
In the end I chose the hash data structure, which simplified a lot. I just need a single key (the filepath) and store the attributes and values in it. This means, not only can I easily range over the keys and retrieve the entries, but one can easily navigate the structure manually in valkey in case of manual changes (like cleaning up certificates or migrating it).
Benchmarking & Testing
In order to test the module under load, I created a small benchmark script which creates a Caddyfile with a few thousand domains and short lifetimes. Due to the simple structure and native client performance, no issues occurred this far.
For testing, I made use of the docker-compose.yml provided by the golang client library for testing various server setups, which allowed me to verify all the configuration options behaving as expected. Which in the end is no magic, as I just parse them and pass them to the golang client options struct.
If you have any feedback in regards to the code, feel free to comment.