I’ve been using Caddy v2 in a clustered mode (globally distributed) and sharing storage using the redis plugin. I make heavy use of the TLS on demand feature as well.
The main issue I’m having with using Caddy this way is that the time it takes to negotiate TLS for each request can be a lot longer if the Caddy instance is far from the redis storage instance (for example, Japan to UK).
So first, does Caddy load the TLS information for every request from storage, or does it keep it in memory? If it keeps it in memory and just uses the storage as persistence, it might be something else I’m doing wrong.
If it does load it from storage every time, I’m wondering if anyone has any thoughts on the best way I can help solve this the latency issue for Caddy. I can probably find a solution for my own use-case outside of Caddy, though it might be hacky, but I’d like to see if I can come up with something useful for everyone.
- Use some kind of replicated, distributed filesystem. Seems like this could have holes though, and not sure how locks would work.
- Contribute to the redis plugin and get it working with redis clusters (maybe using a read-only replica when loading, but saving to main when creating/managing a cert?)
- A new plugin, potentially one that uses local (file?) storage and only checks a remote storage if the cert doesn’t exist locally. With on demand TLS it would be even slower on first request but would be fast after that. Not sure what the renewal flow would look like though.
- A contribution to Caddy core if you have any ideas that you think would be better in that direction
Let me know if any of you have thoughts or ideas on what would be the best way to solve this for all Caddy users. I’d love to contribute to and document whatever we think makes the most sense