Storage Offloading Recommendations

Help
1

I’ve got Caddy in a clustered environment and could use some advice on offloading storage.

My Environment:

  • The latest Caddy build
  • Inside Docker
  • On an Ubuntu virtual machine or three
  • Behind a load balancer
  • On Digital Ocean (though this can change).

Requirements/constraints:

  • I need to accomodate a few thousand domains initially, and more later, if all goes well.
  • I’m mostly a front-end dev, so keeping things simple and offloading as much as possible would be nice.

I’ve done some searching, but am a little fuzzy on which method might be considered the best practice. I’d hate to settle on something that works for now, only to have avoidable headaches later.

As for the options I’ve looked at:

The Redis plugin seems well-liked, but mentions it doesn’t work for clustered DB instances, which I believe is what Digital Ocean’s managed DB service is (which is what I’d be using).

I’m also nervous about storing important things like certs in memory. Wouldn’t a file system be more appropriate, especially for a large number of certs?

The Consul plugin is popular, but I’m completely unfamiliar with Consul and think it may be more practical to stick with methods more in line with my existing architecture/knowledge.

Network drives are also a possibility, but I’ve read on this forum that they can be problematic.

An s3 plugin is the most familiar to me, having used s3 a lot, and I’ve already got it working pretty well for my 2 or 3 test domains.

I’d love for s3 to be the ticket, but I’m worried that I might run into issues as the app scales up to managing thousands of certificates. Will it slow things down?

Would another option be better, like DynamoDB? If I must introduce a new system for storage, I’d prefer it to be something managed.

Any help or direction would be appreciated, thanks so much!

2 Likes
2

Nah, it’s fine. Certs are tiny in the grand scheme of things, and if they’re lost they can just get re-issued. Redis isn’t 100% in-memory, it has a mode that writes to disk to preserve data in case of a restart, see https://redis.io/topics/persistence

S3 is not good to use, it doesn’t properly handle locking.

Very expensive, unfortunately. Dynamo doesn’t do well with key scans, which Caddy performs to delete expired data periodically. Workaround is to make the cleanup interval very long, but it’s still very expensive overall compared to other options.

2 Likes
3

Thanks so much for the response! Would you recommend Redis, then?

4

I think that right now, it’s the most reliable implementation probably we’ve heard the least complaints about the Redis storage driver so far.

1 Like
5

I agree with Francis basically. Storage backend implementations are lacking in practice right now, but have the potential to be really good.

So I think the redis one is good for a lot of people.

There’s also this Postgres backend that has a lot of potential: GitHub - yroc92/postgres-storage

Any backend that can implement this Storage interface should work, though, and they’re pretty basic functions, really: certmagic package - github.com/caddyserver/certmagic - pkg.go.dev

1 Like
6

Thanks so much, guys!

7

This topic was automatically closed after 30 days. New replies are no longer allowed.