Hi there! I’m building a proof-of-concept for hosting thousands to tens of thousands of websites fronted by Caddy. We’d be running multiple instances of Caddy behind a load balancer, and I have two questions that I couldn’t clarify in the docs (Automatic HTTPS — Caddy Documentation as well as the linked CADDY_CLUSTERING readme). If someone clarifies for me, I’d be happy to contribute documentation and even code snippets of our setup for others to use.
Is DNS challenge required once you cluster behind a load balancer? This wouldn’t work for our setup, as we don’t have programmatic access to each of our customers’ DNS providers, and would have to lean on ACME’s HTTP-based proof mechanisms. Older docs reference the DNS challenge being required for clustering behind a load balancer, but I could imagine that servers could coordinate for HTTP-based proofs.
Is there an example of using S3 for CADDY_CLUSTERING? The docs at GitHub - securityclippy/magicstorage: storage backends for certmagic don’t give much of a hint around configuration. As an alternative, we can opt for a shared mount via AWS’s EFS, but I figured I’d as before diving into that.
Thanks so much for your help. With a few pointers toward a completed proof-of-concept, I’d be delighted to contribute documentation to help the next person!
My first thought is that you should absolutely get a support plan for your use case. Caddy handles large clusters fairly well but we want to follow your deployment closely to help address any issues. We also want you to get in on an early beta of Caddy 2 and/or Caddy Enterprise if you’re interested.
Nope. As long as all the instances in the cluster share the same storage configuration, (CADDY_CLUSTERING), any challenge type should work (unless there are other reasons a specific challenge type won’t work, for example, if you terminate TLS before your cluster, the ALPN challenge will not work of course).
Allow me to again plug our commercial support, which I highly recommend for you. Because we’ve had customers report that EFS has been unreliable for this use case (not a Caddy issue, just a failure of EFS). DM me and I’ll hook you up with a deal for support.