I am evaluating Caddy for production use in a whitelabel enabled website (ala blogger, wix, etc.) for SSL termination and rate limiting. We have around 5000 customers who use this feature and almost all of them use a single third-party DNS provider.
Current Setup: www.aaa.com, www.bbb.com → A Record to Netscaler IP → Netscaler service group with two web servers
Planned Setup: www.aaa.com, www.bbb.com → A Record to Netscaler IP → Netscaler service group with caddy servers → Proxied to two web servers
After some research I’ve come up with the following plan and would really appreciate if you can review it
Use redis clustering to minimize LetsEncrypt work
Use wildcard * for hosts
Use tls with the ask directive
Start caddy with -agree and -email flags (hoping this reduces API load on LetsEncrypt)
Does this make sense and would I be able to rollout this change to 5000 domains without hitting LetsEncrypt rate limits? Will all cert rotations happen on the same time and cause another rate limit?
You can use http://, https:// as your site addresses to capture all standard traffic.
I’d advise you to carefully review the LetsEncrypt rate limit documentation in its entirety. Here’s some things that I think you might need to note in particular:
Renewals are treated specially: they don’t count against your Certificates per Registered Domain limit, but they are subject to a Duplicate Certificate limit of 5 per week. Note: renewals used to count against your Certificate per Registered Domain limit until March 2019, but they don’t anymore.
—Rate Limits - Let's Encrypt
So - no, they don’t count in aggregate and you can renew each certificate individually 5 times per week.
These are individual limits, so as long as no single customer has more than 49 subdomains + bare domain, you’re good. For those that do, you’ll need to consider a wildcard cert (which will require DNS validation).
For users of the ACME v2 API you can create a maximum of 300 New Orders per account per 3 hours.
—Rate Limits - Let's Encrypt
This might be what you need to pay the most attention to. While migrating to Caddy, with 5k domains, you might have to stagger this and slowly introduce more domains to Caddy over time.
You could alternately look into the rate limiting form linked from their docs page - you can give them some details, wait a few weeks, and they might raise the limits for you.