Hi. I’d like to learn about the setups for those of you hosting multi-tenant apps with hundreds/thousands of distinct domains pointed at them. I’ve got a NodeJS app I’m looking to deploy soon that requires this functionality, but I’m having a hard time figuring out where to host and how to handle distribution through a CDN, as all of the major CDNs don’t make this functionality easy. So…
What hosting provider are you using?
How are you deploying to different regions? Via CDN or multiple app instances?
What else should I look out for when setting this up?
Please fill out the help topic template (click new topic and choose the help category, the text area will be filled with the template) and post it here – we need more context to better understand your situation to answer your questions.
You probably don’t want a hosting provider, you probably want infrastructure. AWS, Azure, GCP, Linode, Vultr, et all. Hosting providers are usually pretty narrow in terms of what you can run on their platform, and Caddy won’t fit.
As for whether you want multiple app instances, that depends more on your NodeJS app than on Caddy. That said, if you go with multiple instances, Caddy makes it pretty easy. Just configure a storage backend - or even just have all of them map their file system storage (the Caddy data folder, specifically) to the same NFS share through a private network/tunnel/etc. As long as all the Caddy servers have the same storage, they basically work together like magic.
As for the CDN, you can pair this with or without multiple app instances. Depending on how deep you cache the end result (you can cache full HTML or just assets), this might reduce the need for multiple app instances. That said, probably the simplest way by far is to configure a “pull” zone, and then have your app rewrite URLs for assets to your CDN’s zone URL.
I ran a few sites like this for a while on KeyCDN. My site was www.example.com and the zone was key.example.com. Past initial setup I didn’t need to configure the CDN at all; I just had my site issue asset links like key.example.com/image.jpg and the CDN would pull it dynamically from my server and then cache it. I didn’t use this for HTML caching, though - to ensure speed there, globally, I set up a few strategic geographical deployments, 2-4 in total. Not for load balancing, just for latency (CDN handled asset load).
Just for some context, my initial challenge was that the big CDN providers (Cloudfront, Cloudflare, Fastly, etc), don’t make pointing hundreds/thousands of domains at a distribution easy (or possible). So my best bet before now was to deploy a separate cloudfront distro for each domain, pointing all of them at the same app server. (Spread across multiple AWS accounts, because they have limits that prevent me from doing it all with one, with quotas for Route53, Cloudfront, Certificate Manager, and Cloudformation.)
But KeyCDN seems to fit the bill! The costs are an order of magnitude higher than cloudfront, but to be honest, the sites I’ll be managing won’t get a tremendous amount of traffic anyway. This is a great starting point. Thanks!
I run a multi tenant SAAS app, I use Zilore for geo DNS + failover, directing traffic by region to 2 Linode servers (one in USA, one in Australia) and use bunny cdn for serving front end Angular app assets + media storage.
I highly recommend those 3 services, they’ve been working very well for me
Serving the frontend app with bunnycdn is extremely easy, but I think I don’t do it in the ‘traditional’ way… the index.html file of the app is not served via cdn, it’s served via the caddy webservers (on Linode) but all the tenants share the same assets (js, css files) which are served via bunnycdn and fetched / kept updated automatically by bunny using a pull zone. Happy to explain in more detail if you’re interested, it’s a very simple setup that works well.
@jeremyj11 Thanks! How many domains are you managing for your app? I emailed Zilore to ask for their official max, but it would be nice to see the scale it’s functioning at for you.
My product is low-cost, low-traffic website hosting, so the problem I’m bumping into is that CDNs aren’t designed to support pointing 1000+ domains at a single instance.
250+ subdomains of 1 domain… so there is only 1 domain managed in Zilore.
are you managing 1000+ subdomains of a single domain? if yes then you only need the root domain in Zilore and no need to specify any subdomains at all in the cdn.
if it’s 1000+ root domains then Zilore will be too expensive
Yeah, I need to support apex domains. The cost isn’t that bad since my lowest tier is $10/mo, but what I’m really trying to do is reduce the complexity of scaling. Right now, my best option is to create multiple AWS sub-accounts to deal with their quotas (appox 200 users/account), and deploy a cloudfront distro per domain, pointing them all at the same shared app server. I’ve been moving forward on this angle, but I’ll take any chance to avoid having to manage this behemoth.
Why do you need a cloudfront distro per domain?
If they’re all using the same frontend app why not just have the app’s index.html serverd from the shared app server and have all the assets delivered from the cdn? With my app (Angular) it’s as simple as setting --deploy-url to the cdn url.
Every time I need to add a new tenant it’s just a matter of creating the db, folder structure and adding the domain to my caddy config file… no need to touch any settings on Zilore or Bunny or Linode.
For my app, the JS is shared, but everything else is unique to the users, including images, fonts (from pre-optimized bundles), and the server-side rendered html. So even though I am using a shared CDN for what can be shared, I would really prefer to have all of the user’s assets pulled to local servers for site viewers. My users are authors, and english-speaking readers is a worldwide demographic that skews towards bad internet connections.
The CF distro/site is because AWS only allows one SSL certificate, and as users sign up, it doesn’t make sense to keep regenerating certs until I hit 100, only to have to spin up another distro for the next batch. It just adds another thing to keep track of. (Not just site=>account, but site=>account=>distro).
That said, Zilore just got back to me—they support unlimited domains. So I was just gonna think through deploying the app across regions. It definitely reduces complexity, though it opens up managing servers rather than serverless scaling. Trade-offs!
Thanks for the ideas though. Hopefully I can make the jump.
