Best Practices for Hosting Many Websites

Help
1

What is the best way to proceed with this requirement?

I need to host many websites all with different domains. (hundreds or even thousands).

The domains are not known prior to caddy startup.

Each domain will be served out of a different directory based on the domain. For example xyz.com files will be in /path/xyz.com and abc.com will be served in /path/abc.com .

Is there some sort of static configuration I can install that will handle all domains?

p.s. I tried using the API to add domains on the fly and that works sort of… I make a http request to a php program that does curl to invoke the api, but the server reloads when I add the domain and the PHP never completes. At least I think that is what is happening.

2

I should just write a wiki article for this. Gimme a lil’ bit.

But real quick:

Is the PHP script served by Caddy? If any of the API requests go through Caddy first, it causes a deadlock because the API waits for the HTTP server to gracefully shut down, while the HTTP server waits for the API request to finish.

3

A wiki would be great!!!

Yes, the PHP is served by Caddy.

4

That’s why. You’ll either need to make your API request asynchronously (good luck w/ that in PHP) so the server can gracefully – rather than forcefully – shut down, or make your request directly to the API, or really through any route that doesn’t go through the server you’re trying to shut down.

5

I got a phone call and have to run for now, but here’s what I got so far: Serving tens of thousands of domains over HTTPS with Caddy

Edit: It’s almost 2am but I finished stubbing out the rest of the article. It could probably use some improvements but hopefully it’s enough to get you going.

1 Like
6

Not sure if this is best practice, but a part-time hobby is blossoming into a business generating a useful, supplementary, recurrent and passive hosting income and all thanks to Caddy (plus some clever sleight of hand). Using Caddy, I originally started out serving my own self-hosted WordPress blog blog.udance.com.au located on a FreeNAS server from home. Note: My ISP provides me with a residential broadband service and a dynamic IP address.

I’m not sure how it’s happened, word of mouth mostly, and it’s happened surprisingly quickly, I find myself hosting and developing mobile responsive WordPress sites for several customers and there’s no sign of that side business abating. I’m in cahoots with my brother who does the web design, while I focus on the web development side of the business. You can see an example of his animation work at readymcgetty.com.au (the animation appears after a few seconds. Compare the cartoon representation with the actual person here https://www.readymcgetty.com.au/about/. Note the striking likeness).

It’s not big business. There are some risks e.g. several single points of failure, but the current customer base (all small businesses) is able to tolerate this. The customer trade-off is a slightly elevated risk in exchange for considerably lower hosting costs and a personalised, local consulting service. As business needs change, I’ll begin to address some of these single points of failure.

I don’t serve customer sites using subdirectories (paths) or subdomains. They have their own custom domain names. Customers might use their own domain name providers e.g. godaddy.com, crazydomains.com.au, etc. In order to control the domain names, I use Cloudflare as the DNS hosting provider. Cloudflare is well supported by Caddy. The clever sleight of hand is having the custom domain names point to the public, dynamic IP address assigned to my broadband router.

Caddy serves two functions. First, it provides the reverse proxy service linking a custom domain name with the hosted WordPress service. Second, it acts as a web server serving PHP scripts for each WordPress site . I’ve separated these two functions out like in this example Case study: Variations in Caddyfile design for different Nextcloud builds. The reason for this is that Caddy is built from source with the Cloudflare module for the reverse proxy function, whereas, for each hosted website, the static Caddy binary will suffice.

Best practice measures I currently have in place:

  1. A focus on customer data - I use FreeNAS (a FreeBSD derivative) to host customer sites. The strength of FreeNAS is storage including disk redundancy, and onsite and offsite data replication.
  2. Rapid recovery for several scenarios e.g. server and router hardware redundancy.
  3. Integration of statuspage in Cloudflare and on hosted websites to inform clients/visitors of scheduled maintenance or system downtime.
  4. I’ve found the separation of duties between web designer and web developer a key factor in making the websites we’ve set up stand out. Web developers generally don’t make good web designers (I’m a classic example). Having my brother who is an amazing animator onboard has turned a typical, clinical looking website into something with a more human connection. This has given our websites an edge. If you can collaborate with a web designer, I strongly endorse the approach.
2 Likes
7

That wiki is awesome Matt. Really really really helpful. I read the Oh Dear link at the top and it was also helpful. Without this article (and oh dear) I would have never thought about needing to do an ask.

This solves 1/2 of the problem… getting the certificates for domains dynamically. AWESOME.

But the other problem is serving files from a directory based on the domain. I can do that in apache,
but is there a way to do this caddy? Ideally I would like to replace:

reverse_proxy localhost:9000

with something like

root * /path/{host}

This would make my infrastructure much simpler as a single caddy instance can do it all. Otherwise I think I am stuck with having to run apache as well to do the file serving.

8

Thank you Basil. This was really helpful to see how others are doing this.

I will share what I am doing (or trying to do) and people can pick it apart. I operate on the “do the simplest thing that could possibly work” principle that I learned from Extreme Programming. It has served me well and usually dictates my design decisions.

So what I am trying to build is a website that lets people create websites. The users login and design their websites and then deploy them when they are happy. The website builder is tailored to help people implement websites that follow the StoryBrand marketing framework (https://storybrand.com or the books Building a Storybrand, Marketing Made Simple). Hopefully this will be simple enough to use that people with little technical skill can implement their StoryBrand websites.

Technologies I am using are html/css/javascript for client side, and php for the server side application, I am using Linode for the server infrastructure, Debian as the server OS, and caddy/php-fpm as the middleware. I don’t have a DB, but instead use json flat files for my persistent store. I have used Linode for various projects for the last 8+ years and it is awesome!!! About 95% of the code I write is on the client side. The small amount of PHP on the server side is for login, registration, persistent store and user separation purposes.

The URL for my site is https://storybrandwebsites.com . I am hesitant to put this link in this post because it is still a work in progress as of January 20th, 2021. But in a month or two it should be good.

9

Yep, that’s exactly how you would do it.

Don’t forget that you need the file_server directive as well.

And you mentioned PHP, you’ll probably also want to use the php_fastcgi directive.

1 Like
10

THANK YOU. I tried that before and it didn’t work. I was trying to do * instead of https:// which didn’t work so I didn’t think any of it worked. Caddy is freaking AWESOME and the support on this forum is FABULOUS!!!

This makes my life so simple!

11

To finish this post… here is the Caddyfile I am using.

{
  on_demand_tls {
      ask https://storybrand.myphotoapp.com/php/sbCaddyAsk.php
      interval 2m
      burst    5
    }
}

https://

{

tls {
    on_demand
}

encode zstd gzip
file_server {
		index index.html
	}

root * /home/jimcook/storybrand/websites/{host}
php_fastcgi unix//run/php/php7.3-fpm.sock
log {
		output file /home/jimcook/storybrand/logs/web_access.log
}

}

storybrand.myphotoapp.com { 
 encode zstd gzip
 file_server
 root * /home/jimcook/storybrand/builder
 php_fastcgi unix//run/php/php7.3-fpm.sock
 log { 
	output file /home/jimcook/storybrand/logs/web_builder.log
     }
}

websites.myphotoapp.com { 
 encode zstd gzip
 file_server
 root * /home/jimcook/storybrand/websites
 php_fastcgi unix//run/php/php7.3-fpm.sock
 log { 
	output file /home/jimcook/storybrand/logs/web_builder.log
     }
}
12

I’d recommend running caddy fmt on that, it’ll clean up the syntax.

13

Well isn’t that spiffy. Another cool feature.

1 Like
14

Caddy fmt version:

{
	on_demand_tls {
		ask https://storybrand.myphotoapp.com/php/sbCaddyAsk.php
		interval 2m
		burst 5
	}
}

https:// {
	tls {
		on_demand
	}

	encode zstd gzip
	file_server {
		index index.html
	}

	root * /home/jimcook/storybrand/websites/{host}
	php_fastcgi unix//run/php/php7.3-fpm.sock
	log {
		output file /home/jimcook/storybrand/logs/web_access.log
	}
}

storybrand.myphotoapp.com {
	encode zstd gzip
	file_server
	root * /home/jimcook/storybrand/builder
	php_fastcgi unix//run/php/php7.3-fpm.sock
	log {
		output file /home/jimcook/storybrand/logs/web_builder.log
	}
}

websites.myphotoapp.com {
	encode zstd gzip
	file_server
	root * /home/jimcook/storybrand/websites
	php_fastcgi unix//run/php/php7.3-fpm.sock
	log {
		output file /home/jimcook/storybrand/logs/web_builder.log
	}
}
1 Like
15

I can’t quite figure it out… is there a way to also support redirecting www. domains to their base. Basically domain request I receive that starts with www, I want to redirect to without www. Something like:

www.* {
    redir https://{host without www}
}
16

You can use {labels.*} placeholders for this when redirecting

But you’ll need to do it using a header_regexp matcher on Host in your site block, because Caddy doesn’t support wildcards that aren’t on the left, and they must be single-segment (i.e. * doesn’t cross . boundaries)

17

This topic was automatically closed after 30 days. New replies are no longer allowed.