I want to share a project that I’ve been working on full-time for the past two years, and it has turned into something better than I ever imagined. It is an all-in-one home server infrastructure and installer. But even more than that, it’s essentially about having your own private internet.
Caddy® is entirely instrumental in the architecture. This is probably the most novel implementation of Caddy you’ll ever find. Given present company, this is a bold statement, but hear me out.
First, let me talk about the HomeServerHQ project, and then I’ll explain how Caddy is used.
HomeServerHQ
The main site is https://www.homeserverhq.com. From there, you can link over to the Wiki site, Videos, Forum, etc. The GitHub repo is https://github.com/homeserverhq/hshq. The source code is composed of two bash scripts. One is a simple wrapper (hshq.sh), and the other (hshqlib.sh) is where everything is at. The hshqlib.sh is rather large, around 46k lines in total. It employs a function-based methodology (and it contains a lot of heredoc to output configurations). I decided early on to go with one single monolith script to maintain consistency with versioning and help with code-signing. Plus, it’ll be much easier from an ongoing auditing standpoint (there are only six files in the repository!).
Anyhow, the size/style of the script doesn’t matter, it’s what it does that’s important. The idea is to provide anyone, regardless of their background, the ability to set up their own full-scale, production-grade server, at home, on inexpensive off-the-shelf equipment, in a matter of minutes.
There’s so much to it, that I could go on for days on all of the different concepts employed. But rather than that, it’s better if you just see for yourself. On the videos site (https://videos.homeserverhq.com), there are currently only 4 videos:
- Introduction - which gives a broad overview of everything ~ 25 minutes
- A short video on installing Linux Ubuntu ~ 5 minutes
- Installation ~ 38 minutes
- Post-Installation ~ 41 minutes
The installation and post-installation videos are all you need to get fully set-up. They will show you how to do it all on cloud-based VPS’s, just for demo purposes, so you can test things out before committing the time and money to set up your own server at home. Depending on the provider, it should cost you less than 5 bucks if you only have the servers up for a couple of days. The full setup process may seem long and time-consuming (and boring) at first, but when you’re done…well…you’re done. Weeks, if not months of setup time will disappear in the blink of an eye.
Caddy
Once you’ve performed the base installation, either install all available services, or just ensure that you install code-server, which is vs-code in a browser. Log in to code-server, then in the Explorer area, you should see an HSHQ directory. Inside of that is a caddy subdirectory. There are two subdirectories in caddy: caddyfiles and snippets. These directories contain the main configuration files that you will need to edit on a regular basis. As you might be able to tell, they are pieces of the standard Caddyfile, and I’ll discuss this more shortly.
In general, Caddy is primarily used as a reverse proxy within this architecture. To explain how Caddy is used more specifically, let me walk through some problem scenarios, and how they are addressed with Caddy Awesomeness.
Problem
In the HSHQ architecture, you can invite other HomeServers to host on your private network. Conversely, you can host your services on other networks. Even though WireGuard® provides a strong outer layer of encryption, we still want to have secure connections inside of the network. Each server could distribute its root CA to the other devices, but this could get messy real quick, especially for the users/client devices. You could also have a single internal CA sign and distribute long-term wildcard certificates upon joining, but there are issues with this method as well.
Caddy to the Rescue
Thanks to the Smallstep integration, it is easy to run an internal CA to sign and issue certificates via the ACME protocol. In HSHQ, whenever you host your services on another network, a new instance of Caddy is created and bound to the WireGuard interface of the network that it connects to, and it will request the certificates from the remote internal CA authority. Certificates are refreshed on a regular basis automatically and seamlessly. Only one (long-term) root certificate to install per network. Profit.
Problem
Given that a Caddy instance is created for each network that your HomeServer is hosting on, each instance must have it’s own Caddyfile configuration. This, in turn, creates a problem with a ton of repetitive configuration code.
Caddy to the Rescue
In short, snippets = 
PURE GOLD If you look at the svcs.snip file in the snippets directory (described at the beginning of this section), each service that is being reverse-proxied is implemented only once inside of a snippet. Then, if you look in the caddyfiles directory, using only minimal import statements, there is little to no code repetition. It also makes it very easy to add/remove the services that you want to expose on a particular network. Snippets are also very useful with headers, tls, forward auth, remote_ip, etc.
Problem
Due to certificate trust chain issues with certain apps, typically (java-based) mobile apps, there are particular cases where a service needs to serve a certificate chain that is signed by a non-custom root CA (think LetsEncrypt/ZeroSSL). In the HSHQ architecture, for security reasons, the back-end HomeServer is not exposed to the internet for incoming requests, i.e. no ports open on the router. The only way to reach it is via the front-end RelayServer, which is also where the global DNS records point via the domain name provider. So how can we get LE certificates to the back-end? One way would be to let Caddy manage them on the RelayServer, and then copy them to the back-end on a regular automated basis, but this is a messy headache.
Caddy to the Rescue
On the back-end HomeServer, we add a normal reverse-proxy block in Caddy with Automatic HTTPS. On the front-end RelayServer, we only add a pin-sized hole to allow for the http challenge, basically ‘handle /.well-known/acme-challenge/*’, and reverse-proxy it to the back-end. When the back-end performs the request, the challenge passes through the RelayServer to the HomeServer and the certificate is issued, without needing to actually expose the service. I’ll be honest, I was surprised it even worked the first time I tried it, but it does, and it’s very stable even with renewals. I did have to add a small cron job on the back-end to regularly copy the certificates to a separate directory so that the other Caddy instances could serve them, but other than that Caddy does all of the heavy lifting. Works like a charm.
Problem
In some cases, one may want to actually expose a back-end service on the HomeServer to the public internet.
Caddy to the Rescue
Caddy handles this very easily via split-horizon DNS and a simple reverse proxy block. The request hits the RelayServer and is reverse proxied to the HomeServer. Additionally, with the trusted_proxies option, the requesting IP is seamlessly transferred to the back-end.
Summary
There are many other awesome things that Caddy does, in a simple and straightforward manner. When I first started prototyping this idea, I was using Traefik. Their logging was terrible, everything fell into a black hole. After some other frustrations, I started looking around, and discovered Caddy. I have never looked back since.
I do have one beef though, that I should bring up. Why on earth does Caddy enforce the unaligned braces style of indentation in the Caddyfile, aka the (quasi) Kernighan & Ritchie (K&R) style? There is only one correct method, braces should be aligned, aka the Allman style. Every other method is wrong.
A war amongst developers with two equal sides could be fought with these words, but I’m still putting it out there.
Jokes aside, when it comes to how the Caddy team designs, documents, and supports their software, this is how it should be done. This is The Way. There’s a little bit of a learning curve for the configuration process, but it’s not bad at all, especially given all of the complex things that you can do with it. The documentation explains everything in great detail. And if you can’t figure out something from the documentation, the forum fills in the gaps.
Conclusion
Caddy is an integral part of the HSHQ architecture that I designed and developed. It was selected due to its excellent design principles and ease of use. Most of the things that I mentioned in this post will never be seen by the average non-IT operator, because I’ve abstracted away the more difficult configuration details, and reduced most things to simple button clicks. However, it can be customized and extended easily by seasoned IT enthusiasts as well.
I just posted my code publicly a few months ago. It is very stable and highly polished. No one knows about this project yet, because I have not shared it with anyone. I will be posting on other forums and discussion threads in order to get the word out, but this was my first stop. If you find the project useful, pass it along.
