I’ve been running a single caddy instance for a while now and its been working amazingly. I want to add a bit more fault tolerance to my application and have 2 caddy servers behind a load balancer. I have a couple of questions.
Not sure if there is any documentation on this anywhere?
Is there a way for Caddy to share its configuration?
I’m using automatic TLS for Custom Domains. Is there a way to share those certs between servers?
Any Caddy instances that are configured to use the same storage will automatically share those resources and coordinate certificate management as a cluster.
So, simply configure them to use the same storage (for example, same shared folder on a mounted file system, or use another storage backend): JSON Config Structure - Caddy Documentation
Above my root /mnt/efs/fs1 is where I mounted my file system. It can be anywhere technically. Like: /etc/caddy/certificates or whatever.
(edit)
The only load balancer that will work for you in this case is the TCP Network Load Balancers, the application network balancer will not pass HTTPS traffic unless the certificate is added in the load balancers.
If you do go through this, getting the healthcheks right took me some time. Check out my other question and you’ll see the solution.
So I would have to send an API call for each server in the cluster?
If I set the XDG_CONFIG_HOME env variable to point to the same storage across all servers would that possibly work?
So after a little experiment. It looks like it could work. But I would need to watch the autosave.json file for changes and run sudo systemctl restart caddy or something.
So i tried to do a reload but i get this error. Any idea why?
Failed to reload caddy.service: Job type reload is not applicable for unit caddy.service.
See system logs and 'systemctl status caddy.service' for details.
#
# For using Caddy with its API.
#
# This unit is "durable" in that it will automatically resume
# the last active configuration if the service is restarted.
[Unit]
Description=Caddy
Documentation=https://caddyserver.com/docs/
After=network.target
[Service]
Environment="XDG_CONFIG_HOME=/mnt/efs/fs1"
User=ubuntu
Group=www-data
ExecStart=/usr/bin/caddy run --watch --environ --resume
TimeoutStopSec=5s
LimitNOFILE=1048576
LimitNPROC=512
PrivateTmp=true
ProtectSystem=full
AmbientCapabilities=CAP_NET_BIND_SERVICE
[Install]
WantedBy=multi-user.target
If you use --resume, the config file is what was resumed from before (which only changes if you update the config through the API). Usually you won’t use --watch with --resume.
This did update when i changed the caddy.json file. So I think if I have 2 caddy servers share that config file. I can update the json file from my webapp when I add a custom domain.
I’ll let you know if this works. Can you see any issues with this?