Minimizing downtime when moving an HTTPS site from one server to another

Wiki
1

When moving an HTTPS site from one server to another, while keeping the domain the same, and using the default HTTP challenge or TLS-ALPN challenge, there will likely be some downtime, where there will be an SSL error, while the DNS changes take effect. This downtime occurs after the DNS record is changed, and is related to the TTL of the DNS record.

This is because when the SSL Provider, such as Let’s Encrypt, does the ACME challenge, it sends a request to your domain name. It uses DNS to find the IP address for your domain name, and if the change in DNS records hasn’t taken effect, it will go to the old server.

For DNS to work well, it’s good to not have too small of a TTL, because the TTL is used for caching, and caching helps reduce the amount of work networks have to do.

However, you can temporarily reduce the TTL to a small number.

This explains it, in the last paragraph (“The easy solution to this inconsistent state is as follows”).

https://www.dnswatch.info/articles/dns-update

Paraphrased here:

  • jot down what the current TTL is. It’s in seconds. 3600 is 60 minutes, or 1 hour.
  • reduce the TTL of the DNS record to a small number, such as 300 (5 minutes) or 60 (1 minute). The above page also proposes 5 seconds, as part of multiple steps, but another site suggests going below 30 seconds won’t make a big difference. Only change the TTL in this step. Keep the value the same - the old IP address.
  • wait the old TTL value (if 3600, then 60 minutes)
  • change the value of the DNS record to the new IP address. Keep the TTL the same, for now.
  • make sure the change was successful
  • increase the TTL value to what it was before, or a new value, if you prefer. Some suggest a full day, but if you want to be able to migrate in a shorter period like 30 minutes, choose that time (30*60 = 1800 for 30 minutes).

Another way is to copy the SSL certificates from the old server to the new server. To do that, you need to locate the SSL certificates, securely copy them (using rsync), and put them in the correct location. If it’s from one Caddy server to another, they store the certificates in the same place, so it will be a simple rsync or scp from one caddy data directory to another. If it’s another HTTP server like nginx, the certificates will be elsewhere. If this is the case, one option besides copying from an nginx or apache config to a caddy config is to first install caddy on the old server and let its Automatic HTTPS acquire new certificates (which will be instant since it’s the same host). Since caddy ships as a single binary, this should be simple to set up even if you’re migrating, from, say, an old Linux distribution that no longer has up-to-date packages.

1 Like
2

Thanks for contributing, this is a good article for our wiki.

I’m not sure about this paragraph. The easiest way to do things is, in fact, to copy the certs to the new instance – especially if the default file system storage is used. There’s no “hunting” involved: the certs and keys are in Caddy’s data directory: Conventions — Caddy Documentation. If you use another storage module, they might be in a database or whatever other storage backend you have configured.

Just copy the data directory from the old server to the new one. Securely copying them could be as simple as an rsync command.

You don’t need to “configure Caddy to use them” – it will just use them. Hence, automatic HTTPS.

Don’t worry about turning on automatic HTTPS – it is on by default. Don’t worry about DNS propagation either, assuming the cert lifetimes are longer than the few hours it takes to propogate.

I think this article could be much simpler tbh.

3

You’re welcome, and thanks for the feedback. I changed it to not say to hunt for certificates, and added the suggestion of installing caddy on the old host if it isn’t already using caddy, which came to mind. I also removed the references to a couple related posts, to make it a bit shorter.

1 Like
4

Ah, I wasn’t sure whether you were writing about Caddy → Caddy migrations (which I assumed) or other servers. That’s what I get for reading it quickly at an airport. Might be good to divide this into sections based on scenario and individualize each section.

1 Like
5

That makes sense. Reducing the TTL can be easier when moving from a different http server, because of the variety of http servers, versions, and config file structures.

I can also suggest a DNS challenge, but as others have noted, many DNS services don’t provide fine grain permission API keys - DigitalOcean’s allows for spinning up servers (crypto mining, yay!). However, by using Docker or file permissions, you can carefully guard that key. You also could use a different DNS server that does have fine grained permissions - you don’t have to use the same DNS server as your host or domain registrar, and it is worth considering using a different DNS service for this reason. Also if for some reason you don’t want caddy to be in docker, you can make just the ACME part in Docker by using acme-dns once it has a driver, or DuckDNS.

There’s a good amount of info that can go into this article, but I want to make it so the typical reader doesn’t have to read much, and that it’s concise.