Setting up sites in Caddy when the DNS hasn't been set up yet

Help
1

I’ve been using Caddy on Windows servers for a while now, but there is one thing that really bothers me and I don’t know if there is a way to solve it or not.

Typically when I set up a new site using something like Apache I will create the config files I need for a new virtual host, save them, reload the config files by restarting (because I can’t reload, a problem that Caddy also has), and then going and setting up anything I need to in DNS, usually A name records to tell a domain which IP Address to point at.

This works great because it means that even in cases where an IP change for a domain might take hours to propagate everywhere in the world I can be certain that by the time the changes have propagated that the Apache server is already up and running and ready to serve the website.

I can’t do this with Caddy.

I don’t understand exactly how Let’s Encrypt works, but from my basic understanding it needs your domain to be set up correctly in order to do some verification (or something like that?) to provide a certificate for the site.

The fact that Caddy does this all automatically for me is amazing and it’s made my life so much easier in that regard.

The problem this brings up though is that if I set up a config file in Caddy for a new site which either hasn’t had it’s DNS set up yet, or a new server IP happens to have not propagated yet then as soon as I restart the server I get the following messages in the log files:

Activating privacy features...2017/12/04 23:46:52 [www.example.com] failed to get certificate: acme: Error 400 - urn:acme:error:connection - DNS problem: NXDOMAIN looking up A for www.example.com
Error Detail:
    Validation for www.example.com:80
    Resolved to:
        
    Used:

And then after a few of those messages I get this:

Activating privacy features...2017/12/04 23:46:54 [www.example.com] failed to get certificate: acme: Error 429 - urn:acme:error:rateLimited - Error creating new authz :: Too many failed authorizations recently.

This message then gets stuck in a loop and prevents every single other valid site from starting up.

Is there a way for Caddy to ignore sites that it can’t complete the configuration for on startup to avoid this issue? It seems a bit much for every single site to suffer down time because one domain’s new DNS settings hadn’t propagated yet.

I think that further complicating this issue is the fact that I have to use a Windows server. Caddy does provide a way to reload, rather than restart the server (I don’t know if that would solve my issue), but on Windows I can’t do it because it relies on a signal being sent to the process that doesn’t exist on Windows.

Is there not an API of some kind that Caddy could use to allow for universal access to the same controls across platforms? Apache has the same problem for me in that the reload command doesn’t work on Windows, so I have to restart that too, but the difference is that Apache can restart so fast that it makes no real difference despite serving hundreds of sites from it.

So in summary my questions are:

  • Can I configure Caddy to ignore sites with configurations that can’t be verified for certificates?
  • If not then why, and is it something that would be considered for a future version of Caddy?
  • If there isn’t an API for reloading (instead of restarting) then is it possible to build one in the form of a plugin?
  • If so then where should I start looking and what functions would I need to call?
2

Hi @Lucas, that’s a good explanation and good questions. I’ll do my best to answer.

It’s quite unfortunate indeed that you run Windows, and have a usage requirement for reloading functionality…

No, Caddy can’t check if verification is possible and ignore sites for which it is not possible.

It’s designed this way so that if you configure Caddy to serve a site with Automatic HTTPS, and there’s a problem, Caddy fails out early so you can fix the problem. It’s not likely that this design will change.

I haven’t delved into it, but it might be possible… Here’s the guide for extending Caddy, for reference:

Not sure for a plugin, but you could write a tiny Go program, embed Caddy, and restart it based on some trigger available to Go on Windows:

You might look into the On-Demand TLS feature:

You could write a site definition under a wildcard label, with the max_certs subdirective, and Caddy won’t provision a certificate until a client tries to connect to a matching host.

This would be a horridly complex solution requiring several compromises and having multiple pitfalls to be aware of. But it could work.

Better yet, use DNS validation. Give Caddy some credentials and it can get your certificates validated without needing to point A / CNAME records at all. Then you can have Caddy fully configured and ready to go well in advance and pointing DNS over can transition smoothly.

2 Likes
3

Thank you for the quick reply.

I had a feeling that the failure to start on a single site not being able to verify for a certificate would be by design, but thought I’d ask anyway on the off chance that it might be configurable.

Thank you for pointing me to the pages about plugins and giving me some ideas for that too. I’ll have a look into them when I get a bit more time. I’d completely forgotten that Caddy was embedable too, so like you say it might be easier to just embed it into some kind of wrapper program.

I didn’t even realise that the max_certs directive even existed, and it seems like it’s pretty much exactly what I wanted since it’s not really that I want to ignore failures, I just want to allow the server to start with configurations that may not have a domain actually set up yet and log anything about sites that do happen to fail. My original post is probably suffering a bit from the XY problem.

Since the max_certs directive allows a site to get a certificate on the first handshake it looks like it provides exactly what I was expecting.

I’ll have a look into the other options still, especially the embedding option, since the max_certs documentation does have a warning saying that it could be removed later based on what ACME CAs do, and I still want reload functionality, but for now max_certs solves my problem.

1 Like
4

There is one. It’s not perfect though and may not be good for some site as you temporarily lose https. You can force http on that domain while your site propagates then restart caddy 48-72h later with https on.

Try using “http://yoursite.com” in your caddyfile.

See: https://caddyserver.com/docs/http-caddyfile#addresses

5

To be unambiguous, @magikstm’s solution here is to disable Automatic HTTPS (and therefore disable serving this site on HTTPS at all) until your DNS has propagated and then manually update your Caddyfile and restart Caddy at a later date.

2 Likes
6

You could also use:
tls off

You may need to adjust your domain name to:
example.com:80

You could also use manual https in the meantime.

See: https://caddyserver.com/docs/tls

7

Thank you for your suggestion.

Doing something like that did briefly cross my mind, but I decided that I’m not willing to let any site run on HTTP only for any amount of time, so I scrapped that idea pretty early on. I could see how it would work for someone else though :slight_smile:

8

TLS off would only be for the one site…all your others sites would be normal https

www.example.com:2020 {
     bind localhost

     tls off

    errors /var/www/example.com/logs/error.log {
        rotate_size 100 # rotate after 100 MB
        rotate_age  14  # keep log files for 14 days
        rotate_keep 10  # keep at most 10 log files
        rotate_compress
     }

     root /var/www/example.com/my_site_files

     fastcgi / /run/php/php7.0-fpm.sock php

     filter rule {
        content_type (?:text|javascript)
        search_pattern https?://(?:www\.)?example\.(?:com|net|org)
        replacement https://www.example.com
     }
}

www.nextsite.com {
    ....
}
9

Just FYI, tls off is redundant with www.example.com:2020 (it’s not eligible for Automatic HTTPS because you’ve changed the port).

(consider me corrected, immediately below)

10

That’s not really accurate. Only port 80 disables automatic HTTPS.

1 Like
11

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.