Which DNS providers should Caddy support by default?

Caddy 0.9 will support the ACME DNS challenge which allows you to obtain a certificate without the Let’s Encrypt servers having to contact your server directly; no need to start a listener or use port 80 or 443.

The catch is that you have to supply credentials to your DNS provider, and in order to automate this challenge, code has to be written to communicate with the DNS provider’s API.

Our underlying library, xenolf/lego, supports 10 providers already, possibly more in the future. It’s fairly easy to support these in Caddy, but it looks like this right now:

var dnsProv acme.ChallengeProvider
var err error
switch config.DNSProvider {
case "cloudflare":
	dnsProv, err = cloudflare.NewDNSProvider()
case "digitalocean":
	dnsProv, err = digitalocean.NewDNSProvider()
case "dnsimple":
	dnsProv, err = dnsimple.NewDNSProvider()
case "dyn":
	dnsProv, err = dyn.NewDNSProvider()
case "gandi":
	dnsProv, err = gandi.NewDNSProvider()
case "gcloud":
	dnsProv, err = gcloud.NewDNSProvider()
case "namecheap":
	dnsProv, err = namecheap.NewDNSProvider()
case "rfc2136":
	dnsProv, err = rfc2136.NewDNSProvider()
case "route53":
	dnsProv, err = route53.NewDNSProvider()
case "vultr":
	dnsProv, err = vultr.NewDNSProvider()
default:
	if config.DNSProvider != "" {
		return fmt.Errorf("unknown DNS provider '%s'", config.DNSProvider)
	}
}
if err != nil {
	return err
}
if dnsProv != nil {
	client.SetChallengeProvider(acme.DNS01, dnsProv)
}

(Update: Based on discussion below, we’ve changed the DNS providers to a pure plugin model.)

As you can see, each provider imports a different package. Some of these packages, such as route53 and cloudflare, import other packages that help them communicate with the provider. (Others are very lightweight and only use net/http). The size of the Caddy binary grows pretty quickly with this much code, especially if more providers get added later.

The way Caddy 0.9 works is that nearly component is a plugin, which means even the “built-in” providers will be plugged in, but they’ll be plugged in by default. Others, the user will have to select that plugin when they download or build Caddy.

So the question is: Which providers does Caddy support out-of-the-box? Any of them? All of them? Some of them?

In general I think as many thing as possible should be built in, mostly because of the friction involved with installing plugins presently. Its kinda a pain to deal with support requests for people that are not sure what plugins they have installed, and it is also kinda tricky to build caddy with custom plugins if you don’t use the build server.

In the case of dns providers, I also don’t like including a bunch of what I would consider the “fringe” providers that almost nobody will use. If they are implemented only using net/http as dependencies they probably wont add much as far as binary size, but I worry they will add maintenance issues. Code that doesn’t run doesn’t get tested, and tends to have more hidden bugs. I don’t want caddy to be responsible for those bugs.

1 Like

Well, I’m an internet.bs user, so… internet.bs!

I also suggest that Caddy doesn’t support any by default, and instead incorporates a very low-friction method to fetch the appropriate DNS config. There are so many DNS providers in the world, it’d be IMO wasteful to support every single one when the average user is gonna interact with one single DNS provider.

1 Like

@alfiepates internet.bs is a no go for api access. I have tried to get tokens from them and their position is if you are not a reseller they won’t give you access. Unfortunate. You could use another provider for dns and point your internet.bs nameservers at cloudflare or r53 or something.

I have moved my domains to other providers. Been pretty happy with name.com.

@captncraig Gaddammit. I like them for their social engineering-resistance and no-bullshit attitude (and decent support compared to a CERTAIN COMPANY that I shan’t name…cheap) but if they’re skittish about API access then no DNS challenge for me, it seems!

What about Mail-in-A-Box ? It has an API built-in. I use it as my main email server, and the DNS works fine.

Do they have a documented api for setting/removing txt records? If so, it would be fairly easy to add it to xenolf/lego. I couldn’t find that documentation though.

Yes. It follows like so.

# sets laptop.mydomain.com to point to the IP address of the machine you are executing curl on
curl -X PUT https://box.domain.com/admin/dns/custom/laptop.mydomain.com

# deletes that record and all A records for that domain name
curl -X DELETE https://box.domain.com/admin/dns/custom/laptop.mydomain.com

# sets a CNAME alias
curl -X PUT -d "bar.mydomain.com." https://box.domain.com/admin/dns/custom/foo.mydomain.com/cname

# deletes that CNAME and all CNAME records for that domain name
curl -X DELETE https://box.domain.com/admin/dns/custom/foo.mydomain.com/cname

# adds a TXT record using POST to preserve any previous TXT records
curl -X POST -d "some text here" https://box.domain.com/admin/dns/custom/foo.mydomain.com/txt

# deletes that one TXT record while preserving other TXT records
curl -X DELETE -d "some text here" https://box.domain.com/admin/dns/custom/foo.mydomain.com/txt

then it should be pretty easy. Submit a PR to lego, and he’ll probably take it.

Ehh, I’m not too good with Go, sorry!

DNS Made Easy would be a good one to provide

1 Like

most of these suggestions are best made in GitHub - go-acme/lego: Let's Encrypt client and ACME library written in Go. This is simply a question of “should caddy support everything lego does, or just some of them?”

I wonder if there is some way to not need to explicitly add them into caddy, but if lego had a registration functionality where we could say NewDNSProvider("cloudflare","abc","123") and the user would just specify all configuration items in one line:
dns cloudflare abc 123, or something like that. Maybe better to have caddy ignorant of what is available.

1 Like

I concur, my webserver shouldn’t give a crap about who my DNS provider is, that should be abstracted away.

Let’s keep the topic focused on whether we should include any into Caddy by default, and if so, which ones (of the ones that are available) – if you want to add more providers, then you can implement it in Go in your own package; doesn’t even have to be added to lego. Just have to satisfy their interface.

As for keeping Caddy ignorant of your provider, that would be nice, but it has to know which code to execute to negotiate the challenge with your provider’s API. I’m not sure there’s a way around that.

It would require lego have a registration api of some sort, and a generic GetProvider type api. Not sure if they provide anything like that. If they had a providers/all package that imported all of the providers they have, maybe that could provide a convenient way to import them.

It would be unfortunate if multiple projects had to be modified each time someone added a provider to lego. Much better to only update one place and get the benefit everywhere.

Of course that type of solution would likely require work from and with @xenolf

I also know that desire has been expressed to implement those providers with as few dependencies as possible. Not sure if any work is being done to replace the heavy libs with simple net/http only based solutions.

It would be unfortunate if multiple projects had to be modified each time someone added a provider to lego. Much better to only update one place and get the benefit everywhere.

Providers aren’t really “added” to lego, except maybe in the CLI. The underlying acme package is completely agnostic to any DNS providers, I think, except maybe the “manual” provider which waits until you set the record yourself (no API calls). Basically, each provider is a separate package, and there is no central place for them. You just import the package if you use that provider. Then it’s ultimately one line of code to set acme to use that provider. In Caddy’s case, we have to map string values (received from the Caddyfile) to the provider’s package. That’s all the big switch statement does.

I’m inclined to not bundle any in by default, and let users choose the one(s) they need. Most users won’t need more than one or two.

1 Like

I’m inclined to not bundle any in by default, and let users choose the one(s) they need. Most users won’t need more than one or two.

+1, being provider-agnostic is a good thing.

I would say if a provider adds no dependencies there is no reason not to include it in caddy.

If they can be included as seperate plugins, that is an option, but I assume you want to use this like so:

tls { 
  dns cloudflare abc def
}

How could a plugin tell the tls middleware that the cloudflare provider is available? Would be like another mini plugin system for dns providers.