Caddy vs inotify

I’d like to be able to execute some hooks post cert renewal.

I’m aware that 2.6 added events, but there’s no module that’d use them in a standard distro, and I’d like to avoid using custom build, so I’m looking at inotify. Specifically, I’d like to use systemd.path mechanism (specifically, PathChanged=).

My question is which file I should monitor to be sure that renewal is completed and I can safely grab both cert and key, and be sure they are in-sync? Under /var/lib/caddy/certificates/acme-v02.api.letsencrypt.org-directory/example.com there are:

example.com.crt
example.com.json
example.com.key

Is it the .json that gets updated the last? Or .crt, or .key?

Thanks.

1 Like

The JSON file is currently the last one written. (That’s not an exported guarantee though.)

I would just watch the .crt file though since that’s the actual certificate you’re interested in… the key should only change if it gets deleted or if the key is compromised.

What do you want to do on cert renewal? We haven’t baked any handlers into the standard distribution yet because we don’t know what people need.

Hello.

Thank you for your answer.

I’d like to reload or restart specific services that use obtained certificates (like postfix, dovecot, asterisk, radicale, znc, turnserver, rsyncd). Currently, it works well for me with certbot, but since I’d like to get rid of nginxand migrate to caddy, I think I can ditch the certbot too.

In fact, I’ve even started preparing a daemon that listens on dbus, and my initial plan was to use caddy exec module to call something like dbus-send with a cert name, and the daemon would just do the rest. Now I’m in doubt since there’s no exec module in the standard distro, and I’m possibly looking for other solutions.

BTW, strictly speaking, I do not need exec. It would be just fine if caddy could perform a custom method call via dbus.

What do you think?

Tell me more about this – how does this work? Definitely open to looking into that.

Can you help me understand the requirement for no custom build?

I’m not sure how Go deals with dbus and what libraries are available there (quick search shows godbus), but I see it as following. There’s some daemon, independent from caddy, that resides on a system bus. The daemon exports some method (say, for instance, org.foobar.daemon.renew) that is meant to be called by caddy with a renewed hostname as a method parameter. Once caddy calls this method, the daemon does whatever job it wants to do after certificates are renewed (for instance, it can copy cert+key from caddy’s storage to somewhere else, chown/chmod them appropriately and reload/restart some other daemons).

It’s a matter of a) convenience (just use what distro gives you from its repos); and b) questionable production readiness and security concerns (aka why would I trust 3rd-party modules if they are not integrated and devs explicitly say there are concerns with regard to security, running arbitrary commands etc).

1 Like

Thanks for the info, I don’t know much about dbus, so that sounds pretty interesting. It’d probably make a good Caddy plugin for starters!

Fair point. I blame the distros / package managers for not being capable of doing what is needed to provide secure, statically linked binaries. So, this is unfortunate, indeed.

I honestly don’t know what security concerns the caddy-events-exec plugin has. We are just not including it out of an abundance of caution and because we don’t know enough people’s requirements or use cases yet. If the plugin turns out to be popular, we may merge it in.

Go is memory-safe, so you can’t just overflow a buffer and access arbitrary memory or run commands. I’m definitely not worried about that. My bigger concern over “arbitrary commands” is the user’s configuration: how easily can they configure an insecure server?

For example, if we allowed placeholders to be used where the command is specified, then it might definitely be possible for other entities to cause a value of a placeholder to be something other than what was intended, maybe via some sort of injection attack. I dunno. It’s purely speculative, but that’s why I didn’t implement placeholder support for the command to be run. (Placeholders are supported for arguments, just not the command itself.) You could imagine the damage that could be done if an arbitrary curl command was executable, for example.

Similarly, we don’t do any special parsing of the command. No shell expansion, no shell-like parsing. (Only placeholders on the arguments are allowed. But no bash syntax, for example.) You have to give the command and argument, one per token, pretty basic. This prevents weird results of parsing leading to running commands that aren’t expected.

Even with this, I could imagine a curl command being run with an argument that’s expanded from placeholders into something the user doesn’t expect, leading to curl making a request to some arbitrary endpoint…

So really, it’s more of a “here’s a separate plugin so you can shoot yourself in the foot unless you know what you’re doing” – and we’re not ready to bundle that kind of thing into the standard distribution until we understand the userscape better.

But I don’t know of any innate security vulnerabilities. I don’t think you need to be scared of it as long as you don’t create an abusable configuration.

My main worry is that if this functionality exists built into Caddy, then if someone managed to somehow gain access to changing the config via the admin API, then they could run arbitrary commands as the user Caddy runs as.

If Caddy is running as root (it shouldn’t… but still…) then it would be pretty much game-over. Even if not as root, cert private keys could be exfiltrated with curl or whatever.

Of course this requires gaining access to the admin API and that should be difficult, but it still worries me. IMO better to just not have the ability to run arbitrary commands built-in just in case to protect users who aren’t careful enough to properly protect access to the admin API.

dbus would indeed be safer in this regard then.

Yeah having dbus support built-in seems perfectly fine to me because it’s just message passing and someone needs to write something else on the other end for it to work, so the risk is low.

Trying to understand though (never used dbus before) – you said above Caddy should send a message to org.foobar.daemon.renew. This would obviously mean the user configures a certain event to be sent there.

Wouldn’t it make more sense to send it to com.caddyserver.events.cert_obtained so that we have a well-known “channel” (is that the right term?) for other processes to listen to? Because Caddy is defining the message structure, so shouldn’t it define the channel name too? I’m not sure how this typically works (whose job is it to define what etc).

The GitHub - godbus/dbus: Native Go bindings for D-Bus lib looks perfectly fine to me, no objections to pulling that in since it has no extra dependencies of its own and seems pretty well maintained.

PRs welcome if you want to take a crack at implementing this! We don’t really have the expertise here, so some help would be awesome. We’ll be glad to review an implementation :+1:

It would make sense for caddy to be able to call a custom method by name that is specified in caddy’s config file for a simple reason: there might be a need to make multiple calls to notify multiple daemons about certificate renewal.

And it is up to the daemon that listens to the bus to choose the name, hence it should not be hard-coded in caddy.

1 Like