OK, now I detected that there is a SysVinit file at https://github.com/caddyserver/caddy/tree/master/dist/init/linux-sysvinit (looks very similar to my working nginx script). I downloaded it, but now it’s getting more complicated: when I do service caddy start the script seemingly runs successful, and I get the syslog message
systemd[1]: Starting LSB: starts the caddy web server...
caddy[6262]: Starting caddy
but the server is not running anyway. service caddy status yields exited. Again, on the command line, Caddy starts flawlessly.