Caddy won't start using systemd service, runs w/ same command though

Help
1

I’ve followed the systemd service guide and I’m using caddy.service from that link.

When starting via systemctl, caddy does not start and exits.

When I log into www-data via sudo su -l www-data -s /bin/bash and then run
caddy -log stdout -agree=true -email carsten@chagemann.de -conf=/etc/caddy/Caddyfile -root=/var/tmp
caddy starts just fine and starts serving.

Any clue why this is happening? Debian 9, Caddy 0.10.10 with some plugins.

shell@debian-2gb-nbg1-dc3-3:/etc/systemd/system$ sudo systemctl status caddy.service 
● caddy.service - Caddy HTTP/2 web server
   Loaded: loaded (/etc/systemd/system/caddy.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Sun 2018-01-28 21:24:28 CET; 4s ago
     Docs: https://caddyserver.com/docs
  Process: 26946 ExecStart=/usr/local/bin/caddy -log stdout -agree=true -email carsten@chagemann.de -conf=/etc/caddy/Caddyfile -root=/var/tmp (code=exited, status=1/FAILURE)
 Main PID: 26946 (code=exited, status=1/FAILURE)

Jan 28 21:24:26 debian-2gb-nbg1-dc3-3 systemd[1]: Started Caddy HTTP/2 web server.
Jan 28 21:24:26 debian-2gb-nbg1-dc3-3 caddy[26946]: Activating privacy features...2018/01/28 21:24:26 [INFO] acme: Registering account for carsten@chagemann.de
Jan 28 21:24:27 debian-2gb-nbg1-dc3-3 caddy[26946]: 2018/01/28 21:24:27 [INFO][chagemann.de] acme: Obtaining bundled SAN certificate
Jan 28 21:24:28 debian-2gb-nbg1-dc3-3 caddy[26946]: 2018/01/28 21:24:28 [INFO][chagemann.de] AuthURL: https://acme-v01.api.letsencrypt.org/acme/authz/e0StpVHGjKdCZUoc-NeorvZx1SQXbSPfS42Rn
Jan 28 21:24:28 debian-2gb-nbg1-dc3-3 caddy[26946]: 2018/01/28 21:24:28 [INFO][chagemann.de] acme: Trying to solve HTTP-01
Jan 28 21:24:28 debian-2gb-nbg1-dc3-3 caddy[26946]: 2018/01/28 21:24:28 [chagemann.de] failed to get certificate: [chagemann.de] error presenting token: Could not start HTTP server for ch
Jan 28 21:24:28 debian-2gb-nbg1-dc3-3 systemd[1]: caddy.service: Main process exited, code=exited, status=1/FAILURE
Jan 28 21:24:28 debian-2gb-nbg1-dc3-3 systemd[1]: caddy.service: Unit entered failed state.
Jan 28 21:24:28 debian-2gb-nbg1-dc3-3 systemd[1]: caddy.service: Failed with result 'exit-code'.

Entire caddy.service file:

[Unit]
Description=Caddy HTTP/2 web server
Documentation=https://caddyserver.com/docs
After=network-online.target
Wants=network-online.target systemd-networkd-wait-online.service

[Service]
Restart=on-abnormal

; User and group the process will run as.
User=www-data
Group=www-data

; Letsencrypt-issued certificates will be written to this directory.
Environment=CADDYPATH=/etc/ssl/caddy

; Always set "-root" to something safe in case it gets forgotten in the Caddyfile.
ExecStart=/usr/local/bin/caddy -log stdout -agree=true -email carsten@chagemann.de -conf=/etc/caddy/Caddyfile -root=/var/tmp
ExecReload=/bin/kill -USR1 $MAINPID

; Use graceful shutdown with a reasonable timeout
KillMode=mixed
KillSignal=SIGQUIT
TimeoutStopSec=5s

; Limit the number of file descriptors; see `man systemd.exec` for more limit settings.
LimitNOFILE=1048576
; Unmodified caddy is not expected to use more than that.
LimitNPROC=512

; Use private /tmp and /var/tmp, which are discarded after caddy stops.
PrivateTmp=true
; Use a minimal /dev
PrivateDevices=true
; Hide /home, /root, and /run/user. Nobody will steal your SSH-keys.
ProtectHome=true
; Make /usr, /boot, /etc and possibly some more folders read-only.
ProtectSystem=full
; … except /etc/ssl/caddy, because we want Letsencrypt-certificates there.
;   This merely retains r/w access rights, it does not add any new. Must still be writable on the host!
ReadWriteDirectories=/etc/ssl/caddy

; The following additional security directives only work with systemd v229 or later.
; They further retrict privileges that can be gained by caddy. Uncomment if you like.
; Note that you may have to add capabilities required by any plugins in use.
;CapabilityBoundingSet=CAP_NET_BIND_SERVICE
;AmbientCapabilities=CAP_NET_BIND_SERVICE
;NoNewPrivileges=true

[Install]
WantedBy=multi-user.target
2

Alright, I found the issue.

Environment=CADDYPATH=/etc/ssl/caddy somehow broke it. I have no clue how, there’s probably a permission error there which is not displayed. With this commented out it works even as a service.

3

Thanks for posting your fix!

Baffling, though - it really looks like this (partial) log line is the culprit:

Jan 28 21:24:28 debian-2gb-nbg1-dc3-3 caddy[26946]: 2018/01/28 21:24:28 [chagemann.de] failed to get certificate: [chagemann.de] error presenting token: Could not start HTTP server for ch

I didn’t think that bad permissions on the SSL directory would stop the HTTP server starting up at all!

4

Yeah, I really wonder too. Actually, after removing the Environment line it now pulls the repos (I’m using the http.git plugin) and builds the needed stuff. Just after that, it fails to bind to :443.

I’ve run the setcap command. There is no other software listening on :443.

5

On a hunch, try uncommenting AmbientCapabilities=CAP_NET_BIND_SERVICE.

6

I don’t believe I have the exact same issue as carstenhag but this did solve my caddy breaking when upgrading from jessie to stretch. Thanks!

7

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.