Caddyserver on Fedora, Systemd problem

Help
1

I’m basically having the same problem as stated here:

but was closed 3 days ago.

I spun up two servers on vultr, one running Ubuntu17.10, another one running Fedora 26. The Ubuntu one works with the systemd file and instructions stated here:

https://github.com/mholt/caddy/tree/master/dist/init/linux-systemd

but the F26 one does not. The only difference is the user under wich caddy runs, obviously fedora can’t use www-data with uid and gid 33, as it’s taken otherwise. Instead I use user ‘caddy’ as follows:

sudo useradd -r -d /var/www -s /usr/sbin/nologin caddy -U
sudo mkdir -p /var/www
sudo chown caddy:caddy /var/www
sudo chmod 555 /var/www

And executing caddy with the “caddy” user bash

sudo -u caddy bash

starts the caddy server as it should, using the same exec command as from the systemd service file:

/usr/local/bin/caddy -log stdout -agree=true -conf=/etc/caddy/Caddyfile -root=/var/tmp

However, starting it up via systemd, always ends up with the error:

● caddy.service - Caddy HTTP/2 web server
Loaded: loaded (/etc/systemd/system/caddy.service; disabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Wed 2017-11-01 00:11:48 CET; 4s ago
Docs: Welcome — Caddy Documentation
Process: 1475 ExecStart=/usr/local/bin/caddy -log stdout -agree=true -conf=/etc/caddy/Caddyfile -root=/var/tmp (code=exited, status=203/EXEC)
Main PID: 1475 (code=exited, status=203/EXEC)

Nov 01 00:11:48 f26 systemd[1]: Started Caddy HTTP/2 web server.
Nov 01 00:11:48 f26 systemd[1475]: caddy.service: Failed at step EXEC spawning /usr/local/bin/caddy: Permission denied
Nov 01 00:11:48 f26 systemd[1]: caddy.service: Main process exited, code=exited, status=203/EXEC
Nov 01 00:11:48 f26 systemd[1]: caddy.service: Unit entered failed state.
Nov 01 00:11:48 f26 systemd[1]: caddy.service: Failed with result ‘exit-code’.

and journalctl -xe gives me:

Nov 01 00:11:48 f26 audit[1472]: USER_START pid=1472 uid=0 auid=0 ses=5 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='op=PAM:session_open grantors=pam_keyinit,pam_
Nov 01 00:11:48 f26 sudo[1472]: pam_unix(sudo:session): session opened for user root by root(uid=0)
Nov 01 00:11:48 f26 systemd[1]: Started Caddy HTTP/2 web server.
-- Subject: Unit caddy.service has finished start-up
-- Defined-By: systemd
-- Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit caddy.service has finished starting up.
--
-- The start-up result is done.
Nov 01 00:11:48 f26 audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=caddy comm="systemd" exe="/usr/lib/systemd/systemd"
Nov 01 00:11:48 f26 sudo[1472]: pam_unix(sudo:session): session closed for user root
Nov 01 00:11:48 f26 audit[1472]: USER_END pid=1472 uid=0 auid=0 ses=5 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='op=PAM:session_close grantors=pam_keyinit,pam_l
Nov 01 00:11:48 f26 audit[1472]: CRED_DISP pid=1472 uid=0 auid=0 ses=5 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='op=PAM:setcred grantors=pam_env,pam_unix acct=
Nov 01 00:11:48 f26 audit[1475]: AVC avc:  denied  { mounton } for  pid=1475 comm="(caddy)" path="/etc/ssl/caddy" dev="vda1" ino=128077 scontext=system_u:system_r:init_t:s0 tcontext
Nov 01 00:11:48 f26 audit[1475]: AVC avc:  denied  { execute } for  pid=1475 comm="(caddy)" name="caddy" dev="vda1" ino=3735 scontext=system_u:system_r:init_t:s0 tcontext=unconfined
Nov 01 00:11:48 f26 systemd[1475]: caddy.service: Failed at step EXEC spawning /usr/local/bin/caddy: Permission denied
-- Subject: Process /usr/local/bin/caddy could not be executed
-- Defined-By: systemd
-- Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- The process /usr/local/bin/caddy could not be executed and failed.
--
-- The error number returned by this process is 13.
Nov 01 00:11:48 f26 audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=caddy comm="systemd" exe="/usr/lib/systemd/systemd"
Nov 01 00:11:48 f26 systemd[1]: caddy.service: Main process exited, code=exited, status=203/EXEC
Nov 01 00:11:48 f26 systemd[1]: caddy.service: Unit entered failed state.
Nov 01 00:11:48 f26 systemd[1]: caddy.service: Failed with result 'exit-code'.
Nov 01 00:11:53 f26 sudo[1478]:     root : TTY=pts/0 ; PWD=/root ; USER=root ; COMMAND=/bin/systemctl status caddy

I just spent the whole evening trying to figure out why, but to no success… Hope someone can help here or give me some clues what is wrong with the fedora setup. Please!

2

These lines look particularly interesting.

Does the caddy user/group have the correct permissions for that directory/device/binary? Is /usr/local/bin/caddy executable?

3

well, it is executable for everyone:

-rwxr-xr-x. 1 root root 17818944 Oct 31 23:27 /usr/local/bin/caddy

but that is not caddy user/group specific. I’m afraid you mean something else.

The user caddy is set up like this:

[root@f26 ~]# finger caddy
Login: caddy          			Name:
Directory: /var/www                 	Shell: /usr/sbin/nologin
Last login Tue Oct 31 23:25 (CET) on pts/0
No mail.
No Plan.
[root@f26 ~]# id caddy
uid=992(caddy) gid=992(caddy) groups=992(caddy)
[root@f26 ~]# cat /etc/passwd | grep caddy
caddy:x:992:992::/var/www:/usr/sbin/nologin
4

Quick update. Found the culprit: selinux. I’ve set it to permissive for now and it works. Anyone knows whats needed to make caddy work with selinux?

I guess would be good if that is noted somewhere in the systemd setup instructions for fedora, along with the gid/uid 33 not suitable also for fedora, and for ubuntu the www-data user is the default, so would not be necessary to write this down

5

Glad you got it working!

The init docs are maintained by the community, per https://github.com/mholt/caddy/tree/master/dist/init - we’d love to see a PR for documenting Fedora- or SELinux-specific issues.

Unfortunately I’m terrible at SELinux, though. Hopefully someone else around these forums has some expertise to share.