Caddy under CentOS 8 Fedora Redhat

thanks for caddy and the support.

have exactly the same problem starting caddy 2 under CentOS8:

would have been cool if the SELinux fix would have been posted here:

https://caddy.community/t/caddyserver-on-fedora-systemd-problem/2935/6

imho in contrast to above forum post, could it be possible it has to do with:

  • “nologin” not possible

  • means: usr: caddy needs full login/bash capabilities to run caddy?

    adduser -r -d /var/www -s /sbin/nologin caddy
    curl -s https://raw.githubusercontent.com/mholt/caddy/master/dist/init/linux-systemd/caddy.service -o /etc/systemd/system/caddy.service
    systemctl start caddy; # results in
    ==> /var/log/messages <==
    Jan 15 21:15:06 cyclos-dev systemd[1]: Started Caddy HTTP/2 web server.
    Jan 15 21:15:06 cyclos-dev systemd[1731]: caddy.service: Failed to execute command: Permission denied
    Jan 15 21:15:06 cyclos-dev systemd[1731]: caddy.service: Failed at step EXEC spawning /usr/local/bin/caddy: Permission denied
    Jan 15 21:15:06 cyclos-dev systemd[1]: caddy.service: Main process exited, code=exited, status=203/EXEC
    Jan 15 21:15:06 cyclos-dev systemd[1]: caddy.service: Failed with result ‘exit-code’.

Caddy doesn’t need login / bash capabilities at all.

First thought - have you ensured that the Caddy binary has execute permissions?

chmod +x /usr/local/bin/caddy

If you’ve already got that handled, the next step is definitely to look at SELinux and why it might be denying systemd access to the Caddy binary.

thanks for the reply.

# version used    
/usr/local/bin/caddy --version
Caddy v1.0.4 (h1:wwuGSkUHo6RZ3oMpeTt7J09WBB87X5o+IZN4dKehcQE=)

caddy can be started as root with runuser: (consider it a workaround, but would still be nice if it would autostart as systemd service)

runuser -l caddy -c '/usr/local/bin/caddy -conf /etc/caddy/caddy.proxy.config'

binary has chmod ugo+x

ll /usr/local/bin/caddy
-rwxr-xr-x. 1 1002 1002 21M Nov 21 11:53 /usr/local/bin/caddy

caddy user has uid: 992

cat /etc/passwd|grep caddy
caddy:x:992:989::/var/www:/bin/bash

tried to fix it like this:

chown -R caddy:caddy /usr/local/bin/caddy

# allow non root users to bind to privileged ports (below 1000) 80/443
# https://superuser.com/questions/710253/allow-non-root-process-to-bind-to-port-80-and-443
setcap CAP_NET_BIND_SERVICE=+eip /usr/local/bin/caddy

systemctl start caddy.service
[root@domain.com ~]# systemctl status caddy.service
● caddy.service - Caddy HTTP/2 web server
   Loaded: loaded (/etc/systemd/system/caddy.service; enabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Sun 2020-01-19 16:24:15 CET; 1s ago
     Docs: https://caddyserver.com/docs
  Process: 1119 ExecStart=/usr/local/bin/caddy -log stdout -log-timestamps=false -agree=true -conf=/etc/caddy/caddy.proxy.config -root=/var/tmp (code=exited, status=203/EXEC)
 Main PID: 1119 (code=exited, status=203/EXEC)

Jan 19 16:24:15 domain.com.info systemd[1]: Started Caddy HTTP/2 web server.
Jan 19 16:24:15 domain.com.info systemd[1]: caddy.service: Main process exited, code=exited, status=203/EXEC
Jan 19 16:24:15 domain.com.info systemd[1]: caddy.service: Failed with result 'exit-code'.

… service still refuses to start.
this log file might have interesting details:

==> /var/log/audit/audit.log <==
type=SERVICE_START msg=audit(1579447742.468:392093): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=caddy comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"
type=AVC msg=audit(1579447742.483:392094): avc:  denied  { mounton } for  pid=1201 comm="(caddy)" path="/run/systemd/unit-root/etc/ssl/caddy" dev="dm-0" ino=100783586 scontext=system_u:system_r:init_t:s0 tcontext=unconfined_u:object_r:cert_t:s0 tclass=dir permissive=0
type=AVC msg=audit(1579447742.484:392095): avc:  denied  { execute } for  pid=1201 comm="(caddy)" name="caddy" dev="dm-0" ino=34595259 scontext=system_u:system_r:init_t:s0 tcontext=unconfined_u:object_r:user_tmp_t:s0 tclass=file permissive=0
type=SERVICE_STOP msg=audit(1579447742.487:392096): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=caddy comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=failed'UID="root" AUID="unset"

this seems to be the same problem: https://github.com/caddyserver/caddy/issues/2203

https://bugzilla.redhat.com/show_bug.cgi?id=1608548

it reads like this:

"Currently caddy is reusing httpd file labels to function with selinux enforcing.

https://src.fedoraproject.org/rpms/caddy/c/b85070ce0561800bcb5a2eeec08f5896786e2f68

That approach has the benefit of not having to maintain a caddy-specific policy. You’ve discovered a drawback to that approach, which is that selinux doesn’t think that httpd_exec_t type should be binding to 80/udp and 443/udp. I was hoping that I could just report this as bug on the httpd policy, but then I discovered that httpd doesn’t support QUIC, so it probably wouldn’t get much traction.

I’m going to do a bit more research and see what the best path forward would be."

actually using the caddy.service file from here: https://github.com/caddyserver/caddy/tree/master/dist/init/linux-systemd

groupadd --system caddy
useradd --system --gid caddy --create-home --home-dir /var/www --shell /usr/sbin/nologin --comment “Caddy web server” caddy

/usr/local/bin/caddy -conf /etc/caddy/caddy.proxy.config

setenforce 0; # temporarily disable SELinux, then caddy service works fine

so yes definately a SELinux problem.

systemctl status caddy; # works fine with SELinux disabled

caddy.service - Caddy HTTP/2 web server
Loaded: loaded (/etc/systemd/system/caddy.service; enabled; vendor preset: disabled)
Active: active (running) since Fri 2020-02-28 15:07:26 CET; 26s ago
Docs: Welcome — Caddy Documentation
Main PID: 3663 (caddy)
Tasks: 10 (limit: 26213)
Memory: 29.2M
CGroup: /system.slice/caddy.service
└─3663 /usr/local/bin/caddy -log stdout -log-timestamps=false -agree=true -conf=/etc/caddy/caddy.proxy.config -root=/var/tmp

did not solve the problem

https://github.com/caddyserver/caddy/issues/2203
semanage port -a -t http_port_t -p udp 80
semanage port -a -t http_port_t -p udp 443

with SELinux enabled get this error:

==> /var/log/audit/audit.log <==
type=SERVICE_START msg=audit(1582898762.666:821): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=caddy comm=“systemd” exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success’UID=“root” AUID=“unset”
type=AVC msg=audit(1582898762.675:822): avc: denied { mounton } for pid=3637 comm="(caddy)" path="/run/systemd/unit-root/etc/ssl/caddy" dev=“dm-0” ino=100783586 scontext=system_u:system_r:init_t:s0 tcontext=unconfined_u:object_r:cert_t:s0 tclass=dir permissive=0
type=SYSCALL msg=audit(1582898762.675:822): arch=c000003e syscall=165 success=no exit=-13 a0=5595e6302320 a1=5595e6302320 a2=0 a3=5000 items=0 ppid=1 pid=3637 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="(caddy)" exe="/usr/lib/systemd/systemd" subj=system_u:system_r:init_t:s0 key=(null)ARCH=x86_64 SYSCALL=mount AUID=“unset” UID=“root” GID=“root” EUID=“root” SUID=“root” FSUID=“root” EGID=“root” SGID=“root” FSGID=“root”
type=PROCTITLE msg=audit(1582898762.675:822): proctitle="(caddy)"
type=AVC msg=audit(1582898762.676:823): avc: denied { execute } for pid=3637 comm="(caddy)" name=“caddy” dev=“dm-0” ino=34595259 scontext=system_u:system_r:init_t:s0 tcontext=unconfined_u:object_r:user_tmp_t:s0 tclass=file permissive=0
type=SYSCALL msg=audit(1582898762.676:823): arch=c000003e syscall=59 success=no exit=-13 a0=5595e6336030 a1=5595e6428960 a2=5595e634d760 a3=5595e61b6010 items=0 ppid=1 pid=3637 auid=4294967295 uid=992 gid=989 euid=992 suid=992 fsuid=992 egid=989 sgid=989 fsgid=989 tty=(none) ses=4294967295 comm="(caddy)" exe="/usr/lib/systemd/systemd" subj=system_u:system_r:init_t:s0 key=(null)ARCH=x86_64 SYSCALL=execve AUID=“unset” UID=“caddy” GID=“caddy” EUID=“caddy” SUID=“caddy” FSUID=“caddy” EGID=“caddy” SGID=“caddy” FSGID=“caddy”
type=PROCTITLE msg=audit(1582898762.676:823): proctitle="(caddy)"
type=SERVICE_STOP msg=audit(1582898762.681:824): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=caddy comm=“systemd” exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=failed’UID=“root” AUID=“unset”

==> /var/log/messages <==
Feb 28 15:06:02 cyclos-dev systemd[1]: Started Caddy HTTP/2 web server.
Feb 28 15:06:02 cyclos-dev systemd[3637]: caddy.service: Failed to execute command: Permission denied
Feb 28 15:06:02 cyclos-dev systemd[3637]: caddy.service: Failed at step EXEC spawning /usr/local/bin/caddy: Permission denied
Feb 28 15:06:02 cyclos-dev systemd[1]: caddy.service: Main process exited, code=exited, status=203/EXEC
Feb 28 15:06:02 cyclos-dev systemd[1]: caddy.service: Failed with result ‘exit-code’.

1 Like

I wasn’t in this Forum a month ago. I run several VPSs on CentOS 8.
(I’m already more dnf than yum :grin:)
I had this problem the first time I used Caddy. This is the current solution.

It is temporary and will take enabled when the server restarts.
To disable it in the future.

# vi /etc/selinux/config

SELINUX=disabled 👈 Changed from "SELINUX=enforcing"

thanks for your reply.

dnf will at some point completely replace yum. (better faster higher and so on)

good to know.

so the current fix is to disable SELinux?

is this “safe”? X-D

not optimal… but better than always manually starting caddy like this:

runuser -l caddy -c '/usr/local/bin/caddy -conf /etc/caddy/caddy.proxy.config'

The official copr repository added support for CentOS/RHEL 8 yesterday. The packages there are integrated with selinux. If you don’t want to use those, you can use the %post section of the spec file as a guide to do it manually.

1 Like

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.