Caddy stopped receiving requests, and I don't know why

Help
1

I’ve also posted this question on Stack Overflow

I just noticed my website wasn’t serving anymore, so I went to investigate. There’s nothing in the logs that indicates anything is wrong, and yet the server is not responding to requests on any of my hosts, except port 2015. Here’s my full Caddyfile:

direct.frederikcreemers.be, www.frederikcreemers.be {
	root frederikcreemers.be/public
    log
    errors
    
	git {
    	repo https://gitlab.com/frederikcreemers/frederikcreemers.be.git
        path frederikcreemers.be
        hook /pushed none
        then caddy
        interval 3000
    }

     hugo frederikcreemers.be
}

media.frederikcreemers.be, localhost:8080 {
	
    log
	errors
	filemanager /manage {
		no_auth
        commands "git wget"
	}
    
	root public
	browse
	
	jwt {
		path /private
		redirect /login?next={path}
		allow sub frederikcreemers@gmail.com
		allow sub hildepeeters@gmail.com
		allow sub hilde.peeters@gmail.com
		allow sub elisabeth.creemers@gmail.com
		allow sub elisabethcreemers@gmail.com
        allow sub guidocreemers@gmail.com
	}
    
    jwt {
	path /manage
        redirect /private
        allow sub frederikcreemers@gmail.com
    }
	
    login {
        success_url /
        logout_url /
        google "client_id=foo,client_secret=bar,scope=https://www.googleapis.com/auth/userinfo.email https://www.googleapis.com/auth/plus.me"
        cookie_expiry 168h
	    jwt_expiry 168h
    }
}

creetris.frederikcreemers.be {
	root creetris
    errors {
    	404 index.html
    }
}

I ran the following command to find out which applications were listening on which ports:

netstat -tulpn

And here’s the only result I got or caddy:

Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp6       0           0             :::2015                      :::*                           LISTEN  25632/caddy

I’m running caddy using systemd. Here’s the unit file:

[Unit]
Description=Caddy HTTP/2 web server
Documentation=https://caddyserver.com/docs
After=network-online.target
Wants=network-online.target systemd-networkd-wait-online.service

[Service]
WorkingDirectory=/home/media
Restart=on-abnormal
Environment="PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin"

; User and group the process will run as.

; Always set "-root" to something safe in case it gets forgotten in the Caddyfile.
ExecStart=/usr/local/bin/caddy -log stdout -agree=true
ExecReload=/bin/kill -USR1 $MAINPID

; Use graceful shutdown with a reasonable timeout
KillMode=mixed
KillSignal=SIGQUIT
TimeoutStopSec=5s

; Limit the number of file descriptors; see `man systemd.exec` for more limit settings.
LimitNOFILE=1048576
; Unmodified caddy is not expected to use more than that.
LimitNPROC=512

; Use private /tmp and /var/tmp, which are discarded after caddy stops.
;PrivateTmp=true
; Use a minimal /dev
;PrivateDevices=true
; Make /usr, /boot, /etc and possibly some more folders read-only.
;ProtectSystem=full
; … except /etc/ssl/caddy, because we want Letsencrypt-certificates there.
;   This merely retains r/w access rights, it does not add any new. Must still be writable on the host!
;ReadWriteDirectories=/root/.caddy;/home/media

; The following additional security directives only work with systemd v229 or later.
; They further retrict privileges that can be gained by caddy. Uncomment if you like.
; Note that you may have to add capabilities required by any plugins in use.
CapabilityBoundingSet=CAP_NET_BIND_SERVICE
AmbientCapabilities=CAP_NET_BIND_SERVICE
NoNewPrivileges=true

[Install]
WantedBy=multi-user.target

Here are the logs I get for Caddy when I start it up using `systemctl start caddy``

May 01 19:20:21 ubuntu-512mb-fra1-01 systemd[1]: Started Caddy HTTP/2 web server.
May 01 19:20:21 ubuntu-512mb-fra1-01 caddy[25687]: Activating privacy features... done.
May 01 19:20:21 ubuntu-512mb-fra1-01 caddy[25687]: [WARNING] A database is going to be created for your File Manager instance at .caddy/filemanager/8f8ebb77287b3283b2cd93b625547577.db. It is highly recommended that you set th
May 01 19:20:21 ubuntu-512mb-fra1-01 caddy[25687]: [WARNING] A database is going to be created for your File Manager instance at .caddy/filemanager/96706b75e763a6accd04c4736aafe59c.db. It is highly recommended that you set th
May 01 19:20:21 ubuntu-512mb-fra1-01 caddy[25687]: [WARNING] A database is going to be created for your File Manager instance at .caddy/filemanager/a9edda69bab5269b7076ad6739f97621.db. It is highly recommended that you set th
May 01 19:20:21 ubuntu-512mb-fra1-01 caddy[25687]: [WARNING] A database is going to be created for your File Manager instance at .caddy/filemanager/ed3c77e098a98c5598578c43590134dd.db. It is highly recommended that you set th
May 01 19:20:21 ubuntu-512mb-fra1-01 caddy[25687]: [WARNING] A database is going to be created for your File Manager instance at .caddy/filemanager/d225eaa3d82bb1989154334fa90a61f0.db. It is highly recommended that you set th
May 01 19:20:22 ubuntu-512mb-fra1-01 caddy[25687]: From https://gitlab.com/frederikcreemers/frederikcreemers.be
May 01 19:20:22 ubuntu-512mb-fra1-01 caddy[25687]:  * branch            master     -> FETCH_HEAD
May 01 19:20:23 ubuntu-512mb-fra1-01 caddy[25687]: Already up-to-date.
May 01 19:20:23 ubuntu-512mb-fra1-01 caddy[25687]: 2018/05/01 19:20:23 https://gitlab.com/frederikcreemers/frederikcreemers.be.git pulled.
May 01 19:20:23 ubuntu-512mb-fra1-01 caddy[25687]: Activating privacy features... done.
May 01 19:20:23 ubuntu-512mb-fra1-01 caddy[25687]: http://:2015

I don’t see anything that could explain my issue. The filemanager warning is because I haven’t specified a database, but that’s because I’m using the login plugin rather than filemanager’s built-in authentication system.

Any idea what’s going on here, or what I should do to further investigate the issue?

2

I’m not seeing a -conf flag; where is your Caddyfile located on disk?

3

It’s in /home/media That’s why I have the working directory set there:

WorkingDirectory=/home/media

It used to work using this exact setup, and I’m trying to find out what I might have done that caused this to stop working.

4

This part makes me think that Caddy isn’t getting the Caddyfile correctly, because it should be printing a totally different list of hosts.

It looks like you’ve configured Caddy to run… Caddy again, once this git pull is complete (then caddy). It’s also not forked (which you would want to use then_long for instead), meaning it’s a blocking command; the http://:2015 hostname must be output from this child Caddy (defaulting to the basic Caddyfile?). The parent Caddy never proceeds to serve its actual sites, because it’s waiting for the child Caddy to exit.

https://caddyserver.com/docs/http.git

1 Like
5

Ah yep, looks like that might be the issue, checking that now.

I meant to put hugo there.

6

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.