Caddy not starting via systemctl on Ubuntu 20

Help
1

I wonder if someone is able to help with this issue.

1. Caddy version (caddy version):

v2.2.0 h1:sMUFqTbVIRlmA8NkFnNt9l7s0e+0gw+7GPIrhty905A=

2. How I run Caddy:

systemctl start caddy

a. System environment:

Just running on vanilla Ubuntu droplet

b. Command:

systemctl start caddy

d. My complete Caddyfile or JSON config:

Here is the contents of /etc/systemd/system/caddy.service

#
# See https://caddyserver.com/docs/install for instructions.
#
# WARNING: This service does not use the --resume flag, so if you
# use the API to make changes, they will be overwritten by the
# Caddyfile next time the service is restarted. If you intend to
# use Caddy's API to configure it, add the --resume flag to the
# `caddy run` command or use the caddy-api.service file instead.

[Unit]
Description=Caddy
Documentation=https://caddyserver.com/docs/
After=network.target

[Service]
User=caddy
Group=caddy
ExecStart=/usr/bin/caddy run --environ --config /etc/caddy/Caddyfile
ExecReload=/usr/bin/caddy reload --config /etc/caddy/Caddyfile
TimeoutStopSec=5s
LimitNOFILE=1048576
LimitNPROC=512
PrivateTmp=true
ProtectSystem=full
AmbientCapabilities=CAP_NET_BIND_SERVICE

[Install]
WantedBy=multi-user.target

3. The problem I’m having:

When I start caddy using caddy run as the root user (the user I compiled and installed Caddy as), everything is fine. Caddy works.

When I run systemctl start caddy caddy doesn’t run. I can only presume that this is because systemctl runs caddy as the caddy user? But why might this be failing? From what I can tell, I have followed every step in the build and install instructions.

4. Error messages and/or full log output:

I see the following error in syslog:

Oct  4 19:52:00 systemd[1]: Started Caddy.
Oct  4 19:52:00 caddy[1299]: caddy.HomeDir=/var/lib/caddy
Oct  4 19:52:00 caddy[1299]: caddy.AppDataDir=/var/lib/caddy/.local/share/caddy
Oct  4 19:52:00 caddy[1299]: caddy.AppConfigDir=/var/lib/caddy/.config/caddy
Oct  4 19:52:00 caddy[1299]: caddy.ConfigAutosavePath=/var/lib/caddy/.config/caddy/autosave.json
Oct  4 19:52:00 caddy[1299]: caddy.Version=v2.2.0
Oct  4 19:52:00 caddy[1299]: runtime.GOOS=linux
Oct  4 19:52:00 caddy[1299]: runtime.GOARCH=amd64
Oct  4 19:52:00 caddy[1299]: runtime.Compiler=gc
Oct  4 19:52:00 caddy[1299]: runtime.NumCPU=1
Oct  4 19:52:00 caddy[1299]: runtime.GOMAXPROCS=1
Oct  4 19:52:00 caddy[1299]: runtime.Version=go1.14.3
Oct  4 19:52:00 caddy[1299]: os.Getwd=/
Oct  4 19:52:00 caddy[1299]: LANG=C.UTF-8
Oct  4 19:52:00 caddy[1299]: PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin
Oct  4 19:52:00 caddy[1299]: HOME=/var/lib/caddy
Oct  4 19:52:00 caddy[1299]: LOGNAME=caddy
Oct  4 19:52:00 caddy[1299]: USER=caddy
Oct  4 19:52:00 caddy[1299]: INVOCATION_ID=ec2dcb2f46414308920590079793cf71
Oct  4 19:52:00 caddy[1299]: JOURNAL_STREAM=9:25713
Oct  4 19:52:00 caddy[1299]: {"level":"info","ts":1601841120.8948157,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":""}
Oct  4 19:52:00 caddy[1299]: {"level":"info","ts":1601841120.8987932,"logger":"admin","msg":"admin endpoint started","address":"tcp/localhost:2019","enforce_origin":false,"origins":["localhost:2019","[::1]:2019","127.0.0.1:2019"]}
Oct  4 19:52:00 caddy[1299]: {"level":"info","ts":1601841120.8991697,"logger":"caddy.storage.redis","msg":"TLS Storage are using Redis, on 127.0.0.1:6379"}
Oct  4 19:52:00 caddy[1299]: run: loading initial config: loading new config: loading storage module: loading module 'redis': provision caddy.storage.redis: dial tcp 127.0.0.1:6379: connect: connection refused
Oct  4 19:52:00 systemd[1]: caddy.service: Main process exited, code=exited, status=1/FAILURE
Oct  4 19:52:00 systemd[1]: caddy.service: Failed with result 'exit-code'.

5. What I already tried:

Just running it with caddy run. I’m at a bit of a loss to be honest.

2

I actually think this is a problem with the Redis module actually. If I comment that out in my Caddyfile it appears to work:

Oct  4 21:37:44  caddy[1038]: {"level":"info","ts":1601847464.8917046,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":""}
Oct  4 21:37:44  caddy[1038]: {"level":"info","ts":1601847464.8969595,"logger":"admin","msg":"admin endpoint started","address":"tcp/localhost:2019","enforce_origin":false,"origins":["localhost:2019","[::1]:2019","127.0.0.1:2019"]}
Oct  4 21:37:44  caddy[1038]: {"level":"info","ts":1601847464.8989015,"logger":"http","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443}
Oct  4 21:37:44  caddy[1038]: {"level":"info","ts":1601847464.899105,"logger":"http","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
Oct  4 21:37:44  caddy[1038]: {"level":"info","ts":1601847464.9051778,"msg":"autosaved config","file":"/var/lib/caddy/.config/caddy/autosave.json"}
Oct  4 21:37:44  caddy[1038]: {"level":"info","ts":1601847464.9055257,"msg":"serving initial configuration"}
Oct  4 21:37:44  caddy[1038]: {"level":"info","ts":1601847464.9086277,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc00024ac40"}
Oct  4 21:37:44  caddy[1038]: {"level":"info","ts":1601847464.9091916,"logger":"tls","msg":"cleaned up storage units"}

It is perhaps because the Redis module isn’t correctly reading the values from the file. I have this set in my Caddyfile which works when executing caddy run:

{ 
  storage redis {
    host        {$CADDY_CLUSTERING_REDIS_HOST}
    port        {$CADDY_CLUSTERING_REDIS_PORT}
    password    {$CADDY_CLUSTERING_REDIS_PASSWORD}
    db          {$CADDY_CLUSTERING_REDIS_DB}
    tls_enabled {$CADDY_CLUSTERING_REDIS_TLS}
    aes_key     {$CADDY_CLUSTERING_REDIS_AESKEY}
  }
}

The environment variables are definitely present and even if I replace them with hard-coded values in the Caddyfile, they are not being picked by when the Redis module loads via systemctl. I can tell this because:

 127.0.0.1:6379: connect: connection refused

These are the default Redis values and not what is in my Caddyfile.

3

Hi @pagec,

The environment variables are definitely present

Are you sure about this or are you assuming it?

I ask because when Caddy starts it prints the environment, which clearly does not show those variables:

Oct 4 19:52:00 caddy[1299]: LANG=C.UTF-8
Oct 4 19:52:00 caddy[1299]: PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin
Oct 4 19:52:00 caddy[1299]: HOME=/var/lib/caddy
Oct 4 19:52:00 caddy[1299]: LOGNAME=caddy
Oct 4 19:52:00 caddy[1299]: USER=caddy
Oct 4 19:52:00 caddy[1299]: INVOCATION_ID=ec2dcb2f46414308920590079793cf71
Oct 4 19:52:00 caddy[1299]: JOURNAL_STREAM=9:25713

4

Hey Matt, sorry, this is baffling me. I have just now resorted to adding this in the /etc/systemd/system/caddy.service file:

[Service]
Environment="CADDY_CLUSTERING_REDIS_HOST=[MYHOST]"
Environment="CADDY_CLUSTERING_REDIS_PORT=[MYPORT]"
...etc

And this seems to have fixed it! :woman_facepalming:

Sorry about the trouble.

1 Like
5

This topic was automatically closed after 30 days. New replies are no longer allowed.