Hi *,
1. The problem I’m having:
Every 2-3 days caddy suddenly stops returning a response and times out. systemctl status, however, says caddy is just working fine.
The only way to get caddy back working, is to restart it. Reloading doesn’t work. Interestingly, the CPU usage is very low (below 0.1) and RAM usage is fine, and it doesn’t use the swap partition at all. There’s enough free disk space (40GB+) Also, I don’t find any meaningful(to me) logs.
Prior changes: The only thing I did 2-3 weeks ago, was to run apt update && apt upgrade -y. Before that, caddy ran just fine.
I use caddy for a very long time now and on multiple servers, but never ever had this issue, nor I’m having it on the other servers with a very similar setup (Ubuntu LTS + Fail2Ban + eventually PHP, Maria, Redis, …). So any kind of help is very much appreciated. 
2. Error messages and/or full log output:
cURL (sry, I missed to add the L flag):
~ ❯ curl -v https://kgparl.de ↵ INT 4m 35s Ruby 2.7.2 07:26:47
* Host kgparl.de:443 was resolved.
* IPv6: (none)
* IPv4: 195.201.115.119
* Trying 195.201.115.119:443...
* Connected to kgparl.de (195.201.115.119) port 443
* ALPN: curl offers h2,http/1.1
* (304) (OUT), TLS handshake, Client hello (1):
* CAfile: /usr/local/share/curl/curl-ca-bundle.crt
* CApath: none
* (304) (IN), TLS handshake, Server hello (2):
* (304) (IN), TLS handshake, Unknown (8):
* (304) (IN), TLS handshake, Certificate (11):
* (304) (IN), TLS handshake, CERT verify (15):
* (304) (IN), TLS handshake, Finished (20):
* (304) (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / AEAD-AES128-GCM-SHA256 / [blank] / UNDEF
* ALPN: server accepted h2
* Server certificate:
* subject: CN=kgparl.de
* start date: Sep 15 14:12:41 2024 GMT
* expire date: Dec 14 14:12:40 2024 GMT
* subjectAltName: host "kgparl.de" matched cert's "kgparl.de"
* issuer: C=US; O=Let's Encrypt; CN=E5
* SSL certificate verify ok.
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://kgparl.de/
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: kgparl.de]
* [HTTP/2] [1] [:path: /]
* [HTTP/2] [1] [user-agent: curl/8.7.1]
* [HTTP/2] [1] [accept: */*]
> GET / HTTP/2
> Host: kgparl.de
> User-Agent: curl/8.7.1
> Accept: */*
>
* Request completely sent off
^C
Debug log from that cURL request (via journalctl):
Oct 01 07:27:19 kgparl caddy[1129]: {
"level": "debug",
"ts": 1727760439.8568072,
"logger": "events",
"msg": "event",
"name": "tls_get_certificate",
"id": "d569812a-04a9-432e-93e1-2becfc3b32b5",
"origin": "tls",
"data":
{
"client_hello":
{
"CipherSuites":
[
4866,
4867,
4865,
49200,
49196,
49192,
49188,
49172,
49162,
159,
107,
57,
52393,
52392,
52394,
65413,
196,
136,
129,
157,
61,
53,
192,
132,
49199,
49195,
49191,
49187,
49171,
49161,
158,
103,
51,
190,
69,
156,
60,
47,
186,
65,
49169,
49159,
5,
4,
49170,
49160,
22,
10,
255
],
"ServerName": "kgparl.de",
"SupportedCurves":
[
29,
23,
24,
25
],
"SupportedPoints": "AA==",
"SignatureSchemes":
[
2054,
1537,
1539,
2053,
1281,
1283,
2052,
1025,
1027,
513,
515
],
"SupportedProtos":
[
"h2",
"http/1.1"
],
"SupportedVersions":
[
772,
771,
770,
769
],
"RemoteAddr":
{
"IP": "84.161.182.XXX",
"Port": 58229,
"Zone": ""
},
"LocalAddr":
{
"IP": "195.201.115.119",
"Port": 443,
"Zone": ""
}
}
}
}
Oct 01 07:27:19 kgparl caddy[1129]: {
"level": "debug",
"ts": 1727760439.8569062,
"logger": "tls.handshake",
"msg": "choosing certificate",
"identifier": "kgparl.de",
"num_choices": 1
}
Oct 01 07:27:19 kgparl caddy[1129]: {
"level": "debug",
"ts": 1727760439.8569446,
"logger": "tls.handshake",
"msg": "default certificate selection results",
"identifier": "kgparl.de",
"subjects":
[
"kgparl.de"
],
"managed": true,
"issuer_key": "acme-v02.api.letsencrypt.org-directory",
"hash": "6105f9f8414cab9de102c9dafd39ccef161e61e66fcc557d36d1ef6ea6c3a65a"
}
Oct 01 07:27:19 kgparl caddy[1129]: {
"level": "debug",
"ts": 1727760439.8569632,
"logger": "tls.handshake",
"msg": "matched certificate in cache",
"remote_ip": "84.161.182.XXX",
"remote_port": "58229",
"subjects":
[
"kgparl.de"
],
"managed": true,
"expiration": 1734185561,
"hash": "6105f9f8414cab9de102c9dafd39ccef161e61e66fcc557d36d1ef6ea6c3a65a"
}
Oct 01 07:27:19 kgparl caddy[1129]: {
"level": "debug",
"ts": 1727760439.880284,
"logger": "http.handlers.rewrite",
"msg": "rewrote request",
"request":
{
"remote_ip": "84.161.182.XXX",
"remote_port": "58229",
"client_ip": "84.161.182.XXX",
"proto": "HTTP/2.0",
"method": "GET",
"host": "kgparl.de",
"uri": "/",
"headers":
{
"Accept":
[
"*/*"
],
"User-Agent":
[
"curl/8.7.1"
]
},
"tls":
{
"resumed": false,
"version": 772,
"cipher_suite": 4865,
"proto": "h2",
"server_name": "kgparl.de"
}
},
"method": "GET",
"uri": "/index.php"
}
Oct 01 07:27:19 kgparl caddy[1129]: {
"level": "debug",
"ts": 1727760439.8803756,
"logger": "http.handlers.reverse_proxy",
"msg": "selected upstream",
"dial": "127.0.0.1:9010",
"total_upstreams": 1
}
Oct 01 07:27:19 kgparl caddy[1129]: {
"level": "debug",
"ts": 1727760439.8805635,
"logger": "http.reverse_proxy.transport.fastcgi",
"msg": "roundtrip",
"request":
{
"remote_ip": "84.161.182.XXX",
"remote_port": "58229",
"client_ip": "84.161.182.XXX",
"proto": "HTTP/2.0",
"method": "GET",
"host": "kgparl.de",
"uri": "/index.php",
"headers":
{
"X-Forwarded-For":
[
"84.161.182.XXX"
],
"X-Forwarded-Proto":
[
"https"
],
"X-Forwarded-Host":
[
"kgparl.de"
],
"User-Agent":
[
"curl/8.7.1"
],
"Accept":
[
"*/*"
]
},
"tls":
{
"resumed": false,
"version": 772,
"cipher_suite": 4865,
"proto": "h2",
"server_name": "kgparl.de"
}
},
"env":
{
"DOCUMENT_URI": "/index.php",
"GATEWAY_INTERFACE": "CGI/1.1",
"REQUEST_METHOD": "GET",
"SCRIPT_FILENAME": "/var/www/kgparl/index.php",
"SCRIPT_NAME": "/index.php",
"REMOTE_IDENT": "",
"PATH_INFO": "",
"SERVER_SOFTWARE": "Caddy/v2.8.4",
"HTTP_HOST": "kgparl.de",
"SERVER_PORT": "443",
"CONTENT_TYPE": "",
"QUERY_STRING": "",
"REMOTE_PORT": "58229",
"REMOTE_USER": "",
"HTTP_X_FORWARDED_FOR": "84.161.182.XXX",
"CONTENT_LENGTH": "",
"REQUEST_URI": "/",
"SSL_PROTOCOL": "TLSv1.3",
"REMOTE_HOST": "84.161.182.XXX",
"SERVER_PROTOCOL": "HTTP/2.0",
"SSL_CIPHER": "TLS_AES_128_GCM_SHA256",
"HTTP_USER_AGENT": "curl/8.7.1",
"HTTP_X_FORWARDED_HOST": "kgparl.de",
"AUTH_TYPE": "",
"REMOTE_ADDR": "84.161.182.XXX",
"SERVER_NAME": "kgparl.de",
"HTTPS": "on",
"HTTP_ACCEPT": "*/*",
"HTTP_X_FORWARDED_PROTO": "https",
"REQUEST_SCHEME": "https",
"DOCUMENT_ROOT": "/var/www/kgparl"
},
"dial": "127.0.0.1:9010"
}
3. Caddy version:
v2.8.4 h1:q3pe0wpBj1OcHFZ3n/1nl4V4bxBrYoSoab7rL9BMYNk=
4. How I installed and ran Caddy:
As described here: Install — Caddy Documentation
Adding the Caddy sources and installing it via apt.
a. System environment:
Ubuntu 20.04.6 LTS
b. Command:
- Caddy runs as service and starts on boot.
- If necessary:
systemctl [stop|start|status] caddy - for
fmt,validateandreloadI use the caddy binary directly.
c. Service/unit/compose file:
➜ ~ cat /lib/systemd/system/caddy.service
# caddy.service
#
# For using Caddy with a config file.
#
# Make sure the ExecStart and ExecReload commands are correct
# for your installation.
#
# See https://caddyserver.com/docs/install for instructions.
#
# WARNING: This service does not use the --resume flag, so if you
# use the API to make changes, they will be overwritten by the
# Caddyfile next time the service is restarted. If you intend to
# use Caddy's API to configure it, add the --resume flag to the
# `caddy run` command or use the caddy-api.service file instead.
[Unit]
Description=Caddy
Documentation=https://caddyserver.com/docs/
After=network.target network-online.target
Requires=network-online.target
[Service]
Type=notify
User=caddy
Group=caddy
ExecStart=/usr/bin/caddy run --environ --config /etc/caddy/Caddyfile
ExecReload=/usr/bin/caddy reload --config /etc/caddy/Caddyfile --force
TimeoutStopSec=5s
LimitNOFILE=1048576
PrivateTmp=true
ProtectSystem=full
AmbientCapabilities=CAP_NET_ADMIN CAP_NET_BIND_SERVICE
[Install]
WantedBy=multi-user.target
d. My complete Caddy config:
Caddyfile:
# -----------------------------------------------------------------------------
# Global settings
# -----------------------------------------------------------------------------
{
debug
email user@example.com
http_port 80
https_port 443
}
# -----------------------------------------------------------------------------
# Snippets
# -----------------------------------------------------------------------------
import snippet.*
#
# vhost default config
#
# use it like this: import defaults "domain.tld"
#
(defaults) {
file_server
encode zstd gzip
import logfile {args[0]}
import cacheAssets
request_body {
max_size 350MB
}
}
www.kgparl.de {
redir https://kgparl.de{uri}
}
kgparl.de {
root * /var/www/kgparl
import defaults "kgparl.de"
php_fastcgi 127.0.0.1:9010 {
env SERVER_PORT 443
}
import wp
}
Imports:
➜ ~ cat /etc/caddy/snippet.*
#
# Cache all static files
#
(cacheAssets) {
@cachedFiles {
path *.jpg *.jpeg *.png *.gif *.ico *.woff *.woff2 *.css *.js *.svg
}
header @cachedFiles Cache-Control "public, max-age=604800, must-revalidate"
}
#
# Default logging
#
(logfile) {
log {
output file /var/log/caddy/{args[0]}.log
format json
}
}
#
# wordpress core rules
#
(wp) {
respond /xmlrpc.php 410
@wp-admin {
path not ^/wp-admin/*
}
rewrite @wp-admin {path}/index.php?{query}
}
5. Links to relevant resources:
Sry, none.
