1. Output of caddy version
:
v2.6.2 h1:wKoFIxpmOJLGl3QXoo6PNbYvGW4xLEgo32GPBEjWL8o=
2. How I run Caddy:
sudo systemctl start caddy.service
sudo systemctl.status.caddy.service
● caddy.service - Caddy
Loaded: loaded (/lib/systemd/system/caddy.service; enabled; vendor preset: enabled)
Active: active (running) since Sat 2022-11-05 13:26:17 UTC; 5s ago
Docs: https://caddyserver.com/docs/
Main PID: 2955 (caddy)
Tasks: 9 (limit: 4433)
CGroup: /system.slice/caddy.service
└─2955 /usr/bin/caddy run --environ --config /etc/caddy/Caddyfile
Nov 05 13:26:17 userdomain caddy[2955]: {"level":"info","ts":1667654777.0024934,"logger":"tls","msg":"cleaning storage unit","description":"FileStorage:/var/lib/caddy/.local/share/caddy"}
Nov 05 13:26:17 userdomain caddy[2955]: {"level":"info","ts":1667654777.0026119,"logger":"http","msg":"enabling HTTP/3 listener","addr":":443"}
Nov 05 13:26:17 userdomain caddy[2955]: {"level":"info","ts":1667654777.0029204,"msg":"failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 2048 kiB, got: 416 kiB). See https://github.com/lucas-clemente/quic-go/wiki/UD>
Nov 05 13:26:17 userdomain caddy[2955]: {"level":"info","ts":1667654777.003377,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2","h3"]}
Nov 05 13:26:17 userdomain caddy[2955]: {"level":"info","ts":1667654777.003602,"logger":"http.log","msg":"server running","name":"remaining_auto_https_redirects","protocols":["h1","h2","h3"]}
Nov 05 13:26:17 userdomain caddy[2955]: {"level":"info","ts":1667654777.0036354,"logger":"http","msg":"enabling automatic TLS certificate management","domains":["sub.userdomain.co.uk"]}
Nov 05 13:26:17 userdomain caddy[2955]: {"level":"info","ts":1667654777.0052269,"logger":"tls","msg":"finished cleaning storage units"}
Nov 05 13:26:17 userdomain caddy[2955]: {"level":"info","ts":1667654777.0060437,"msg":"autosaved config (load with --resume flag)","file":"/var/lib/caddy/.config/caddy/autosave.json"}
Nov 05 13:26:17 userdomain caddy[2955]: {"level":"info","ts":1667654777.0062468,"msg":"serving initial configuration"}
Nov 05 13:26:17 userdomain systemd[1]: Started Caddy.
a. System environment:
Ubuntu 20.04LTS on rPi 4 8GB
b. Command:
sudo systemctl start caddy.service
d. My complete Caddy config:
sub.userdomain.com {
log {
level INFO
output file /var/log/caddy/caddy.log {
roll_size 10MB
roll_keep 10
}
}
tls user@userdomain.com
encode gzip
reverse_proxy localhost:8080
}
3. The problem I’m having:
Did an apt upgrade, rebooted & caddy is no longer working as the reverse proxy that had been running for over a year with no problems
4. Error messages and/or full log output:
Nov 05 13:11:02 userdomain caddy[2384]: HOME=/var/lib/caddy
Nov 05 13:11:02 userdomain caddy[2384]: LOGNAME=caddy
Nov 05 13:11:02 userdomain caddy[2384]: USER=caddy
Nov 05 13:11:02 userdomain caddy[2384]: INVOCATION_ID=e7f2e3679dab44babad0b7d832e55392
Nov 05 13:11:02 userdomain caddy[2384]: JOURNAL_STREAM=9:32697
Nov 05 13:11:02 userdomain caddy[2384]: {"level":"info","ts":1667653862.572663,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":""}
Nov 05 13:11:02 userdomain caddy[2384]: {"level":"warn","ts":1667653862.5763557,"msg":"Caddyfile input is not formatted; run the 'caddy fmt' command to fix inconsistencies","adapter":"caddyfile","file":"/etc/caddy/Caddyfile","line":2}
Nov 05 13:11:02 userdomain caddy[2384]: {"level":"info","ts":1667653862.5788927,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
Nov 05 13:11:02 userdomain caddy[2384]: {"level":"info","ts":1667653862.5794866,"logger":"http","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443}
Nov 05 13:11:02 userdomain caddy[2384]: {"level":"info","ts":1667653862.5795627,"logger":"http","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
Nov 05 13:11:02 userdomain caddy[2384]: {"level":"info","ts":1667653862.5795789,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0x400058ad90"}
Nov 05 13:11:02 userdomain caddy[2384]: {"level":"info","ts":1667653862.5804887,"logger":"tls","msg":"cleaning storage unit","description":"FileStorage:/var/lib/caddy/.local/share/caddy"}
Nov 05 13:11:02 userdomain caddy[2384]: {"level":"info","ts":1667653862.5806038,"logger":"http","msg":"enabling HTTP/3 listener","addr":":443"}
Nov 05 13:11:02 userdomain caddy[2384]: {"level":"info","ts":1667653862.58086,"msg":"failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 2048 kiB, got: 416 kiB). See https://github.com/lucas-clemente/quic-go/wiki/UDP-Receive-Buffer-Size for details."}
Nov 05 13:11:02 userdomain caddy[2384]: {"level":"info","ts":1667653862.5811415,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2","h3"]}
Nov 05 13:11:02 userdomain caddy[2384]: {"level":"info","ts":1667653862.5813503,"logger":"http.log","msg":"server running","name":"remaining_auto_https_redirects","protocols":["h1","h2","h3"]}
Nov 05 13:11:02 userdomain caddy[2384]: {"level":"info","ts":1667653862.5822768,"logger":"http","msg":"enabling automatic TLS certificate management","domains":["sub.userdomain.co.uk"]}
Nov 05 13:11:02 userdomain caddy[2384]: {"level":"info","ts":1667653862.5856338,"logger":"tls","msg":"finished cleaning storage units"}
Nov 05 13:11:02 userdomain caddy[2384]: {"level":"info","ts":1667653862.588049,"msg":"autosaved config (load with --resume flag)","file":"/var/lib/caddy/.config/caddy/autosave.json"}
Nov 05 13:11:02 userdomain caddy[2384]: {"level":"info","ts":1667653862.5882688,"msg":"serving initial configuration"}
Nov 05 13:11:02 userdomain systemd[1]: Started Caddy.
Nov 05 13:12:57 userdomain caddy[2384]: {"level":"info","ts":1667653977.1751099,"msg":"shutting down apps, then terminating","signal":"SIGTERM"}
Nov 05 13:12:57 userdomain caddy[2384]: {"level":"warn","ts":1667653977.1754177,"msg":"exiting; byeee!! 👋","signal":"SIGTERM"}
Nov 05 13:12:57 userdomain caddy[2384]: {"level":"info","ts":1667653977.1760979,"logger":"tls.cache.maintenance","msg":"stopped background certificate maintenance","cache":"0x400058ad90"}
Nov 05 13:12:57 userdomain caddy[2384]: {"level":"info","ts":1667653977.1776397,"logger":"admin","msg":"stopped previous server","address":"localhost:2019"}
Nov 05 13:12:57 userdomain caddy[2384]: {"level":"info","ts":1667653977.1777112,"msg":"shutdown complete","signal":"SIGTERM","exit_code":0}
Nov 05 13:12:57 userdomain systemd[1]: Stopping Caddy...
Nov 05 13:12:57 userdomain systemd[1]: caddy.service: Succeeded.
Nov 05 13:12:57 userdomain systemd[1]: Stopped Caddy.
Nov 05 13:13:04 userdomain systemd[1]: Starting Caddy...
Nov 05 13:13:04 userdomain caddy[2477]: caddy.HomeDir=/var/lib/caddy
Nov 05 13:13:04 userdomain caddy[2477]: caddy.AppDataDir=/var/lib/caddy/.local/share/caddy
Nov 05 13:13:04 userdomain caddy[2477]: caddy.AppConfigDir=/var/lib/caddy/.config/caddy
Nov 05 13:13:04 userdomain caddy[2477]: caddy.ConfigAutosavePath=/var/lib/caddy/.config/caddy/autosave.json
Nov 05 13:13:04 userdomain caddy[2477]: caddy.Version=v2.6.2 h1:wKoFIxpmOJLGl3QXoo6PNbYvGW4xLEgo32GPBEjWL8o=
Nov 05 13:13:04 userdomain caddy[2477]: runtime.GOOS=linux
Nov 05 13:13:04 userdomain caddy[2477]: runtime.GOARCH=arm64
Nov 05 13:13:04 userdomain caddy[2477]: runtime.Compiler=gc
Nov 05 13:13:04 userdomain caddy[2477]: runtime.NumCPU=4
Nov 05 13:13:04 userdomain caddy[2477]: runtime.GOMAXPROCS=4
Nov 05 13:13:04 userdomain caddy[2477]: runtime.Version=go1.19.2
Nov 05 13:13:04 userdomain caddy[2477]: os.Getwd=/
Nov 05 13:13:04 userdomain caddy[2477]: LANG=C.UTF-8
Nov 05 13:13:04 userdomain caddy[2477]: PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin
Nov 05 13:13:04 userdomain caddy[2477]: NOTIFY_SOCKET=/run/systemd/notify
Nov 05 13:13:04 userdomain caddy[2477]: HOME=/var/lib/caddy
Nov 05 13:13:04 userdomain caddy[2477]: LOGNAME=caddy
Nov 05 13:13:04 userdomain caddy[2477]: USER=caddy
Nov 05 13:13:04 userdomain caddy[2477]: INVOCATION_ID=f4bf885304704217b6f6c7fdc550c18a
Nov 05 13:13:04 userdomain caddy[2477]: JOURNAL_STREAM=9:36875
Nov 05 13:13:04 userdomain caddy[2477]: {"level":"info","ts":1667653984.296173,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":""}
Nov 05 13:13:04 userdomain caddy[2477]: {"level":"warn","ts":1667653984.3000758,"msg":"Caddyfile input is not formatted; run the 'caddy fmt' command to fix inconsistencies","adapter":"caddyfile","file":"/etc/caddy/Caddyfile","line":2}
Nov 05 13:13:04 userdomain caddy[2477]: {"level":"info","ts":1667653984.3025498,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
Nov 05 13:13:04 userdomain caddy[2477]: {"level":"info","ts":1667653984.3030994,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0x4000696e00"}
Nov 05 13:13:04 userdomain caddy[2477]: {"level":"info","ts":1667653984.303185,"logger":"http","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443}
Nov 05 13:13:04 userdomain caddy[2477]: {"level":"info","ts":1667653984.3032284,"logger":"http","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
Nov 05 13:13:04 userdomain caddy[2477]: {"level":"info","ts":1667653984.304105,"logger":"tls","msg":"cleaning storage unit","description":"FileStorage:/var/lib/caddy/.local/share/caddy"}
Nov 05 13:13:04 userdomain caddy[2477]: {"level":"info","ts":1667653984.304253,"logger":"http","msg":"enabling HTTP/3 listener","addr":":443"}
Nov 05 13:13:04 userdomain caddy[2477]: {"level":"info","ts":1667653984.3045518,"msg":"failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 2048 kiB, got: 416 kiB). See https://github.com/lucas-clemente/quic-go/wiki/UDP-Receive-Buffer-Size for details."}
Nov 05 13:13:04 userdomain caddy[2477]: {"level":"info","ts":1667653984.3048193,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2","h3"]}
Nov 05 13:13:04 userdomain caddy[2477]: {"level":"info","ts":1667653984.3050392,"logger":"http.log","msg":"server running","name":"remaining_auto_https_redirects","protocols":["h1","h2","h3"]}
Nov 05 13:13:04 userdomain caddy[2477]: {"level":"info","ts":1667653984.3050745,"logger":"http","msg":"enabling automatic TLS certificate management","domains":["sub.userdomain.co.uk"]}
Nov 05 13:13:04 userdomain caddy[2477]: {"level":"info","ts":1667653984.308436,"logger":"tls","msg":"finished cleaning storage units"}
Nov 05 13:13:04 userdomain caddy[2477]: {"level":"info","ts":1667653984.3114586,"msg":"autosaved config (load with --resume flag)","file":"/var/lib/caddy/.config/caddy/autosave.json"}
Nov 05 13:13:04 userdomain caddy[2477]: {"level":"info","ts":1667653984.3117049,"msg":"serving initial configuration"}
Nov 05 13:13:04 userdomain systemd[1]: Started Caddy.
5. What I already tried:
I recall having a problem when I first installed Caddy, & after the usual searching found a stack exchange article that suggested running:
sudo setcap 'cap_net_bind_service=+ep' /usr/bin/caddy
which got the service running - I must admit I blidnly followed the suggestion as I was despearate to get it to work, & this worked. I also admit not knowing exactly what this does, but it got Caddy working back then.
I’ve tried stopping caddy.service:
sudo systemctl stop caddy
& then running from command line (instead of as a service):
caddy reverse-proxy --from :443 --to :8080 &
which returned:
[1] 2846
ubuntu@userdomain:~$ 2022/11/05 13:23:49.546 WARN admin admin endpoint disabled
2022/11/05 13:23:49.546 INFO http server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS {"server_name": "proxy", "https_port": 443}
2022/11/05 13:23:49.546 INFO http enabling automatic HTTP->HTTPS redirects {"server_name": "proxy"}
2022/11/05 13:23:49.546 INFO tls.cache.maintenance started background certificate maintenance {"cache": "0x40004479d0"}
2022/11/05 13:23:49.547 INFO http enabling HTTP/3 listener {"addr": ":443"}
2022/11/05 13:23:49.548 INFO failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 2048 kiB, got: 416 kiB). See https://github.com/lucas-clemente/quic-go/wiki/UDP-Receive-Buffer-Size for details.
2022/11/05 13:23:49.548 INFO http.log server running {"name": "proxy", "protocols": ["h1", "h2", "h3"]}
2022/11/05 13:23:49.548 INFO http.log server running {"name": "remaining_auto_https_redirects", "protocols": ["h1", "h2", "h3"]}
Caddy proxying http://:443 -> :8080
2022/11/05 13:23:49.548 INFO tls cleaning storage unit {"description": "FileStorage:/home/ubuntu/.local/share/caddy"}
2022/11/05 13:23:49.549 INFO tls finished cleaning storage units
Not sure where to go from here.
Running vaultWarden in docker, so might try to go down that route i.e. install caddy in a docker container
Would appreciate any help.