1. Caddy version (caddy version):
v2.4.6 h1:HGkGICFGvyrodcqOOclHKfvJC0qTU7vny/7FhYp9hNw=
2. How I run Caddy:
a. System environment:
Ubuntu 20.04
Systemd
xcaddy installed from CloudSmith repo
caddy built with xcaddy build --with github.com/caddy-dns/cloudflare --output /usr/bin/caddy.custom
Followed the instructions here to apt divert and use the systemd services provided by the package.
The above was automated with an ansible playbook
b. Command:
I used systemd...
c. Service/unit/compose file:
# /lib/systemd/system/caddy.service
# caddy.service
#
# For using Caddy with a config file.
#
# Make sure the ExecStart and ExecReload commands are correct
# for your installation.
#
# See https://caddyserver.com/docs/install for instructions.
#
# WARNING: This service does not use the --resume flag, so if you
# use the API to make changes, they will be overwritten by the
# Caddyfile next time the service is restarted. If you intend to
# use Caddy's API to configure it, add the --resume flag to the
# `caddy run` command or use the caddy-api.service file instead.
[Unit]
Description=Caddy
Documentation=https://caddyserver.com/docs/
After=network.target network-online.target
Requires=network-online.target
[Service]
Type=notify
User=caddy
Group=caddy
ExecStart=/usr/bin/caddy run --environ --config /etc/caddy/Caddyfile
ExecReload=/usr/bin/caddy reload --config /etc/caddy/Caddyfile
TimeoutStopSec=5s
d. My complete Caddyfile or JSON config:
# BEGIN ANSIBLE MANAGED BLOCK for Cloudflare TLS
(cf_tls) {
tls <cloudflare email> {
dns cloudflare <cloudflare token>
resolvers 8.8.8.8
}
}
# END ANSIBLE MANAGED BLOCK for Cloudflare TLS
# The Caddyfile is an easy way to configure your Caddy web server.
#
# Unless the file starts with a global options block, the first
# uncommented line is always the address of your site.
#
# To use your own domain name (with automatic HTTPS), first make
# sure your domain's A/AAAA DNS records are properly pointed to
# this machine's public IP, then replace ":80" below with your
# domain name.
:80 {
# Set this path to your site's directory.
root * /usr/share/caddy
# Enable the static file server.
file_server
# Another common task is to set up a reverse proxy:
# reverse_proxy localhost:8080
# Or serve a PHP site through php-fpm:
# php_fastcgi localhost:9000
}
# Refer to the Caddy docs for more information:
# https://caddyserver.com/docs/caddyfile
# BEGIN ANSIBLE MANAGED BLOCK for static
static.pitwinken.com {
root * /srv
file_server browse
import cf_tls
}
# END ANSIBLE MANAGED BLOCK for static
# BEGIN ANSIBLE MANAGED BLOCK for filestash
filestash.pitwinken.com {
reverse_proxy docker.lan:8334
import cf_tls
}
# END ANSIBLE MANAGED BLOCK for filestash
# BEGIN ANSIBLE MANAGED BLOCK for hass
hass.pitwinken.com {
reverse_proxy docker.lan:8123
import cf_tls
}
# END ANSIBLE MANAGED BLOCK for hass
# BEGIN ANSIBLE MANAGED BLOCK for nodered
nodered.pitwinken.com {
reverse_proxy docker.lan:1880
import cf_tls
}
# END ANSIBLE MANAGED BLOCK for nodered
# BEGIN ANSIBLE MANAGED BLOCK for speedtest
speedtest.pitwinken.com {
reverse_proxy docker.lan:8080
import cf_tls
}
# END ANSIBLE MANAGED BLOCK for speedtest
# BEGIN ANSIBLE MANAGED BLOCK for portainer
portainer.pitwinken.com {
reverse_proxy docker.lan:9000
import cf_tls
}
# END ANSIBLE MANAGED BLOCK for portainer
# BEGIN ANSIBLE MANAGED BLOCK for syncthing
syncthing.pitwinken.com {
reverse_proxy docker.lan:8384
import cf_tls
}
# END ANSIBLE MANAGED BLOCK for syncthing
# BEGIN ANSIBLE MANAGED BLOCK for uptime
uptime.pitwinken.com {
reverse_proxy docker.lan:3001
import cf_tls
}
# END ANSIBLE MANAGED BLOCK for uptime
# BEGIN ANSIBLE MANAGED BLOCK for pve1
pve1.pitwinken.com {
reverse_proxy pve1.lan:8006 {
transport http {
tls_insecure_skip_verify
}
}
import cf_tls
}
# END ANSIBLE MANAGED BLOCK for pve1
# BEGIN ANSIBLE MANAGED BLOCK for webdav
webdav.pitwinken.com {
reverse_proxy tkfs.lan:443 {
transport http {
tls_insecure_skip_verify
}
}
import cf_tls
}
# END ANSIBLE MANAGED BLOCK for webdav
# BEGIN ANSIBLE MANAGED BLOCK for transmission
transmission.pitwinken.com {
reverse_proxy transmission.lan:9091
import cf_tls
}
# END ANSIBLE MANAGED BLOCK for transmission
# BEGIN ANSIBLE MANAGED BLOCK for netdata
netdata.pitwinken.com {
reverse_proxy pve1.lan:19999
import cf_tls
}
# END ANSIBLE MANAGED BLOCK for netdata
# BEGIN ANSIBLE MANAGED BLOCK for homer
pitwinken.com, www.pitwinken.com {
reverse_proxy docker.lan:80 {
header_down -Cookie
}
import cf_tls
}
# END ANSIBLE MANAGED BLOCK for homer
# BEGIN ANSIBLE MANAGED BLOCK for semaphore
semaphore.pitwinken.com {
reverse_proxy docker.lan:3000
import cf_tls
}
# END ANSIBLE MANAGED BLOCK for semaphore
Note that the it is a split-DNS domain (i.e. my home router DNS is configured to return the local IP address with *.pitwinken.com, but the public DNS record points to a domain parking page)
3. The problem I’m having:
When I access any of the reverse-proxied sites, Chrome returns the following error:
Error code to allow searching later: ERR_SSL_VERSION_OR_CIPHER_MISMATCH
4. Error messages and/or full log output:
Feb 26 18:01:04 caddy systemd[1]: Starting Caddy...
Feb 26 18:01:05 caddy caddy[123]: caddy.HomeDir=/var/lib/caddy
Feb 26 18:01:05 caddy caddy[123]: caddy.AppDataDir=/var/lib/caddy/.local/share/caddy
Feb 26 18:01:05 caddy caddy[123]: caddy.AppConfigDir=/var/lib/caddy/.config/caddy
Feb 26 18:01:05 caddy caddy[123]: caddy.ConfigAutosavePath=/var/lib/caddy/.config/caddy/autosave.json
Feb 26 18:01:05 caddy caddy[123]: caddy.Version=v2.4.6 h1:HGkGICFGvyrodcqOOclHKfvJC0qTU7vny/7FhYp9hNw=
Feb 26 18:01:05 caddy caddy[123]: runtime.GOOS=linux
Feb 26 18:01:05 caddy caddy[123]: runtime.GOARCH=amd64
Feb 26 18:01:05 caddy caddy[123]: runtime.Compiler=gc
Feb 26 18:01:05 caddy caddy[123]: runtime.NumCPU=1
Feb 26 18:01:05 caddy caddy[123]: runtime.GOMAXPROCS=1
Feb 26 18:01:05 caddy caddy[123]: runtime.Version=go1.17.5
Feb 26 18:01:05 caddy caddy[123]: os.Getwd=/
Feb 26 18:01:05 caddy caddy[123]: LANG=C
Feb 26 18:01:05 caddy caddy[123]: PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
Feb 26 18:01:05 caddy caddy[123]: NOTIFY_SOCKET=/run/systemd/notify
Feb 26 18:01:05 caddy caddy[123]: HOME=/var/lib/caddy
Feb 26 18:01:05 caddy caddy[123]: LOGNAME=caddy
Feb 26 18:01:05 caddy caddy[123]: USER=caddy
Feb 26 18:01:05 caddy caddy[123]: INVOCATION_ID=d54ebc0f4d9c468da5015cf50f13bddd
Feb 26 18:01:05 caddy caddy[123]: JOURNAL_STREAM=8:48140765
... unrelated logs ... (Sorry, this part contains unrelated connection refused errors happening yesterday which is not related to this problem. There are many websocket keys which takes a very long time to redact)
Feb 28 05:01:05 caddy caddy[123]: {"level":"info","ts":1645995665.3835557,"logger":"tls.cache.maintenance","msg":"advancing OCSP staple","identifiers":["www.pitwinken.com"],"from":1646297998,"to":1646513998}
Feb 28 05:01:05 caddy caddy[123]: {"level":"info","ts":1645995665.6229663,"logger":"tls.cache.maintenance","msg":"advancing OCSP staple","identifiers":["transmission.pitwinken.com"],"from":1646297998,"to":1646513998}
Feb 28 05:01:05 caddy caddy[123]: {"level":"info","ts":1645995665.8541203,"logger":"tls.cache.maintenance","msg":"advancing OCSP staple","identifiers":["uptime.pitwinken.com"],"from":1646297998,"to":1646513998}
Feb 28 05:01:06 caddy caddy[123]: {"level":"info","ts":1645995666.087536,"logger":"tls.cache.maintenance","msg":"advancing OCSP staple","identifiers":["pve1.pitwinken.com"],"from":1646297998,"to":1646513998}
Feb 28 05:01:06 caddy caddy[123]: {"level":"info","ts":1645995666.3223412,"logger":"tls.cache.maintenance","msg":"advancing OCSP staple","identifiers":["speedtest.pitwinken.com"],"from":1646297998,"to":1646513998}
Feb 28 05:01:06 caddy caddy[123]: {"level":"info","ts":1645995666.556865,"logger":"tls.cache.maintenance","msg":"advancing OCSP staple","identifiers":["portainer.pitwinken.com"],"from":1646297998,"to":1646513998}
Feb 28 05:01:06 caddy caddy[123]: {"level":"info","ts":1645995666.7912898,"logger":"tls.cache.maintenance","msg":"advancing OCSP staple","identifiers":["webdav.pitwinken.com"],"from":1646297998,"to":1646513998}
Feb 28 05:01:07 caddy caddy[123]: {"level":"info","ts":1645995667.021122,"logger":"tls.cache.maintenance","msg":"advancing OCSP staple","identifiers":["pitwinken.com"],"from":1646297998,"to":1646513998}
Feb 28 05:01:07 caddy caddy[123]: {"level":"info","ts":1645995667.252352,"logger":"tls.cache.maintenance","msg":"advancing OCSP staple","identifiers":["nodered.pitwinken.com"],"from":1646297998,"to":1646513998}
Feb 28 05:01:07 caddy caddy[123]: {"level":"info","ts":1645995667.4877195,"logger":"tls.cache.maintenance","msg":"advancing OCSP staple","identifiers":["hass.pitwinken.com"],"from":1646297998,"to":1646513998}
Feb 28 05:01:07 caddy caddy[123]: {"level":"info","ts":1645995667.719005,"logger":"tls.cache.maintenance","msg":"advancing OCSP staple","identifiers":["filestash.pitwinken.com"],"from":1646297998,"to":1646513998}
Feb 28 05:01:07 caddy caddy[123]: {"level":"info","ts":1645995667.9508748,"logger":"tls.cache.maintenance","msg":"advancing OCSP staple","identifiers":["syncthing.pitwinken.com"],"from":1646297998,"to":1646513998}
Feb 28 18:01:05 caddy caddy[123]: {"level":"info","ts":1646042465.0584407,"logger":"tls","msg":"cleaning storage unit","description":"FileStorage:/var/lib/caddy/.local/share/caddy"}
Feb 28 18:01:05 caddy caddy[123]: {"level":"info","ts":1646042465.0779898,"logger":"tls","msg":"finished cleaning storage units"}
No logs appear when I access the site with Chrome with the above errors
5. What I already tried:
Chrome 98.0.4758.102 doesn’t work. But it does work on Firefox
I found this StackOverflow answer which seems to be related. One answer suggested to disable QUIC in chrome flags which I did. After disabling QUIC, the sites work fine and can be accessed normally.
The sites should work around one month before (not sure the exact time as I don’t use Chrome frequently). It might be related to a specific version of Chrome (v 98) but I’m not sure
From what I learnt in the StackOverflow answer, disabling QUIC is a security workaround for TLS 1.0/1.1. However, AFAIK caddy 2.0 doesn’t use TLS 1.0/1.1, so not sure what is the proper way to solve this. Any ideas?
Update: I tried using same version of Chrome on Linux and on Android and the sites work. So probably it is related to my particular Windows machine. It is getting late now and will do more troubleshooting later.