HTTP/3 alt-svc log spam on reload

Help
1

1. The problem I’m having:

Reloading caddy configuration causes strange logs, reloading it several times increases the frequency of these logs until the next full restart.

2. Error messages and/or full log output:

{"level":"error","ts":1716331446.7550735,"logger":"http.log","msg":"setting HTTP/3 Alt-Svc header","error":"no port can be announced, specify it explicitly using Server.Port or Server.Addr"}

3. Caddy version:

v2.7.6

4. How I installed and ran Caddy:

Arch Linux package: caddy 2.7.6-1

a. System environment:

Arch Linux 6.7.6-arch1-1, systemd 255.6-1-arch

b. Command:

PASTE OVER THIS, BETWEEN THE ``` LINES.
Please use the preview pane to ensure it looks nice.

c. Service/unit/compose file:

# /usr/lib/systemd/system/caddy.service
# caddy.service
#
# For using Caddy with a config file.
#
# Make sure the ExecStart and ExecReload commands are correct
# for your installation.
#
# See https://caddyserver.com/docs/install for instructions.
#
# WARNING: This service does not use the --resume flag, so if you
# use the API to make changes, they will be overwritten by the
# Caddyfile next time the service is restarted. If you intend to
# use Caddy's API to configure it, add the --resume flag to the
# `caddy run` command or use the caddy-api.service file instead.

[Unit]
Description=Caddy web server
Documentation=https://caddyserver.com/docs/
After=network-online.target
Wants=network-online.target systemd-networkd-wait-online.service
StartLimitIntervalSec=14400
StartLimitBurst=10

[Service]
Type=notify
User=caddy
Group=caddy
Environment=XDG_DATA_HOME=/var/lib
Environment=XDG_CONFIG_HOME=/etc
ExecStartPre=/usr/bin/caddy validate --config /etc/caddy/Caddyfile
ExecStart=/usr/bin/caddy run --environ --config /etc/caddy/Caddyfile
ExecReload=/usr/bin/caddy reload --config /etc/caddy/Caddyfile --force
ExecStopPost=/usr/bin/rm -f /run/caddy/admin.socket

# Do not allow the process to be restarted in a tight loop. If the
# process fails to start, something critical needs to be fixed.
Restart=on-abnormal

# Use graceful shutdown with a reasonable timeout
TimeoutStopSec=5s

LimitNOFILE=1048576
LimitNPROC=512

# Hardening options
AmbientCapabilities=CAP_NET_BIND_SERVICE
CapabilityBoundingSet=CAP_NET_BIND_SERVICE
DevicePolicy=closed
LockPersonality=true
MemoryAccounting=true
MemoryDenyWriteExecute=true
NoNewPrivileges=true
PrivateDevices=true
PrivateTmp=true
ProcSubset=pid
ProtectClock=true
ProtectControlGroups=true
ProtectHome=true
ProtectHostname=true
ProtectKernelLogs=true
ProtectKernelModules=true
ProtectKernelTunables=true
ProtectProc=invisible
ProtectSystem=strict
RemoveIPC=true
ReadWritePaths=/var/lib/caddy /var/log/caddy /run/caddy
RestrictNamespaces=true
RestrictRealtime=true
RestrictSUIDSGID=true

[Install]
WantedBy=multi-user.target

# /etc/systemd/system/caddy.service.d/override.conf
[Service]
ExecReload=/usr/bin/bash -c '/usr/bin/caddy validate -c /etc/caddy/Caddyfile > /dev/null && /usr/bin/caddy reload --config /etc/caddy/Caddyfile --force'

d. My complete Caddy config:

Entire config is ~1.2k lines, I’ll isolate this tomorrow & figure out a minimal example config that can reproduce the issue.

5. Links to relevant resources:

Relevant things I found that may or may not be related:
Setting HTTP/3 Alt-Svc header","error":"no port can be announced, specify it explicitly using Server.Port or Server.Addr (closed)
No port can be announced (closed)
Caddy 2.6.x - handle_path behavior change in relation to php_fastcgi · Issue #5073 · caddyserver/caddy · GitHub (closed)

2

Please share your Caddyfile. Can’t help without seeing what you actually have in your config.

That error typically means you’re listening on a unix socket or something, therefore Go’s stdlib can’t automatically determine a port to use for the Alt-Svc header to announce that HTTP/3 support is available to clients. It’s not really a problem, it just means that the Alt-Svc header won’t be used so clients won’t try HTTP/3.

2 Likes
3

Hmm, I’ve been trying to isolate this and even if I copy the entire 1.2k lines of caddy config into an arch VM running the same binary, I can’t reproduce it, even when I connect to it with a http2 and http3 client. My gut feeling says this is some weird interaction between me having multiple server blocks that bind to different combinations of addresses (partially overlapping), possibly my manual tls config, catch-all/fallback http & https blocks, and http2 and http3 clients connecting to one or more of them. I’ll keep trying, and will post a config that can actually reproduce the issue here when I’ve found the cause.

1 Like
4

Aight, while I haven’t managed to reproduce it in a VM, I have verified that the problem still occurs when I have no sites enabled, so here’s the full config:

{
	default_bind [2a01:4f8:10a:23a7::1]
	auto_https off

	log {
		level WARN
		output stderr
	}
}

&(hsts) {
	header >Strict-Transport-Security "max-age=63072000; includeSubDomains; preload"
}

&(frame) {
	header ?X-Frame-Options SAMEORIGIN
}

&(xss) {
	header >X-XSS-Protection 0
}

&(nosniff) {
	header >X-Content-Type-Options nosniff
}

&(referer) {
	header >Referrer-Policy same-origin
}

(security) {
	invoke hsts
	invoke frame
	invoke xss
	invoke nosniff
	invoke referer
}

&(errors-template) {
	respond <<EOF
	<html>
	<head><title>{{placeholder "http.error.status_code"}} {{placeholder "http.error.status_text"}}</title></head>
	<body>
	<center><h1>{{placeholder "http.error.status_code"}} {{placeholder "http.error.status_text"}}</h1></center>
	<hr><center>Caddy</center>
	</body>
	</html>
	EOF
}

(errors) {
	handle_errors {
		import security
		templates
		header Content-Type text/html
		invoke errors-template
	}
}

(certs) {
	tls /etc/dehydrated/certs/{args[0]}/fullchain.pem /etc/dehydrated/certs/{args[0]}/privkey.pem
}

caddy.localhost {
	import certs zotan.pw
	import certs ztn.sh
}

https:// {
	bind [2a01:4f8:10a:23a7::1] [2a01:4f8:10a:23a7::666] 88.99.139.92
	import errors
	error 421
}

http:// {
	bind 88.99.139.92 88.99.139.96 [2a01:4f8:10a:23a7::1] [2a01:4f8:10a:23a7::666]
	import errors

	handle_path /.well-known/acme-challenge/* {
		root * /var/www/dehydrated
		file_server
	}

	redir https://{host}{uri} 308
}

With this as /etc/caddy/Caddyfile I just have to type sudo systemctl restart caddy and then sudo systemctl reload caddy (a couple times) and it’ll start spamming the log message.

1 Like
5

You mean this type of log entry will keep showing up even if the config is not changed for a while, not around when the config is reloaded (During reloading it’s possible and totally normal to encounter this log, and depending on requests, it’s possible to have many entries). It’s a bug if it keeps showing up after reload is complete.

1 Like
6

I initially thought it was showing up even if the config hadn’t been changed in a while, but this seems to not be the case. Still, it seems odd to me to log something that’s expected around reloads as an error message.

7

I do get the same flurry of error messages every once in a while on one of my servers, but only one. If I restart caddy, the problem disappears for a while.

8

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.