Caddy failed to run on boot: binding error

1. The problem I’m having:

I’m running Caddy server inside VM (host: Debian 12 + libvirt + qemu + guest: Debian 13). I assigned static ip address to VM in libvirt network conf and made script to forward ports that executes on VM start. Everything works fine if I run caddy when VM is booted already. When VM starts, caddy fails to run before I manually restart it. I want to figure out why is it happening.

I’ve already made changes in Caddyfile that solve the issue, but it’s not enough for me, cause I haven’t faced this issue before (when I played with caddy inside LXC containers some time ago). I want to assign local port to another app, binding caddy only to external address or vice versa.

2. Error messages and/or full log output:

Command: `sudo systemctl status caddy`

Output:

× caddy.service - Caddy
     Loaded: loaded (/usr/lib/systemd/system/caddy.service; enabled; preset: enabled)
     Active: failed (Result: exit-code) since Mon 2025-09-01 00:06:32 MSK; 1min 19s ago
 Invocation: 52d6636b1ff4422cb37cac961d7c35b8
       Docs: https://caddyserver.com/docs/
    Process: 694 ExecStart=/usr/bin/caddy run --environ --config /etc/caddy/Caddyfile (code=exited, status=1/FAILURE)
   Main PID: 694 (code=exited, status=1/FAILURE)
     Status: "loading new config: http app module: start: listening on 192.168.120.20:443: listen tcp 192.168.120.20:443: bind: cannot assign requested address"
   Mem peak: 49.8M
        CPU: 94ms

Sep 01 00:06:32 entry-guest caddy[694]: {"level":"info","ts":1756674392.4919827,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//>
Sep 01 00:06:32 entry-guest caddy[694]: {"level":"info","ts":1756674392.492658,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc00002b280"}
Sep 01 00:06:32 entry-guest caddy[694]: {"level":"info","ts":1756674392.4964783,"logger":"http.auto_https","msg":"automatic HTTPS is completely disabled for server","server_name":"srv0"}
Sep 01 00:06:32 entry-guest caddy[694]: {"level":"info","ts":1756674392.4967372,"logger":"tls.cache.maintenance","msg":"stopped background certificate maintenance","cache":"0xc00002b280"}
Sep 01 00:06:32 entry-guest caddy[694]: {"level":"info","ts":1756674392.4967535,"logger":"http","msg":"servers shutting down with eternal grace period"}
Sep 01 00:06:32 entry-guest caddy[694]: {"level":"info","ts":1756674392.496955,"msg":"maxprocs: No GOMAXPROCS change to reset"}
Sep 01 00:06:32 entry-guest caddy[694]: Error: loading initial config: loading new config: http app module: start: listening on 192.168.120.20:443: listen tcp 192.168.120.20:443: bind: cannot >
Sep 01 00:06:32 entry-guest systemd[1]: caddy.service: Main process exited, code=exited, status=1/FAILURE
Sep 01 00:06:32 entry-guest systemd[1]: caddy.service: Failed with result 'exit-code'.
Sep 01 00:06:32 entry-guest systemd[1]: Failed to start caddy.service - Caddy.

3. Caddy version:

v2.10.2

4. How I installed and ran Caddy:

From Caddyserver official page for Debian users. Using apt package manager. Run as a systemd service.

a. System environment:

OS: Host - Debian 12, Guest - Debian 13

Arch: amd64

virsh version output:

Using library: libvirt 9.0.0 
Using API: QEMU 9.0.0 
Running hypervisor: QEMU 7.2.17

libvirt network xml:

<network connections="2">
  <name>default</name>
  <uuid>HIDDEN</uuid>
  <forward mode="nat">
    <nat>
      <port start="1024" end="65535"/>
    </nat>
  </forward>
  <bridge name="virbr0" stp="on" delay="0"/>
  <mac address="HIDDEN"/>
  <dns enable="no"/>
  <ip address="192.168.120.1" netmask="255.255.255.0">
    <dhcp>
      <range start="192.168.120.2" end="192.168.120.254"/>
      <host mac="HIDDEN" ip="192.168.120.10"/>
      <host mac="HIDDEN" ip="192.168.120.20"/> <!-- caddy -->
    </dhcp>
  </ip>
</network>

b. Command:

# How I run caddy
sudo systemctl start caddy

# How I check caddy's status
sudo systemctl status caddy

c. Service/unit/compose file:

Default systemd unit file, hasn’t been changed.

d. My complete Caddy config:

Config that causes error (response is for testing purposes):

{
	servers 192.168.120.20:443 {
		listener_wrappers {
			proxy_protocol {
				fallback_policy use
			}
			tls
		}
		protocols h2 h1
	}
}

https://example.com:443 {
    bind 192.168.120.20
	tls /etc/caddy/cert.crt /etc/caddy/key.key
	header Content-Type text/html
	respond * 200 {
		body <<HTML
			<html>
			<head><title>Respond</title></head>
			<body>
			<div style="padding-top:80px;font-size:28px;">
			<h2 style="text-align:center;">Remote Host: {remote_host}</h2>
			<h2 style="text-align:center;">Remote Port: {remote_port}</h2>
			<h2 style="text-align:center;">Client IP: {client_ip}</h2>
			</div>
			</body>
			</html>
			HTML
	}
}

Config that doesn’t have issues:

{
	servers :443 {
		listener_wrappers {
			proxy_protocol {
				fallback_policy use
			}
			tls
		}
		protocols h2 h1
	}
}

https://example.com {
	tls /etc/caddy/cert.crt /etc/caddy/key.key
	header Content-Type text/html
	respond * 200 {
		body <<HTML
			<html>
			<head><title>Respond</title></head>
			<body>
			<div style="padding-top:80px;font-size:28px;">
			<h2 style="text-align:center;">Remote Host: {remote_host}</h2>
			<h2 style="text-align:center;">Remote Port: {remote_port}</h2>
			<h2 style="text-align:center;">Client IP: {client_ip}</h2>
			</div>
			</body>
			</html>
			HTML
	}
}

5. Links to relevant resources:

It looks like you want Caddy only to listen on 192.168.120.20, but you don’t have such an interface?

The log line is incomplete. Use this command to get the full lines

journalctl -u caddy --no-pager | less +G

Yes. I researched it myself. It looks like caddy starts before VM gets IP address through DHCP, and lack of address causes error. So, I suppose, to solve this problem, I have to assign IP address inside VM. But it would be more convenient if caddy simply started after VM gets IP from DHCP. And that’s what I want to achieve.

Hardly more logs will give us a useful info. But here it is:

Sep 01 19:51:05 entry-guest systemd[1]: Starting caddy.service - Caddy...
Sep 01 19:51:05 entry-guest caddy[690]: {"level":"info","ts":1756745465.6164706,"msg":"maxprocs: Leaving GOMAXPROCS=2: CPU quota undefined"}
Sep 01 19:51:05 entry-guest caddy[690]: {"level":"info","ts":1756745465.6170301,"msg":"GOMEMLIMIT is updated","package":"github.com/KimMachineGun/automemlimit/memlimit","GOMEMLIMIT":3696607641,"previous":9223372036854775807}
Sep 01 19:51:05 entry-guest caddy[690]: caddy.HomeDir=/var/lib/caddy
Sep 01 19:51:05 entry-guest caddy[690]: caddy.AppDataDir=/var/lib/caddy/.local/share/caddy
Sep 01 19:51:05 entry-guest caddy[690]: caddy.AppConfigDir=/var/lib/caddy/.config/caddy
Sep 01 19:51:05 entry-guest caddy[690]: caddy.ConfigAutosavePath=/var/lib/caddy/.config/caddy/autosave.json
Sep 01 19:51:05 entry-guest caddy[690]: caddy.Version=v2.10.2 h1:g/gTYjGMD0dec+UgMw8SnfmJ3I9+M2TdvoRL/Ovu6U8=
Sep 01 19:51:05 entry-guest caddy[690]: runtime.GOOS=linux
Sep 01 19:51:05 entry-guest caddy[690]: runtime.GOARCH=amd64
Sep 01 19:51:05 entry-guest caddy[690]: runtime.Compiler=gc
Sep 01 19:51:05 entry-guest caddy[690]: runtime.NumCPU=2
Sep 01 19:51:05 entry-guest caddy[690]: runtime.GOMAXPROCS=2
Sep 01 19:51:05 entry-guest caddy[690]: runtime.Version=go1.25.0
Sep 01 19:51:05 entry-guest caddy[690]: os.Getwd=/
Sep 01 19:51:05 entry-guest caddy[690]: LANG=en_US.UTF-8
Sep 01 19:51:05 entry-guest caddy[690]: LANGUAGE=en_US:en
Sep 01 19:51:05 entry-guest caddy[690]: PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin
Sep 01 19:51:05 entry-guest caddy[690]: NOTIFY_SOCKET=/run/systemd/notify
Sep 01 19:51:05 entry-guest caddy[690]: USER=caddy
Sep 01 19:51:05 entry-guest caddy[690]: LOGNAME=caddy
Sep 01 19:51:05 entry-guest caddy[690]: HOME=/var/lib/caddy
Sep 01 19:51:05 entry-guest caddy[690]: INVOCATION_ID=10c763c3cf5b402aba81d0e1ca159030
Sep 01 19:51:05 entry-guest caddy[690]: JOURNAL_STREAM=9:6549
Sep 01 19:51:05 entry-guest caddy[690]: SYSTEMD_EXEC_PID=690
Sep 01 19:51:05 entry-guest caddy[690]: MEMORY_PRESSURE_WATCH=/sys/fs/cgroup/system.slice/caddy.service/memory.pressure
Sep 01 19:51:05 entry-guest caddy[690]: MEMORY_PRESSURE_WRITE=c29tZSAyMDAwMDAgMjAwMDAwMAA=
Sep 01 19:51:05 entry-guest caddy[690]: {"level":"info","ts":1756745465.6176333,"msg":"using config from file","file":"/etc/caddy/Caddyfile"}
Sep 01 19:51:05 entry-guest caddy[690]: {"level":"info","ts":1756745465.6213255,"msg":"adapted config to JSON","adapter":"caddyfile"}
Sep 01 19:51:05 entry-guest caddy[690]: {"level":"info","ts":1756745465.6232865,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//[::1]:2019","//127.0.0.1:2019","//localhost:2019"]}
Sep 01 19:51:05 entry-guest caddy[690]: {"level":"info","ts":1756745465.6246011,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc00053b900"}
Sep 01 19:51:05 entry-guest caddy[690]: {"level":"info","ts":1756745465.6271558,"logger":"http.auto_https","msg":"automatic HTTPS is completely disabled for server","server_name":"srv0"}
Sep 01 19:51:05 entry-guest caddy[690]: {"level":"info","ts":1756745465.6273735,"logger":"tls.cache.maintenance","msg":"stopped background certificate maintenance","cache":"0xc00053b900"}
Sep 01 19:51:05 entry-guest caddy[690]: {"level":"info","ts":1756745465.627389,"logger":"http","msg":"servers shutting down with eternal grace period"}
Sep 01 19:51:05 entry-guest caddy[690]: {"level":"info","ts":1756745465.6282666,"msg":"maxprocs: No GOMAXPROCS change to reset"}
Sep 01 19:51:05 entry-guest caddy[690]: Error: loading initial config: loading new config: http app module: start: listening on 192.168.120.20:443: listen tcp 192.168.120.20:443: bind: cannot assign requested address
Sep 01 19:51:05 entry-guest systemd[1]: caddy.service: Main process exited, code=exited, status=1/FAILURE
Sep 01 19:51:05 entry-guest systemd[1]: caddy.service: Failed with result 'exit-code'.
Sep 01 19:51:05 entry-guest systemd[1]: Failed to start caddy.service - Caddy. 

I’ve found a root of the problem. And it’s both - Debian and Caddy. Gonna complement a bit later.

This is important and useful info. As @hmoffatt said, your machine probably doesn’t have that address assigned to it. It can’t be Caddy issue.

1 Like

Caddy provides official instructions for installation on Debian that fails to work with simple configuration. Not blaming Caddy team, although provide systemd service that relies on systemd-networkd or NetworkManager those doesn’t come by default on clear headless installation and not telling or doing anything about it is not great either.

So, there are two workarounds, might be more, but these are the most recommended.

Set up systemd-networkd

By default, Debian (trixie at least) after headless installation on VM uses networking.service (ifupdown, whatever that is) and because of that, network-online.target that caddy systemd service requires before it starts doesn’t stop caddy from runnig before interface get ip address via DHCP, and that’s why binding operation fails so fails caddy. Enabling systemd-networkd that also enables systemd-networkd-wait-online.service solves the issue because it reliably prevents caddy from running until ip address will be assigned.

Check your interfaces:

ip addr

Find your interface name (for example, eth0 or enp1s0).

Create network config with .network extension:

sudo nano /etc/systemd/network/10-enp1s0.network

Write next lines:

[Match]
Name=enp1s0

[Network]
DHCP=yes

Enable systemd-networkd service:

sudo systemctl enable systemd-networkd

chatgpt suggests to restart service, although I think adding `—now` option is enough, but still:

sudo systemctl restart systemd-networkd

And comment out settings related to interface in /etc/network/interface file (I think it’s better not to touch loopback related settings).

Additionally, I installed systemd-resolved because I disabled DNS server in libvirt network config and put additional line to /etc/systemd/network/10-enp1s0.network with my local dns server address:

[Match]
Name=enp1s0

[Network]
DHCP=yes
DNS=192.168.0.1 # this line, your's could be different

Reboot and see it works. At least, for me. For now…

Do not bind to specific ip

Previously I provided config where I simply listen port on all interfaces, on all addresses. It might be enough and is the simplest solution. But, if you want caddy to listen only on external ip address provided via DHCP (for whatever weird reason), well it will fail on system boot, without proper settings.


Anyone can complement my solution with their thoughts and workarounds. I think Caddy’s team can improve some configs to prevent this behavior. Or maybe it’s time for Debian to finally move to systemd stack completely, idk. This waste of time was completely unnecessary.

I don’t think that’s a fair assessment. The provided systemd unit ensures that Caddy doesn’t start until the network is online:

After=network.target network-online.target
Requires=network-online.target

Lots of packages provide units with the same clauses.

If your network configuration fails to properly implement those targets (smbd, nfs-kernel-server, nginx, etc etc), then that is a problem with that network configuration, not caddy. Lots of other services will also be starting before the network is ready.

2 Likes

How are you configuring the network in the guest?

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.