What is a good way to verify Caddy's timeout functionality?

Help
1

1. The problem I’m having:

Hi! I’ve been writing some E2E tests in order to verify that Caddy does what we tell it to do via config, and I’ve hit a snag: When the backend spends longer than servers.timeouts.write to generate its response, I expect Caddy to return an HTTP 502, or just close the connection. Instead, it seems to keep the connection open until the test client times out. The debug log does not indicate that the server actively decided to do anything once the timeout was reached.

What I need to know then is:

  1. Is this behavior expected?
  2. Does Caddy free itself from the resources associated with the request and response immediately after the configured timeout, or does it hang on to the connection and request / response data until the client closes its connection?
  3. Other than using timing, are there any reliable ways to test and verify this functionality in an E2E context?

2. Error messages and/or full log output:

2025/01/09 09:15:34.988 DEBUG   http.handlers.reverse_proxy     selected upstream       {"dial": "localhost:9003", "total_upstreams": 1}
2025/01/09 09:15:55.013 DEBUG   http.handlers.reverse_proxy     upstream roundtrip      {"upstream": "localhost:9003", "duration": 20.021060094, "request": {"remote_ip": "127.0.0.1", "remote_port": "33196", "client_ip": "127.0.0.1", "proto": "HTTP/1.1", "method": "GET", "host": "localhost:8080", "uri": "/app/e2e-test-api/api/v1/slow", "headers": {"X-Forwarded-Copy": [""], "X-Forwarded-For": ["127.0.0.1"], "X-Forwarded-Proto": ["http"], "X-Forwarded-Host": ["localhost:8080"], "User-Agent": ["Go-http-client/1.1"], "Accept-Encoding": ["gzip"]}}, "headers": {"Date": ["Thu, 09 Jan 2025 09:15:54 GMT"]}, "status": 204}
2025/01/09 09:15:55.036 DEBUG   http.handlers.reverse_proxy     selected upstream       {"dial": "localhost:9003", "total_upstreams": 1}

3. Caddy version:

v2.9.0 h1:rteY8N18LsQn+2KVk6R10Vg/AlNsID1N/Ek9JLjm2yE=

4. How I installed and ran Caddy:

I execute the Caddy binary from the e2e framework.
In bash, the command would look something like:
caddy run --adapter caddyfile --config {{path-to-caddyfile}}

a. System environment:

Many flavors of Linux.

b. Command:

caddy run --adapter caddyfile --config {{path-to-caddyfile}}

d. My complete Caddy config:

{
	admin off
	debug
	auto_https off
	http_port 8080
	https_port 8080

	servers {
		timeouts {
			read_body 2s
			read_header 2s
			write 10s
			idle 10s
		}
	}
}

:8080 {
	# Handle requests to e2e-test-api
	handle /app/e2e-test-api* {
		reverse_proxy http://localhost:9003 {
			# Send the original X-Forwarded-For header to the backend as X-Forwarded-Copy
			header_up X-Forwarded-Copy {header.X-Forwarded-For}
		}
	}
}

5. Links to relevant resources:

2

Figured it out:
The global timeout settings are applied to tcp connections, not requests, and aren’t useful in this context.

In order to get the desired behavior, one needs to set the transport.response_header_timeout setting on a per-proxy basis. This will cause caddy to respond with a HTTP 504 if the response takes too long.

There doesn’t seem to be any way to globally alter the default value for this setting.

1 Like