Custom Header not sent to client when using 'header' directive with 'defer' in conjunction with 'handle_errors'

[borjapazr (Borja Paz Rodríguez)]

1. The problem I’m having:

Hello!

When I use the header directive with defer to set a custom header in conjunction with the handle_errors directive, the custom header is not sent to client when an error occurs. However, if I remove the defer keyword, the header is sent correctly. I show the examples below:

With defer:

*.{$DOMAINS__001}, {$DOMAINS__001} {
	header {
		Example "example"
		defer
	}

	import docker-service example.{$DOMAINS__001} example 80 # Create a reverse proxy to a Docker service that is down to generate a 502 error.

	handle_errors {
		handle {
			respond <<JSON
			{
				"status_code": {err.status_code},
				"status_text": "{err.status_text}",
				"message": "😵‍💫 {err.status_code} - {err.status_text}"
			}
			JSON
		}
	}
}

~ ❯ http --headers https://example.mydomain.dev
HTTP/1.1 502 Bad Gateway
Alt-Svc: h3=":443"; ma=2592000
Content-Length: 99
Content-Type: application/json
Date: Mon, 09 Sep 2024 16:50:20 GMT
Server: Caddy

Without defer:

*.{$DOMAINS__001}, {$DOMAINS__001} {
	header {
		Example "example"
	}

	import docker-service example.{$DOMAINS__001} example 80 # Create a reverse proxy to a Docker service that is down to generate a 502 error.

	handle_errors {
		handle {
			respond <<JSON
			{
				"status_code": {err.status_code},
				"status_text": "{err.status_text}",
				"message": "😵‍💫 {err.status_code} - {err.status_text}"
			}
			JSON
		}
	}
}

~ ❯ http --headers https://example.mydomain.dev
HTTP/1.1 502 Bad Gateway
Alt-Svc: h3=":443"; ma=2592000
Content-Length: 99
Content-Type: application/json
Date: Mon, 09 Sep 2024 16:51:28 GMT
Example: example
Server: Caddy

Is this the expected behavior or is it a bug? In my opinion the header called Example should be set and send to client both with and without defer.

Thanks in advance!

2. Error messages and/or full log output:

-

3. Caddy version:

v2.8.4 h1:q3pe0wpBj1OcHFZ3n/1nl4V4bxBrYoSoab7rL9BMYNk=

4. How I installed and ran Caddy:

Using Docker and Docker Compose.

a. Service/unit/compose file:

services:
  caddy:
    build:
      context: .
      dockerfile: Dockerfile
    image: example/caddy
    container_name: caddy
    hostname: caddy
    restart: unless-stopped
    environment:
      - TLS__EMAIL=${TLS__EMAIL}
      - TLS__CLOUDFLARE_API_TOKEN=${TLS__CLOUDFLARE_API_TOKEN}
      - DOMAINS__001=${DOMAINS__001}
      - TZ=${GLOBAL__TIMEZONE}
    volumes:
      - ./Caddyfile:/etc/caddy/Caddyfile
      - ./site:/srv
      - ${GLOBAL__VOLUME_DIR}/data:/data
      - ${GLOBAL__VOLUME_DIR}/config:/config
      - ${GLOBAL__VOLUME_DIR}/logs:/logs
    networks:
      - default
      - reverse-proxy
    ports:
      - 80:80
      - 443:443
      - 443:443/udp

networks:
  default:
    name: caddy
  reverse-proxy:
    name: reverse-proxy
    external: true

b. My complete Caddy config:

(docker-service) {
	@{args[1]} host {args[0]}

	handle @{args[1]} {
		reverse_proxy {args[1]}:{args[2]}
	}
}

# Sites
## Configuration for each site proxied by Caddy.
*.{$DOMAINS__001}, {$DOMAINS__001} {
	header {
		Example "example"
		#defer
	}

	import docker-service example.{$DOMAINS__001} example 80

	handle_errors {
		handle {
			respond <<JSON
			{
				"status_code": {err.status_code},
				"status_text": "{err.status_text}",
				"message": "😵‍💫 {err.status_code} - {err.status_text}"
			}
			JSON
		}
	}
}

5. Links to relevant resources:

-

[Whitestrake (Matthew Fay)]

Howdy @borjapazr!

@borjapazr

Is this the expected behavior or is it a bug?

It is indeed expected behaviour.

Deferred headers essentially wait for a request to go all the way down the processing chain to a terminating handler (i.e. something that writes a response) and slaps the headers on as that response comes back up the middleware chain on its way to be returned to the client.

Errors cut that chain short, forcing us to switch to error handling immediately. Since deferred headers are among the last operations to take place during the request processing, they won’t get a chance to execute in the event of an error.

If you need to ensure the presence of a specific header in your error handling, you’ll need to specify it inside your handle_errors block as well.