The reason is that fastcgi server responds with a body and Caddy prioritises the upstream response over writing another response.
Maybe we should make it user configurable for Caddy to override fastcgi upstream response body when the response code is > 400.