4xx responses indicate client errors. The server received and responded to the request as expected (although with 404, for example, it obviously did not find what the client requested, so it appears the client made a mistaken request).
5xx responses indicate server errors and the actual generated errors should be logged in the error logs.
edit: I’m incorrect and >=400 should be logged as indicated (according to the errors docs).
Does /log/site-error.log exist and is it writable by Caddy?