Might be wrong but I don’t think you can without disabling HTTP/3 entirely using Global options (Caddyfile) — Caddy Documentation
Though, the “0-RTT security concern/risk/vulnerability” stuff has been brought up pretty frequently with Caddy, and it all really roots down to concerns of replay attacks, but that’s something you should be fixing at the application level instead of just disabling 0-RTT at the web server level. Browsers have already been “replaying” POST requests without changing the contents for a long time now with the “resend form on reload” stuff.
https://caddy.community/t/is-it-safe-to-use-http-3-0-rtt/20923/2
https://github.com/caddyserver/caddy/issues/5754#issuecomment-1680695803