The PHP preset should handle that - only PHP scripts should be sent when the preset is added. Here’s what php does:
ext .php # specifies extensions which will get proxied
split .php # tells Caddy that anything after this goes in PATH_INFO
index index.php # specifies a default file to try for an index request
So you can see with it enabled (or these settings manually specified - php is just shorthand), no non-PHP file should ever be sent via FastCGI.
Can you double check it’s present in the Caddyfile, restart Caddy, and let me know if the CSS and JS requests still get sent to PHP?