Yeah - because the .dev TLD is in the HSTS preload list. I recommend using .localhost instead, which should resolve to 127.0.0.1 without any changes to the hosts file on most Linux systems.
Caddy can manage local TLS for you using its internal CA. Use tls internal to force Caddy to do this. If you use a .localhost TLD, this is automatic (because Caddy knows that that TLD cannot be public).
This should be automatic, since Caddy attempts to install the root CA certificate to your system’s trust store, if it can.