I think this is an issue of Caddy v2 which is easily reproducable.
Have a Caddyfile in caddyfile-style and put one FQDN with certificate and key path, and one without specific paths for cert files in (so Automatic Lets Encrypt generates one for you).
You’ll see, that even in latest master, Caddy will fail to serve the FQDN that has specified cert and key paths.