No, the file exists in the correct location when I install with the new package. Old package: https://aur.archlinux.org/packages/caddy/ New package: https://aur.archlinux.org/packages/caddy-full-bin/
It is quite fast for me to switch the packages, so keep the tips coming 
Maybe I should ping klingt as well.