Caddy can get certificates for my duckdns ip address if I run it from cli, but can't, when I lauch it as a systemd service

It could indeed be the issue.

I am not very familiar with Debian, but you do have ProtectSystem=full in your service file.

Take a look at this, it was kind of a similar problem.

You may need to adjust ReadWritePaths accordingly and add the folders Caddy uses to store its data and logs.

Refer to the following:

or check your Caddy debug log to see where it is trying to store data.

Also, make sure caddy has read/write access to those folders.