You could run sudo apt purge caddy which would remove it, I think.
But anyway emptying /var/lib/caddy and restarting Caddy would’ve been enough.
AFAIK, this happens if the ACME account files in storage are corrupted in some way. But since you already blew away the contents, it’s impossible to say in what way they were corrupted.
I think this should be smoothed over in the next release, we think fix: use a tempfile to write files in filestorage. by elee1766 · Pull Request #300 · caddyserver/certmagic · GitHub should resolve some issues with storage writing which might cause some corruption from time to time.