So the way on-demand TLS works, is a TLS handshake for a domain (specifically: the ClientHello contains the domain in its ServerName field) comes in will trigger an obtain/renewal operation.
If Caddy is still trying to renew it then it’s because it’s still getting TLS handshakes for that domain.
Typically, that doesn’t happen because once the DNS goes away then clients stop resolving it to your server, because the DNS entry changed. Sometimes that can take a couple days depending on propagation.
Are you sure you adjusted your ‘ask’ endpoint properly? If it was returning non-200, then Caddy will emit a log saying so, and it won’t try to get a cert from the CA.