Caddy supports both Let’s Encrypt and ZeroSSL, and will issue certificates as fast as it can. Let’s Encrypt rate limits would probably slow you down from doing this, but ZeroSSL has no rate limits. So Caddy should be able to get them all issued pretty quickly.
Is using the DNS challenge an option for that domain? If so then that would just be a single certificate to replace that one by getting a wildcard cert from LE/ZeroSSL.
I think what’s going on is that this {} at the end acts as a catch-all which ends up using one of the loaded certificates, i.e. your wildcard.
I think instead you’d have to configure the tls app to automate the certificates with the names you want (see Using Caddy to keep certificates renewed for some explanation of how to configure that) and add a new connection policy to match those domains, without requiring a tag (I think).