How do I tell Caddy which sub domain to use?

I believe if you have multiple subdomains in your Caddyfile, Caddy will try to get a certificate for each one. For each server, you’ll need to remove the subdomains it can’t serve from the Caddyfile (or otherwise disable automatic HTTPS for these subdomains).

Alternately, using DNS validation you could bypass the requirement for the domain to be pointed at the actual server in question and have each server get its own set of certificates for the same set of subdomains. I don’t otherwise see how you could use Caddy, with automatic HTTPS enabled, to successfully request the same set of certificates on different servers with the same Caddyfile.