That’s what snippets are for:
This won’t work, this will rewrite the URL to literally /core/*. You probably want rewrite * /core{uri}
You can use handle instead of handle_path + rewrite here, it’s redundant to strip /core from the URL and then re-add it.
If each route may change the upstream address, you could set the upstream address with vars (Caddyfile directive) — Caddy Documentation and then only have one reverse_proxy at the end which reads the address with {vars.upstream} or whatever.
Make sure to configure reverse_proxy > transport http > tls though, because https:// scheme can’t be used when using a placeholder for the address (see the last paragraph in this section of the docs: reverse_proxy (Caddyfile directive) — Caddy Documentation)