How do I set up multiple ports on localhost to a single backend?

You need to use the rewrite directive to change the request path. You can’t pass the request path directly to reverse_proxy, it only takes upstream addresses.

But keep in mind:

You’re probably better off using subdomains for each, like foo.localhost and bar.localhost.