Oh, well something like this might work then:
@hasBaseUrl {
query baseUrl=*
}
rewrite @hasBaseUrl /baseUrl/{path}
reverse_proxy /baseUrl/* {http.request.uri.query.baseUrl}
Just a guess.
I think this might fail if the baseUrl has a path on it though. If you were able to split the baseUrl into something like baseDomain and basePath you might get better results.