To use the replace-response plugin, you need to make a custom build Caddy with the plugin added. See the docs on Docker Hub for how to do this.
But that said, replacing responses is non-trivial and quite error-prone. It requires a good understanding of the underlying application to know which headers need replacing and how paths are returned in response body.
As the article The "subfolder problem", OR, "why can't I reverse proxy my app into a subfolder?" explains, it’s much easier to use subdomains instead. I strongly recommend you take that approach.
That said, it does seem like Dozzle does support changing the base URL. See Base URL · Issue #4 · amir20/dozzle · GitHub