Caddy requests the ip addresses belonging to an hostname via getaddrinfo (via /run/systemd/resolve/io.systemd.Resolve), this will return IP6 ::1 and IP4 127.0.0.1.
Caddy tries to connect to the first IP address. If it returns a failure during the connect call, it tries the second one. (verified this using strace -f caddy ...)
The problem in your case is that the connect call succeeds, so Caddy start sending the body, but then your remote server (podman) closes the connection with a reset (likely because podman tried to connect using IPv6 to your container, but failed, so it closed the connection directly after it accepted it)
Fix podman to not listen on IPv6 if it cannot proxy calls on IPv6
EDIT: If podman acts like docker, if the container has access to IPv6, make sure that the program in the container is listing on :: instead of 0.0.0.0