Well, it depends on how the connections are handled. If you have something acting as a gateway that proxies the request (i.e. consumes it, then makes a new request), then it would need to make available the original IP address somehow.
Typically when it’s HTTP all the way down, headers like X-Forwarded-For are used to tell the next proxy upstream the remote IP. Caddy’s reverse_proxy directive does automatically, for example. If you’re doing the proxying at the TCP level, then headers can’t be used, so the PROXY protocol can be used (there’s a Caddy plugin for this, currently only supports JSON config, but Caddyfile support is coming soon).
Either way, whatever you have in front of Caddy needs to support either of these solutions, otherwise you’ll continue to just see the IP address from the proxy.