Looking at Dedicated servers - Official Satisfactory Wiki, it definitely looks like it’s UDP, so I’m pretty sure it’s not HTTP.
You won’t be able to use vanilla Caddy to proxy it. Caddy ships with an HTTP server and proxy.
If you want to proxy raw TCP or UDP, then you’d need to use caddy-l4, as mentioned earlier.