Jellyfin videos not playing behind Caddy on VPS but WebUI works perfectly

Help
1
  1. The problem I’m having:
    I’m trying to set up remote access to my Jellyfin server. I’m behind CGNAT so have a VPS to provide the external IP address.
    Internet —> Caddy on VPS —> Wireguard Tunnel —> Home Server with Jellyfin and other WebUIs

Almost everything is working, I can access JF WebUI and other WebUIs without issue. However, videos will not play in Jellyfin. Looking at logs the Jellyfin websocket video stream is immediately dropped with status 500 error.

2. Error messages and/or full log output:

Jellyfin Log

[2023-11-09 15:40:07.639 +00:00] [INF] WS "185.132.40.84" request
[2023-11-09 15:40:55.672 +00:00] [INF] Sending ForceKeepAlive message to 1 inactive WebSockets.
[2023-11-09 15:41:07.670 +00:00] [INF] Sending ForceKeepAlive message to 1 inactive WebSockets.
[2023-11-09 15:41:19.671 +00:00] [INF] Lost 1 WebSockets.
[2023-11-09 15:46:15.729 +00:00] [WRN] WS "185.132.40.84" error receiving data: "The remote party closed the WebSocket connection without completing the close handshake."
[2023-11-09 15:46:15.752 +00:00] [INF] WS "185.132.40.84" closed
[2023-11-09 15:48:22.028 +00:00] [INF] WS "192.168.1.219" request
[2023-11-09 15:48:28.815 +00:00] [INF] WS "185.132.40.84" request
[2023-11-09 15:48:37.625 +00:00] [ERR] Error processing request: "Reading the request body timed out due to data arriving too slowly. See MinRequestBodyDataRate". URL "POST" "/Items/79f4b98a7a4a98ebe94bcb28828f114b/PlaybackInfo".
[2023-11-09 15:48:37.625 +00:00] [WRN] Slow HTTP Response from "https://jf.121fitz.org/Items/79f4b98a7a4a98ebe94bcb28828f114b/PlaybackInfo?UserId=46c31364365c4b259d5b5dde9dce063f&StartTimeTicks=0&IsPlayback=true&AutoOpenLiveStream=true&MaxStreamingBitrate=400000000" to "185.132.40.84" in 0:00:05.0720818 with Status Code 500
[2023-11-09 15:49:22.031 +00:00] [INF] Sending ForceKeepAlive message to 1 inactive WebSockets.
[2023-11-09 15:49:31.633 +00:00] [ERR] Error processing request: "Reading the request body timed out due to data arriving too slowly. See MinRequestBodyDataRate". URL "POST" "/Items/29bc54335a2e4554ebb9005d3e69536e/PlaybackInfo".
[2023-11-09 15:49:31.633 +00:00] [WRN] Slow HTTP Response from "https://jf.121fitz.org/Items/29bc54335a2e4554ebb9005d3e69536e/PlaybackInfo?UserId=46c31364365c4b259d5b5dde9dce063f&StartTimeTicks=0&IsPlayback=true&AutoOpenLiveStream=true&MaxStreamingBitrate=350000000" to "185.132.40.84" in 0:00:05.5765341 with Status Code 500
[2023-11-09 15:49:34.031 +00:00] [INF] Lost 1 WebSockets. [2023-11-09 15:50:58.648 +00:00] [ERR] Error processing request: "Reading the request body timed out due to data arriving too slowly. See MinRequestBodyDataRate". URL "POST" "/Items/dad8f7b62c69fc6e3037fd8d923ea4a3/PlaybackInfo".
[2023-11-09 15:50:58.648 +00:00] [WRN] Slow HTTP Response from "https://jf.121fitz.org/Items/dad8f7b62c69fc6e3037fd8d923ea4a3/PlaybackInfo?UserId=46c31364365c4b259d5b5dde9dce063f&StartTimeTicks=0&IsPlayback=true&AutoOpenLiveStream=true&MaxStreamingBitrate=350000000" to "185.132.40.84" in 0:00:05.1655555 with Status Code 500
[2023-11-09 15:52:46.034 +00:00] [INF] Sending ForceKeepAlive message to 1 inactive WebSockets.

3. Caddy version:

V2.7.5

4. How I installed and ran Caddy:

a. System environment:

Ubuntu 23.04 on both VPS and home server

b. Command:

systemctl enable caddy
systemctl start caddy

d. My complete Caddy config:

Caddyfile

jf.121fitz.org {
        reverse_proxy 10.0.0.1:8096
}

qb.121fitz.org {
        reverse_proxy 10.0.0.1:8080
}

pr.121fitz.org {
        reverse_proxy 10.0.0.1:9696
}

rad.121fitz.org {
        reverse_proxy 10.0.0.1:7878
}

so.121fitz.org {
        reverse_proxy 10.0.0.1:8989
}

re.121fitz.org {
        reverse_proxy 10.0.0.1:8787
}

abr.121fitz.org {
        reverse_proxy 10.0.0.1:7474
}

5. Links to relevant resources:

Caddy debug log: https://github.com/caddyserver/caddy/files/13317422/jellyfin.log

2

I don’t think Caddy is the problem. We have plenty of users (myself included) using Caddy to proxy Jellyfin and it works fine.

I think you should reach out for help from the Jellyfin devs to see if they make anything of it, since the warnings/errors appear at the application layer.

3

I agree, I also have experience using Caddy to successfully proxy Jellyfin when caddy is running on the same computer.

Could there be an interaction with the wireguard tunnel?

Do people have experience proxying Jellyfin with caddy on another device or only to localhost?

I will raise this on the jellyfin forum as well.

4

I use Jellyfin + Caddy through a Wireguard tunnel (technically it’s Tailscale) without issues. :man_shrugging:

5

I think you do it differently though Matt, they have Caddy on the VPS and wireguard from the VPS to home and Jellyfin is at home. I think you run both Caddy and Jellyfin at home and connect to Caddy through Tailscale?

6

True; I wonder if OP’s problem is related to network latency or configuration between home and VPS?

7

Gentlemen, thank you so much for your help.

The issue turned out to be the MTU in the wireguard tunnel. The tunnel MTU was set to 1340 but no packets larger than 1268 bytes were able to pass through it. I’m guessing some downstream equipment simply dropped packets larger than 1268 + wireguard overhead. Setting MTU to 1260 for the tunnel resolved the issue.

MTU seemed to only affect video streaming probably because all the webui packets were small and the streaming packets were as big as possible.

2 Likes