Reverse Proxy timeouts for tusd

Help
1

1. Caddy version (caddy version):

v2.2.1

2. How I run Caddy:

Using standard systemd startup script.

a. System environment:

AWS Amazon Linux 2 on Lightsail nano instance

b. Command:

No commands. Purely systemd.

c. Service/unit/compose file:

    # caddy.service
    #
    # For using Caddy with a config file.
    #
    # Make sure the ExecStart and ExecReload commands are correct
    # for your installation.
    #
    # See https://caddyserver.com/docs/install for instructions.
    #
    # WARNING: This service does not use the --resume flag, so if you
    # use the API to make changes, they will be overwritten by the
    # Caddyfile next time the service is restarted. If you intend to
    # use Caddy's API to configure it, add the --resume flag to the
    #  command or use the caddy-api.service file instead.
    [Unit]
    Description=Caddy
    Documentation=https://caddyserver.com/docs/
    After=network.target network-online.target
    Requires=network-online.target
    [Service]
    User=caddy
    Group=caddy
    ExecStart=/usr/bin/caddy run --environ --config /var/lib/caddy/Caddyfile
    ExecReload=/usr/bin/caddy reload --config /var/lib/caddy/Caddyfile
    TimeoutStopSec=5s
    LimitNOFILE=1048576
    LimitNPROC=512
    PrivateTmp=true
    ProtectSystem=full
    AmbientCapabilities=CAP_NET_BIND_SERVICE
    [Install]
    WantedBy=multi-user.target

d. My complete Caddyfile or JSON config:

(server) :443 {
    reverse_proxy /files/* http://127.0.0.1:10080 {
        flush_interval -1
        header_up Connection Upgrade
        header_up Upgrade Websocket
    }

    root * /var/lib/caddy/wwwroot
    file_server
    log {
        level DEBUG
        output file /var/lib/caddy/access.log
    }
}

3. The problem I’m having:

I’ve been using tusd lately and trying to use Caddy to terminate SSL and act as a reverse proxy. The https part is working fine and the reverse proxy works to a degree but seems to fail under load. I’ve attempted to convert the reverse proxy settings in this nginx config and so far have got this:

reverse_proxy /files/* http://127.0.0.1:10080 {
  flush_interval -1
  header_up Connection Upgrade
  header_up Upgrade Websocket
}

Like I said, it seems to work, but not under load where I start getting “read tcp: i/o timeout” errors in the tusd logs when it tries to send a response back. I believe the data is streamed from the client so the connection could be getting dropped?

Bypassing Caddy and going direct to tusd works as expected.

4. Error messages and/or full log output:

tusd[1127]: [tusd] 2020/12/16 10:37:32 event=“ResponseOutgoing” status=“500” method=“PATCH” path=“snip” error=“read tcp: i/o timeout” requestId=""

5. What I already tried:

The config above.

6. Links to relevant resources:

2

That looks strange, is that a typo? You shouldn’t have a snippet and a site address on the same line.

You don’t need these lines, Caddy passes through all headers transparently by default. The browser will be setting these headers when it makes the websocket request.

I’m not sure why tusd would give errors, the tusd team would probably have more to say, since they also wrote it in Go (they should be able to compare how Caddy handles the connections compared to their server, I suppose). I see you’ve opened an issue there already

FWIW it would be pretty cool to see a tus plugin for Caddy (i.e. running tus as a request handler embedded within Caddy)

3

This topic was automatically closed after 30 days. New replies are no longer allowed.