Help with Caddy + Mainsail + WebRTC

1. The problem I’m having:

I have managed to set up Caddy (with security plugins) to reverse proxy my Mainsail 3D printing server to a domain name, however I am unable to get the webcam streaming to forward correctly.

My Mainsail instance uses a Go2RTC stream from Frigate on a different IP address. This stream displays fine when accessing my Mainsail instance locally, over http, via the IP address, but shows ‘Disconnected’ when accessed via the domain name.

2. Error messages and/or full log output:

[Error] WebSocket connection to 'wss://192.168.1.28:5000/live/webrtc/api/ws?src=3D_Printer&media=video' failed: An SSL error has occurred and a secure connection to the server cannot be made.

3. Caddy version:

v2.8.0

4. How I installed and ran Caddy:

a. System environment:

delver/caddy-security docker image
docker-compose installation on debian 12 container in proxmox

b. Command:

docker compose up -d

c. Service/unit/compose file:

services:
  caddy:
    image: delver/caddy-security:latest
    container_name: caddy
    restart: unless-stopped
    cap_add:
      - NET_ADMIN
    ports:
      - "80:80"
      - "443:443"
      - "443:443/udp"
    volumes:
      - ./caddy/Caddyfile:/etc/caddy/Caddyfile
      - ./caddy/srv:/srv
      - ./caddy/data:/data
      - ./caddy/config:/config
  ddclient:
    image: lscr.io/linuxserver/ddclient:latest
    container_name: ddclient
    environment:
      - PUID=0
      - PGID=0
      - TZ=Europe/London
    volumes:
      - ./ddclient/config:/config
    restart: unless-stopped

d. My complete Caddy config:

{
  order authenticate before respond
  order authorize before reverse_proxy

  security {
      local identity store localdb {
          realm local
          path /data/caddy/auth/local/users.json
      }

      authentication portal myportal {
          enable identity store localdb
          cookie domain mysite.com
          cookie lifetime 864000
          crypto default token lifetime 864000
          ui {
              links {
                  "Frigate" https://frigate.mysite.com
                  "Mainsail" https://mainsail.mysite.com
                  "Redlib" https://redlib.mysite.com
              }
          }
          transform user {
              match email my@email.com
              action add role authp/user
              ui link "Settings" "/auth/profile/" icon "las la-cog"
          }
      }


      authorization policy admin_policy {
        set auth url https://auth.mysite.com
        allow roles authp/user
    }
 }

}


auth.mysite.com {
 authenticate with myportal
}

home.mysite.com: {
 reverse_proxy 192.168.1.200:8123
}

diskstation.mysite.com: {
 reverse_proxy 192.168.1.23:5000
}

mainsail.mysite.com {
 authorize with admin_policy
 reverse_proxy 192.168.1.32

    handle /printer* {
        reverse_proxy 192.168.1.32:7125
    }
    handle /api* {
        reverse_proxy 192.168.1.32:7125
    }
    handle /access* {
        reverse_proxy 192.168.1.32:7125
    }
    handle /websocket* {
        reverse_proxy 192.168.1.32:7125
    }
    handle /machine* {
        reverse_proxy 192.168.1.32:7125
    }
    handle /server* {
        reverse_proxy 192.168.1.32:7125
    }
}

frigate.mysite.com {
 authorize with admin_policy
 reverse_proxy 192.168.1.28:5000
}

redlib.mysite.com {
 authorize with admin_policy
 reverse_proxy 192.168.1.45:8080
}

scrypted.mysite.com {
 reverse_proxy 192.168.1.35:11080
}

5. Links to relevant resources:

Please upgrade to v2.8.4

Caddy doesn’t have a certificate for 192.168.1.28, it’s configured to listen for specific domains. You’ll need to adjust the app to try to connect using your domain name on port 443.

2 Likes

Thanks for your quick and helpful reply! So rather than pulling the webcam from Frigate over the local IP (192.168.1.28) I set it to pull from the domain (frigate.mysite.com) instead - I had to remove the auth protocol from frigate.mysite.com to get it to work, but I am now able to get the webcam streaming correctly on mainsail.mysite.com - excellent!

I still want to have frigate.mysite.com protected by caddy-security and require auth though - is there a way I can set it up so that it bypasses auth for requests made from my mainsail server at 192.168.1.32?

You can use the remote_ip request matcher to apply a condition to a handler.

2 Likes
frigate.mysite.com {
  @internal_network {
        remote_ip xxxxx         
  }
  handle @internal_network {
        reverse_proxy 192.168.1.28:5000
  }
  handle {
        authorize with admin_policy
        reverse_proxy 192.168.1.28:5000
  }
}

I presume it should look something like this? However there’s an issue - if mainsail is requesting the webrtc stream from frigate via the public url, then the remote_ip is going to be the wan ip of my router, which will change regularly because of my ISP?

Sorry if this is all really basic stuff, I’m quite new to Caddy and feel I’ve hit the steeper part of the learning curve…!

Or alternatively:

@external not remote_ip <ip-ranges>
authorize @external ...

That way you only have one reverse_proxy and no handle.

The remote_ip will be that of the client making the request, not the address being requested by the client. So it wouldn’t change if your ISP changes your WAN IP.

Unless you mean you’re hosting this somewhere outside your home network and are connecting to it from home.

1 Like

Thanks, that’s a much neater code block :wink:

Still can’t quite get this right. Changing my code to:

frigate.mysite.com {
  @external not remote_ip 192.168.1.0/24
  authorize @external with admin_policy
  reverse_proxy 192.168.1.28:5000
}

to allow local ips (like the mainsail server) to access streams still results in an error:

[Error] WebSocket connection to 'wss://frigate.mysite.com/live/webrtc/api/ws?src=3D_Printer&media=video' failed: There was a bad response from the server.

Looking at the Caddy logs I can see the request producing an error as follows:

{"level":"error","ts":1721551517.0589218,"logger":"http.handlers.authentication","msg":"auth provider returned error","provider":"authorizer","error":"user authorization failed: src_ip=81.132.166.142, src_conn_ip=172.70.86.190, reason: no token found"}

where src_ip is my current isp-assigned wan ip address and src_conn_ip is an ip address that doesn’t mean anything to me but seems to vary depending on whether I’m accessing Mainsail via my phone or my laptop (appears to be device-dependent).

I did try changing my Caddyfile to:

frigate.mysite.com {
  @external not remote_ip 172.0.0.0/8
  authorize @external with admin_policy
  reverse_proxy 192.168.1.28:5000
}

to account for these ip addresses but whilst it fixed the streaming, it also stopped auth from working - I could access Frigate without having to log in using a cache/cookie-cleared phone on an external network.

Unless you mean you’re hosting this somewhere outside your home network and are connecting to it from home.

Both the Mainsail and Frigate servers are running on separate containers on my Proxmox server.

I should probably add that my domain name is being provided by Cloudflare which I believe masks my home ip address, would that be affecting things?

Yes it would. In that case, you need to use the client_ip matcher instead, and set up trusted_proxies in global options to trust Cloudflare’s IP ranges. See Global options (Caddyfile) — Caddy Documentation, you’ll likely want to use this plugin to make it easier for trusted IPs: GitHub - WeidiDeng/caddy-cloudflare-ip

1 Like

Thanks so much for taking the time to explain all this to me, I really appreciate it.

I’ll need to go and rebuild my Caddy instance so I can include that module (and update to 2.8.4) but just to clarify - would trusting the Cloudflare IP ranges still mean my Frigate server is secured by my auth setup? Would any server running through Cloudflare potentially be able to access my Frigate server? I think I’m confused about the chain of requests - sorry :frowning:

Yes, all it does is parse the real IP from HTTP headers then make it available to the logs and to client_ip etc.

I don’t think anything “runs” under those IPs except for Cloudflare’s proxy. But in theory yes if someone was able to make requests appearing to come from Cloudflare, and override HTTP headers for those requests to have your specific IP address inside the Cf-Connecting-Ip header, then they would bypass auth. But that’s very unlikely and it would require knowledge of your config. It would need to be someone being intentionally malicious, not some bot.

1 Like

EDITED:

Nearly there! So I’ve now rebuilt to 2.8.4 and have the requested modules installed. My caddyfile now looks like this:

{
  servers {
    trusted_proxies cloudflare {
      interval 12h
      timeout 15s
    }
  }

  order authenticate before respond
  order authorize before reverse_proxy

...

frigate.mysite.com {
  @external not client_ip 192.168.1.32
  authorize @external with admin_policy
  reverse_proxy 192.168.1.28:5000
}

I’m very pleased to report that this setup does work correctly and streams the webcam through Mainsail!

I do have a slightly odd issue in that the stream doesn’t work when I’m trying to connect on my phone externally to my local network (it just says ‘connecting’) yet my laptop loads the stream fine when hotspotting through my phone - however I can’t find any logs or errors to explain what’s going on yet so I will have to keep digging…

In the meantime - can’t thank you enough for your help in getting this fixed, really appreciate it!

I can’t help with the auth plugins, I have no experience with them.

1 Like

I spoke too soon - see edited post!

1 Like

Fixed the final issue with external streaming - I hadn’t forwarded the WebRTC port correctly on my router for the STUN server to work. With the port now opened it all works great!

1 Like

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.