AsRock IPMI websocket connection failed Caddy V2

1. Output of caddy version:

# caddy version
v2.5.2 h1:eCJdLyEyAGzuQTa5Mh3gETnYWDClo1LjtQm2q9RNZrs=

2. How I run Caddy:

a. System environment:

Running on opnsense (freebsd).

b. Command:

runs as a service
https://paste.ktz.cloud/3yelQl9XLbRYJPRxhh40Y

c. Service/unit/compose file:

Paste full file contents here.
Make sure backticks stay on their own lines,
and the post looks nice in the preview pane. -->

d. My complete Caddy config:

(cloudflare) {
  tls {
    dns cloudflare 123
}

# blue iris
bi.wd.alexktz.com {
  reverse_proxy http://10.42.1.100:83
  import cloudflare
}

# ipmi
ipmi.m.wd.alexktz.com {
  reverse_proxy https://10.42.1.11 {
    transport http {
      tls_insecure_skip_verify
    }
  }
  import cloudflare
}

3. The problem I’m having:

ASRockRack IPMI (HTML5) requires websockets to load the KVM IPMI interface. When accessing via Caddy a websocket error occurs in the web developer tools.

4. Error messages and/or full log output:

WebSocket connection to 'wss://ipmi.m.wd.alexktz.com/kvm' failed:

image

5. What I already tried:

Multiple options pertaining to adding a dedicated @ws section.

Hi @alexktz, welcome to the Caddy community.

Just to confirm, Caddy should not require any additional configuration to be able to handle websockets out of the box, so there’s something going wrong here.

This console log appears to be truncated (and so is your little screenshot) - normally I’d expect an actual error to follow (“Unexpected response code” or a timeout or something) that would tell us how this connection failed and give us an insight into why.

Can you make a websocket connection without Caddy in the way?

Yup. Direct IP is totally fine. Actually been using IPMI this way for years but thought I’d give Caddy a go.

The screenshot is not truncated, that is all the error I get.

I enabled debugging just incase it helps.

https://paste.ktz.cloud/fHI7uG1IRGIa3ke8BqjE_

All I see is that the websocket appears to return a 200 as far as Caddy is concerned. Is there anything else I can do to debug on the IPMI app side?

{"remote_ip":"10.42.13.15","remote_port":"60778","proto":"HTTP/1.1","method":"GET","host":"ipmi.m.wd.ktz.me","uri":"/kvm","headers":{"Sec-Websocket-Version":["13"],"Sec-Websocket-Key":["T963Bo8S8dUkFnk7KGDMUw=="],"Sec-Websocket-Extensions":["permessage-deflate; client_max_window_bits"],"Pragma":["no-cache"],"Accept-Language":["en-US,en-GB;q=0.9,en;q=0.8"],"X-Forwarded-For":["10.42.13.15"],"X-Forwarded-Host":["ipmi.m.wd.ktz.me"],"Cache-Control":["no-cache"],"Accept-Encoding":["gzip, deflate, br"],"Connection":["Upgrade"],"Cookie":[],"Upgrade":["websocket"],"Origin":["https://ipmi.m.wd.ktz.me"],"User-Agent":["Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36"],"Sec-Websocket-Protocol":["binary, base64"],"X-Forwarded-Proto":["https"]},"tls":{"resumed":true,"version":772,"cipher_suite":4867,"proto":"http/1.1","server_name":"ipmi.m.wd.ktz.me"}},"headers":{"Cache-Control":["no-store, no-cache, must-revalidate, private"],"Accept-Ranges":["bytes"],"Etag":["\"3577637142\""],"Date":["Fri, 26 Aug 2022 17:26:13 GMT"],"Server":["lighttpd"],"Content-Encoding":["gzip"],"X-Frame-Options":["SAMEORIGIN"],"X-Xss-Protection":["1; mode=block"],"Content-Type":["text/html"],"Pragma":["no-cache"],"Last-Modified":["Thu, 01 Jan 1970 00:00:00 GMT"],"X-Content-Type-Options":["nosniff"],"Content-Length":["607"]},"status":200}

Anyone? Sorry for the bump.

I suspect there’s been a dearth of help here because the logs provided are all on a single line, so tedious to read (we’d have to paste it into a text editor and find where the line breaks should be and/or put it into a JSON formatter that supports multiple JSON objects) – the screenshot doesn’t seem helpful (what is the code at viewer.min.js:11?) and we don’t know anything about AsRock IPMI. Have you tried asking their support?

Does the raw paste mode help at all?

https://paste.ktz.cloud/raw/fHI7uG1IRGIa3ke8BqjE_

The screenshot is all I got and it was included to show that the error was indeed WS related. I can dump the code from viewer.min.js if you like - https://paste.ktz.cloud/raw/cbWd_H01DP2Urqezdjj2e.

Asrock support don’t know either.

This is the relevant bit in the logs:

{"level":"debug","ts":1661534774.9021204,"logger":"http.handlers.reverse_proxy","msg":"upstream roundtrip","upstream":"10.42.1.11:443","duration":0.131407508,"request":{"remote_ip":"10.42.13.15","remote_port":"60777","proto":"HTTP/1.1","method":"GET","host":"ipmi.m.wd.ktz.me","uri":"/kvm","headers":{"Cookie":[],"Sec-Websocket-Protocol":["binary, base64"],"Pragma":["no-cache"],"User-Agent":["Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36"],"X-Forwarded-For":["10.42.13.15"],"X-Forwarded-Proto":["https"],"X-Forwarded-Host":["ipmi.m.wd.ktz.me"],"Origin":["https://ipmi.m.wd.ktz.me"],"Sec-Websocket-Extensions":["permessage-deflate; client_max_window_bits"],"Connection":["Upgrade"],"Upgrade":["websocket"],"Accept-Language":["en-US,en-GB;q=0.9,en;q=0.8"],"Sec-Websocket-Version":["13"],"Cache-Control":["no-cache"],"Accept-Encoding":["gzip, deflate, br"],"Sec-Websocket-Key":["roNAoZAleIiMqX9VCLaWiw=="]},"tls":{"resumed":true,"version":772,"cipher_suite":4867,"proto":"http/1.1","server_name":"ipmi.m.wd.ktz.me"}},"headers":{"Server":["lighttpd"],"X-Frame-Options":["SAMEORIGIN"],"X-Content-Type-Options":["nosniff"],"Content-Length":["607"],"Etag":["\"3577637142\""],"Date":["Fri, 26 Aug 2022 17:26:13 GMT"],"Cache-Control":["no-store, no-cache, must-revalidate, private"],"Pragma":["no-cache"],"Accept-Ranges":["bytes"],"Last-Modified":["Thu, 01 Jan 1970 00:00:00 GMT"],"Content-Encoding":["gzip"],"X-Xss-Protection":["1; mode=block"],"Content-Type":["text/html"]},"status":200}

Caddy proxies the request successfully, but lighttpd seems to not properly handle it, and just returns some HTML instead.

Maybe try making the same request with curl -v (with the same headers, your browser should let you copy the request as a curl request if you right-click on the websocket request in the network tab) and see what it actually writes back. There might be an error message in there that could help.

1 Like

This topic was automatically closed after 30 days. New replies are no longer allowed.