Help with HTTP GET vs websocket on same port

1. Caddy version (caddy version):

v2.4.3 h1:Y1FaV2N4WO3rBqxSYA8UZsZTQdN+PwcoOcAiZTM8C0I=

2. How I run Caddy:

systemd

a. System environment:

systemd

b. Command:

Paste command here.

c. Service/unit/compose file:

Paste full file contents here.
Make sure backticks stay on their own lines,
and the post looks nice in the preview pane.

d. My complete Caddyfile or JSON config:

I don’t need to post the full config, just this part, where the first one is not working, but the second one is:

	route /liveview/snapshotjson/* {
		uri strip_prefix /liveview
		reverse_proxy * http://localhost:3004 {
			import SecurityHeaders
		}
	}
	route /liveview/* {
		uri replace /liveview /socket.io
		reverse_proxy * http://localhost:3004 {
			import SecurityHeaders
		}
	}

3. The problem I’m having:

I am using JavaScript with express as the server and socket.io as the websocket. In this case, the web server and socket are sharing the same port.

In the development environment, I cannot use port 80, so I am using port 3020. And, the dev environment uses a proxy server to manage the reverse proxy.

It looks like this and works properly for both paths:

        '/liveview/snapshotjson': {
          target: 'http://localhost:3004',
          pathRewrite: { '^/liveview/': '/' },
          secure: false,
          changeOrigin: true,
          ws: false
        },
        '/liveview': {
          target: 'ws://localhost:3004',
          secure: false,
          changeOrigin: true,
          ws: true
        },

My goal is to get Caddy to do the same thing. I obviously have it wrong, because on the browser URL, I can put http://localhost/liveview/snapshotjson?camera_id=1&overlay=1 and the result is {"code":0,"message":"Transport unknown"} which is coming from socket.io

However, if I do this http://localhost:3020/liveview/snapshotjson?camera_id=1&overlay=1 while running the dev environment, then that proxy server is working because it serves up an image.

Notice above that the targets are different for the two different paths as they specify a different protocol.

Also, if I do this from the browser url, it also works: http://localhost:3004/snapshotjson?camera_id=1&overlay=1

But, I need the reverse proxy to work as the ports will be locked down from being accessed externally.

4. Error messages and/or full log output:

This is what I get in the browser:

GET http://localhost/liveview/snapshotjson?camera_id=1&overlay=1 400 (Bad Request)

5. What I already tried:

I have tried both uri strip_prefix and uri replace to no avail. :frowning:

6. Links to relevant resources:

I don’t understand. Are you saying you’re making an HTTP request from the browser, that you want Caddy to proxy as a websocket connection? That doesn’t really make sense.

Caddy can proxy incoming websocket connections, but it cannot initiate websocket connections if the incoming request was not websockets.

Make sure your browser is connecting with ws:// and not http://

What logs do you see in Caddy when you turn on debug mode?

I don’t understand. Are you saying you’re making an HTTP request from the browser, that you want Caddy to proxy as a websocket connection? That doesn’t really make sense.

No, that’s not it at all. When the Caddyfile is converted to JSON, the dial part is "upstreams":[{"dial":"localhost:3004"}]. Is there no way to distinguish protocol on that so the single connection can be split appropriately?

Not sure if you understand JavaScript, but this is how it’s done:

  // Port to use
  const port = process.env.PORT || 3004

  const server = http.createServer(app).listen(port, '127.0.0.1', () => {
    logger.log(`LiveView service using socket port ${port}`)
  })

  // set up socket.io server using same connection
  io = require('socket.io').listen(server)

  io.on('connection', (client) => {
    logger.log(`SOCKET: a client connected ${client.id}`)

    client.on('disconnect', () => {
      logger.log(`SOCKET: a client disconnected ${client.id}`)
    })
  })
  logger.log(`LiveView Service listening for socket connections on port ${port}`)

There is a difference between the http and ws protocols. And HTTP GET, for instance, will go to the “express” routes, and the sockets goes to the socket.io handler. But, for some reason, in production mode, when I don’t have the dev proxy server running (webpack-dev-server) and it’s up to Caddy, it’s not working as expected.

I was hoping you guys might be able to help with that (the Caddyfile part).

What I’m getting as is that reverse_proxy will retain the protocol of the original request. If the incoming request was a websocket upgrade request, it’ll send over a websocket upgrade request to the upstream then turn the connection into a streaming pipe. If the original request was a regular HTTP request (or HTTPS or HTTP2/3 etc) then it’ll proxy by making a regular HTTP request to the upstream. This is all automatic.

I’m not clear on what’s actually not working, because of the lack of logs. All you shared was an error response from upstream which is cryptic and could mean anything. You’ll need to dig deeper and find what code path on your upstream server the request is hitting, and find why that happens. Like I said, turn on debug mode in Caddy to see additional information about what Caddy is doing.

Aug 04 11:37:26 jeff-ubuntu caddy[13781]: {"level":"error","ts":1628098646.5160387,"logger":"http.log.error","msg":"dial tcp 127.0.0.1:3004: connect: connection refused","request":{"remote_addr":"[::1]:42788","proto":"HTTP/1.1","method":"GET","host":"localhost","uri":"/liveview/snapshotjson?camera_id=1&overlay=1","headers":{"Sec-Fetch-Site":["same-origin"],"Sec-Fetch-Dest":["empty"],"Connection":["keep-alive"],"Sec-Ch-Ua":["\"Chromium\";v=\"92\", \" Not A;Brand\";v=\"99\", \"Google Chrome\";v=\"92\""],"Sec-Ch-Ua-Mobile":["?0"],"Accept-Language":["en-US,en;q=0.9,sv;q=0.8,de;q=0.7,fr;q=0.6,pt-BR;q=0.5,pt;q=0.4"],"Cookie":["io=e-LKpVBDbmBqXSmpAAAL"],"User-Agent":["Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.131 Safari/537.36"],"Accept":["*/*"],"Sec-Fetch-Mode":["cors"],"Accept-Encoding":["gzip, deflate, br"]}},"duration":0.000375211,"status":502,"err_id":"1771jtsrq","err_trace":"reverseproxy.statusError (reverseproxy.go:857)"}
Aug 04 11:37:26 jeff-ubuntu caddy[13781]: {"level":"error","ts":1628098646.7719269,"logger":"http.log.error","msg":"dial tcp 127.0.0.1:3004: connect: connection refused","request":{"remote_addr":"[::1]:42788","proto":"HTTP/1.1","method":"GET","host":"localhost","uri":"/liveview/snapshotjson?camera_id=1&overlay=1","headers":{"Connection":["keep-alive"],"Sec-Fetch-Site":["same-origin"],"Accept-Encoding":["gzip, deflate, br"],"Accept-Language":["en-US,en;q=0.9,sv;q=0.8,de;q=0.7,fr;q=0.6,pt-BR;q=0.5,pt;q=0.4"],"Cookie":["io=e-LKpVBDbmBqXSmpAAAL"],"Sec-Ch-Ua-Mobile":["?0"],"User-Agent":["Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.131 Safari/537.36"],"Accept":["*/*"],"Sec-Fetch-Mode":["cors"],"Sec-Fetch-Dest":["empty"],"Sec-Ch-Ua":["\"Chromium\";v=\"92\", \" Not A;Brand\";v=\"99\", \"Google Chrome\";v=\"92\""]}},"duration":0.000367098,"status":502,"err_id":"4wgwewvmp","err_trace":"reverseproxy.statusError (reverseproxy.go:857)"}

Hope that helps

Are you sure your server is listening on port 3004, and that it can be reached on localhost? Clearly Caddy isn’t able to reach it.

This isn’t a problem with your Caddyfile, it’s a problem with your environment.

Sorry, wrong data…

Aug 05 05:47:16 jeff-ubuntu caddy[13781]: {"level":"debug","ts":1628164036.417864,"logger":"http.handlers.rewrite","msg":"rewrote request","request":{"remote_addr":"[::1]:45838","proto":"HTTP/1.1","method":"GET","host":"localhost","uri":"/liveview/snapshotjson?camera_id=1&overlay=1","headers":{"Accept-Language":["en-US,en;q=0.9,sv;q=0.8,de;q=0.7,fr;q=0.6,pt-BR;q=0.5,pt;q=0.4"],"Connection":["keep-alive"],"Sec-Ch-Ua":["\"Chromium\";v=\"92\", \" Not A;Brand\";v=\"99\", \"Google Chrome\";v=\"92\""],"Accept":["*/*"],"Sec-Fetch-Dest":["empty"],"Sec-Fetch-Mode":["cors"],"Accept-Encoding":["gzip, deflate, br"],"Cookie":["io=5UezAAVqXWoW_1z-AAAA"],"Sec-Ch-Ua-Mobile":["?0"],"User-Agent":["Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.131 Safari/537.36"],"Sec-Fetch-Site":["same-origin"]}},"method":"GET","uri":"/socket.io/snapshotjson?camera_id=1&overlay=1"}

It looks to me like

"method":"GET","host":"localhost","uri":"/liveview/snapshotjson?camera_id=1&overlay=1"

is getting converted to

"method":"GET","uri":"/socket.io/snapshotjson?camera_id=1&overlay=1"

Which is the second rule:

	route /liveview/snapshotjson/* {
		uri strip_prefix /liveview
		reverse_proxy * http://localhost:3004 {
			import SecurityHeaders
		}
	}
	route /liveview/* {
		uri replace /liveview /socket.io
		reverse_proxy * http://localhost:3004 {
			import SecurityHeaders
		}
    }

Ah, I see the problem now.

It’s because /liveview/snapshotjson/* does not match /liveview/snapshotjson, because the latter does not end in a /.

1 Like

Does that mean this:

route /liveview/snapshotjson/*

should be rewritten to this?

route /liveview/snapshotjson

or this?

route /liveview/snapshotjson*

The latter because * will ensure paths under that will also match. But it depends what your request paths will be. I can’t answer that for you.

1 Like

@francislavoie Hey, it worked! Thanks for your help and patience. :heart:

2 Likes
Aug 05 09:22:27 jeff-ubuntu caddy[10527]: {"level":"debug","ts":1628176947.4314816,"logger":"http.handlers.reverse_proxy","msg":"upstream roundtrip","upstream":"localhost:3004","request":{"remote_addr":"127.0.0.1:55968","proto":"HTTP/1.1","method":"GET","host":"localhost","uri":"/snapshotjson?camera_id=2&overlay=1","headers":{"Cookie":["io=ggeQ3ueO1e7SekqxAAAE"],"Sec-Fetch-Mode":["cors"],"User-Agent":["Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.131 Safari/537.36"],"X-Forwarded-Proto":["http"],"X-Real-Ip":["127.0.0.1"],"Sec-Ch-Ua":["\"Chromium\";v=\"92\", \" Not A;Brand\";v=\"99\", \"Google Chrome\";v=\"92\""],"Accept-Encoding":["gzip, deflate, br"],"Sec-Fetch-Dest":["empty"],"Accept-Language":["en-US,en;q=0.9,sv;q=0.8,de;q=0.7,fr;q=0.6,pt-BR;q=0.5,pt;q=0.4"],"Referer":["http://localhost:3020/"],"Access-Control-Allow-Origin":["*"],"Access-Control-Allow-Credentials":["true"],"Access-Control-Allow-Headers":["Cache-Control,Content-Type"],"Sec-Fetch-Site":["same-origin"],"Sec-Ch-Ua-Mobile":["?0"],"Accept":["*/*"],"X-Forwarded-For":["127.0.0.1"]}},"headers":{"Strict-Transport-Security":["max-age=15552000; includeSubDomains"],"X-Permitted-Cross-Domain-Policies":["none"],"X-Xss-Protection":["0"],"Vary":["Origin, Accept-Encoding"],"Content-Type":["application/json"],"Content-Security-Policy":["default-src 'self';base-uri 'self';block-all-mixed-content;font-src 'self' https: data:;frame-ancestors 'self';img-src 'self' data:;object-src 'none';script-src 'self';script-src-attr 'none';style-src 'self' https: 'unsafe-inline';upgrade-insecure-requests"],"Expect-Ct":["max-age=0"],"X-Content-Type-Options":["nosniff"],"Content-Encoding":["gzip"],"Date":["Thu, 05 Aug 2021 15:22:27 GMT"],"X-Dns-Prefetch-Control":["off"],"X-Download-Options":["noopen"],"Referrer-Policy":["no-referrer"],"Server":["SmrtHVR"],"X-Frame-Options":["SAMEORIGIN"],"Access-Control-Allow-Origin":["*"]},"status":200}

In case you wanted to see the results…

This topic was automatically closed after 30 days. New replies are no longer allowed.