502 Gateway Error

Hello, I’m not really sure what the issue is… After, it appears that after WebSocket gets used too often, the server will start displaying a 502 Gateway error until I shut down the server and restart the computer. Any suggestions? Please and thank you!

A 502 from Caddy means that the proxied back end isn’t responding properly, so it can’t complete your request.

In order to sort out the problem, you’ll want to focus on the network between Caddy and the backend, and the backend itself, to make sure that everything’s still working fine. You could try curling the backend from the Caddy host next time it starts acting up, for example.

1 Like

Hmm… I’ll see if I can track it down - just to rule this out, does the Caddyfile look okay?

social.nofftopia.com {
proxy / http://localhost:3000/ {
transparent
}
proxy /api/v1/streaming http://localhost:4000/ {
websocket
}
}

Don’t forget that you can preserve your code formatting by highlighting it and hitting the preformatted code button:

social.nofftopia.com {
  proxy / http://localhost:3000/ {
    transparent
  }
  proxy /api/v1/streaming http://localhost:4000/ {
    websocket
  }
}

This looks like a good Caddyfile to me, though! Definitely nothing in here that would inherently cause your 502 errors.

2 Likes

Thanks, and sorry, I’m new here. So just curl -L social.nofftopia.com ? Is there anything I’m looking for?

Do you need to change the address of the websocket upstream to start with “ws://” instead of “http://”? Just a hunch, I can’t remember, but I would think that http:// should work since it does an upgrade…

No worries. Don’t curl the URL, you’ll just get Caddy again; instead, curl -i localhost:3000 and curl -i localhost:4000 to check what’s going on with your backend and API.

The -i will give you the returned headers as well as the message body, which will help build a picture of why Caddy is having trouble getting anything from them.

You’re looking for timeouts, 500 series errors, or anything else that could indicate a problem with whatever is running on :3000 or :4000. As I said, Caddy giving you a 502 generally means that there’s something wrong further up the chain, so that’s what you’re investigating.

It seems you are trying to run Mastodon with Caddy. I tried, and was not able to make it work. Ended up going with Nginx instead.

1 Like

Why wouldn’t it work? What errors were you getting?

1 Like

Can’t remember. I am no longer running Mastodon either. :frowning:

1 Like

It crashes whenever it sends out a lot of notifications / streaming. Something with WebSocket because if I disable that part, it works fine. But as far as the WebSocket goes, it takes a few hours for it to actually crash. Next time it happens, I’ll try to grab up all the error logs I can find

1 Like

Okay, just got the error, this is what Curl -i did…

Glens-MacMini:www glen$ curl -i localhost:3000
HTTP/1.1 301 Moved Permanently
Server: Mastodon
X-Frame-Options: DENY
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Location: https://localhost/
Content-Type: text/html; charset=utf-8
Vary: Accept-Encoding
Cache-Control: no-cache
X-Request-Id: 2e175575-ad9c-4050-aae5-9dda49d80ab4
X-Runtime: 0.006892
Transfer-Encoding: chunked

You are being redirected.

Glens-MacMini:www glen$ curl -i localhost:4000
HTTP/1.1 401 Unauthorized
X-Powered-By: Express
X-Request-Id: 98b0ab68-1320-44b9-8835-d8b24086aa64
Access-Control-Allow-Origin: *
Access-Control-Allow-Headers: Authorization, Accept, Cache-Control
Access-Control-Allow-Methods: GET, OPTIONS
Content-Type: application/json
Date: Wed, 03 May 2017 17:54:40 GMT
Connection: keep-alive
Transfer-Encoding: chunked

This is interesting. Can you instruct your Mastodon server not to redirect to HTTPS? Generally a bad idea to have this redirect happening between the proxy and the back end; better to let Caddy handle HTTP->S upgrading. Could also be causing issues - there’s very little chance that it will present a certificate that Caddy trusts for localhost.

A 401 unauthorized is probably a good sign that the API is working as expected, though.

Good catch! Just made the change and now its displaying this.

HTTP/1.1 302 Found Server: Mastodon X-Frame-Options: DENY X-Content-Type-Options: nosniff X-XSS-Protection: 1; mode=block Location: http://localhost:3000/about Content-Type: text/html; charset=utf-8 Vary: Accept-Encoding Cache-Control: no-cache Set-Cookie: _mastodon_session=bXhnZnczL3FNeXVaMzR1bWkySGloN0FBenFhckkyOThzVG05d2VpV2tpd1J6emdvVnVvQXlmWTFLelp3Zm15bmdvMHovRVIvcmFhRkNJNHRUMGVKZ3VkTUVCOWhmVHBEcW1McGJrano0Nzg9LS1QZ0w5WEI5c0Z0dkJBYnYzRkVVaGJBPT0%3D--ce17fcc0119e69845adb7156bcc43f977faeb973; path=/; HttpOnly X-Request-Id: ca95ce6e-64dc-4af3-9feb-1896752c521d X-Runtime: 0.048450 Transfer-Encoding: chunked

Do you think that is what was causing the issue, or not?

Actually, I can not turn it off - with it off, WebSocket doesn’t work at all

I’m leaning towards not - it doesn’t really fit the profile of the described issue.

Earlier you said that rebooting the host serves to resolve the issue - might be a good idea to try and narrow down which measure is actually effective - a Caddy restart, a Mastodon restart, or maybe even the server’s networking stack for some reason. See if you can restart them one by one next time and test in between.

I meant to post that earlier - sorry! It seems that restarting Docker itself is what brings the server back online. Not sure how to diagnose Docker for Mac though :disappointed:

Without concrete details, I’m afraid there’s just way too many ways you could have configured your Docker containers and networking for me to make guesses on where to investigate further.

If you mean “restarting Docker engine”, maybe, although it’s more likely that the containers restarting during that process is what actually resolves it.

My general advice still stands; try and provoke the issue, then restart services one by one to narrow down the guilty party. If Caddy’s in a container, restart the container, re-test, then try Mastodon’s container, etc… If they’re all in one container (unlikely - bad practice, but quite doable), you can exec in and restart them manually.

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.