Caddy reverse proxy for OpenHAB errors

Help
1

1. The problem I’m having:

I can’t make OpenHAB work correctly with Caddy as the reverse proxy. The OpenHAB sitemap itself loads fine, but something with websockets is just not working right: e.g. Firefox keeps throwing Firefox can’t establish a connection to the server at https://openhab.elektrik.link/rest/sitemaps/events/e289c7e0-f1ee-44e1-9516-abd1d44fcd84?sitemap=openhab&pageid=openhab. in the console.

With NGINX it all appears to be working fine or at least I am not getting any errors either within Firefox or NGINX’s logs.

2. Error messages and/or full log output:

3. Caddy version:

v2.8.4 h1:q3pe0wpBj1OcHFZ3n/1nl4V4bxBrYoSoab7rL9BMYNk=

4. How I installed and ran Caddy:

Downloaded a binary.

a. System environment:

None of this seems relevant, since Caddy isn’t crashing or something.

d. My complete Caddy config:

{
        email nita.vesa@outlook.com
        acme_dns porkbun {
        REDACTED
        }
}
openhab.elektrik.link {
        reverse_proxy * http://localhost:8080 {
                transport http {
                    keepalive 1h
                }
                header_up Strict-Transport-Security max-age=31536000
                header_up X-Real-IP {remote_host}
                header_up X-Forwarded-Proto {scheme}
                header_up Connection Upgrade
                header_up Upgrade websocket
                header_up Access-Control-Allow-Origin *
                header_up Access-Control-Allow_Credentials true
                header_up Access-Control-Allow-Headers Authorization,Accept,Origin,DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Content-Range,Range
                header_up Access-Control-Allow-Methods GET,POST,OPTIONS,PUT,DELETE,PATCH
                stream_timeout 24h
        }
}

5. Links to relevant resources:

The NGINX config that appears to be working fine:

server {
    listen                          80;
    server_name                               openhab.elektrik.link;
    return 301                      https://$server_name$request_uri;
}
server {
    listen                          443 ssl;
    server_name                               openhab.elektrik.link;

    # Cross-Origin Resource Sharing
    add_header 'Access-Control-Allow-Origin' '*' always;
    add_header 'Access-Control-Allow_Credentials' 'true' always;
    add_header 'Access-Control-Allow-Headers' 'Authorization,Accept,Origin,DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Content-Range,Range' always;
    add_header 'Access-Control-Allow-Methods' 'GET,POST,OPTIONS,PUT,DELETE,PATCH' always;

    ssl_certificate /var/lib/caddy/.local/share/caddy/certificates/acme-v02.api.letsencrypt.org-directory/openhab.elektrik.link/openhab.elektrik.link.crt;
    ssl_certificate_key /var/lib/caddy/.local/share/caddy/certificates/acme-v02.api.letsencrypt.org-directory/openhab.elektrik.link/openhab.elektrik.link.key;
    add_header                      Strict-Transport-Security "max-age=31536000";

    location / {
        proxy_pass                            http://localhost:8080/;
        proxy_set_header Host                 $http_host;
        proxy_set_header X-Real-IP            $remote_addr;
        proxy_set_header X-Forwarded-For      $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto    $scheme;
        proxy_set_header Upgrade                $http_upgrade;
        proxy_set_header Connection             "Upgrade";
        proxy_read_timeout 3600;
    }
}

Especially the proxy_read_timeout directive seems important as without it I’d get similar errors with NGINX. Alas, I cannot for the life of me find an equivalent in Caddy.

2

Remove all of this. None of it is relevant. Many of those are response headers, but you’re adding it to the request being sent upstream. And touching Connection and Upgrade is likely to break things in unintended ways.

If you want to set response headers, you should use the header directive (outside of reverse_proxy).

That’s really long. I recommend you leave it at the default, i.e. 2 minutes. There’s almost never any reason to keep an idle connection open for a whole hour.

2 Likes
3

Removing all of that doesn’t improve anything, though. I only added those in an effort to imitate the NGINX config in the hopes of getting things working.

4

Please collect new logs with your config cleaned up and simplified and show what you get. The logs are a mess because of the header_up lines confusing the content (can’t tell which requests are actually websockets and which aren’t because you’re adding those headers to every request)

1 Like
5

Well, here: caddylog - Pastebin.com

Ignore anything referring to influxdb or grafana. Caddy was serving as reverse proxy for those as well, but I removed the entries for this log (and they were working perfectly well, they’re not related to this)

6

I’m not seeing any websocket requests in there now.

The /sitemaps/events/ stuff looks like it’s a text/event-stream, not websockets. It’s probably Server-Sent Events (SSE) Using server-sent events - Web APIs | MDN

From the log, "error":"context canceled" tells me that either the client (your browser) or the upstream (OpenHAB/Jetty) cancelled the request after 30 seconds.

Could you also add log to your site, this will enable access logs as well. It would be good to see what that shows, it should show the actual overall request/response Caddy wrote instead of just the proxy part from the debug logs.

2 Likes
7

Oh, I thought it was websockets. I’ll have a squizz at the link, just to learn something new. Thanks.

As for the access log, you didn’t specify if you’d like it formatted in some manner, so I just used the defaults. The log can be found at caddyaccesslog - Pastebin.com

EDIT:
Adding the following to Caddyfile in the global section possibly helped. I’ll have to see over a couple of hours how it behaves. I don’t really understand why this would help, though, but I guess it does the same thing as NGINX’s proxy_read_timeout?

        servers {
                timeouts {
                        idle        1h
                }
        }

EDIT2: Nope, still not working.

8

I’ve tried disabling compression, setting all sorts of timeouts to 1h, fiddling with keepalive, tuning flush_interval and no matter what I do, I still cannot make this SSE work. It’s ridiculous to have to use NGINX as the reverse proxy just for this one, single thing, when I am using Caddy for everything else.

9

Can you disable HTTP/3 to see if sse works?

10

I’ve only tested for a few minutes now, but it seems like that might have worked. But…why? Why would HTTP/3 break SSE?

If you can provide one, I would very much appreciate some sort of an explanation.

EDIT:
Disabling HTTP/3 appears to have worked. For anyone stumbling upon this topic, all that’s apparently needed to use Caddy as reverse proxy for OpenHAB amounts to:

my.openhab.domain {
	# Remove the header that browsers use to determine that HTTP/3 is available
        header -Alt-Svc
        reverse_proxy localhost:8080
}

Removing the header instead of disabling HTTP/3 in Caddy globally allows any other reverse proxies you might have defined in the same file to still use HTTP/3.

11

Odd. Is there a simple way to reproduce this?

1 Like
12

Not that I can tell, no. The only situation where I’ve had this issue is with OpenHAB and, well, having to install it and all that just to test this out would be quite a nuisance.

13

Tnx, Works for me when removeing the Alt-Svc header.
But when i use the android app the authentication fails.
@WereCatf have you tried any of the apps and got it working?

14

I have no issues with the official OpenHAB app on my phone.

15

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.