I did try to set keepalive off in the reverse_proxy section but it didn’t have any effect. Apparently with websockets only the initial handshake is HTTP, after that it’s TCP which is probably why it had no effect.
I decided to set up TLS properly in rabbitmq and terminate it there rather than Caddy, and it actually wasn’t that difficult. Connecting straight to the broker fixes the TCP keepalive problem, disconnections and allows much higher keepalive values like 10 minutes.
Having read the rabbitmq TLS documentation I now think that using Caddy for proxying MQTT isn’t a good idea. It does allow a secure connection and allows the client to verify the server, but it doesn’t allow the server to verify the client. Most people will be using LetsEncrypt certificates for a public website where you don’t need to verify the client, but with MQTT you will want to do this, and additionally you probably want each device to have a unique client certificate so they can be revoked if necessary. It doesn’t seem like you can generate LetsEncrypt client certificates because then anyone with a LetsEncrypt server certificate could also generate one.
Proxying through Caddy also makes it more complicated to diagnose any problems as now the connection is going through two things, and there is also the whole issue of keepalives.
I think most people doing IOT will actually be using some hosted service like AWS/GCP which takes away all this complexity and deals with provisioning certificates etc.