TLS offload no longer working in V2

1. Caddy version (caddy version):

Caddy 2.0.0h1

2. How I run Caddy:

Caddy as a reverse Proxy

a. System environment:

Debian Jessie

b. Command:

caddy run --watch

c. Service/unit/compose file:

systemd

d. My complete Caddyfile or JSON config:

########V1 Caddyfile########
mesh.mydomain.com {
    proxy / 192.168.1.63:443 {
        transparent
        websocket
  }
      tls {
        protocols tls1.2
        }
  timeouts none
}
########################

###########Full Caddyfile V2#################

shadowcontrol.mydomain.com {
     reverse_proxy 192.168.1.230:80
}

nextcloud.mydomain.com {
      reverse_proxy 192.168.1.95:80
 }

mail.mydomain.com, secure.mydomain.com, backup.mydomain.com, helpdesk.mydomain.com, monitor.mydomain.com, remote.mydomain.com, traveler.mydomain.com,
       reverse_proxy 192.168.1.1:80
 }

community.mydomain.com  {
       reverse_proxy 192.168.1.17:80
 }

backup2.mydomain.com  {
       reverse_proxy 192.168.1.21
 }

backup3.mydomain.com  {
       reverse_proxy 192.168.1.65
 }

wordpress.mydomain.com  {
        reverse_proxy 192.168.1.75:80
}

mesh.mydomain.com  {
        reverse_proxy 192.168.1.63:443
  tls {
     protocols tls1.2
    }
}
##############################


3. The problem I’m having:

The last directive (mesh.mydomain.com) is producing Handshake errors

4. Error messages and/or full log output:

“proto_mutual”: true, “server_name”: “mesh.mydomain.com”}}, “duration”: 0.001492795, “status”: 502, “err_id”: “jisht93ij”, “err_trace”: “reverseproxy.(*Handler).ServeHTTP (reverseproxy.go:380)”}
2020/05/14 16:25:26.119 ERROR http.log.error tls: first record does not look like a TLS handshake {“request”: {“method”: “GET”, “uri”: “/agent.ashx”, “proto”: “HTTP/1.1”, “remote_addr”: “96.89.67.73:60421”, “host”: “mesh.mydomain.com”, “headers”: {“Upgrade”: [“websocket”], “Connection”: [“Upgrade”], “Sec-Websocket-Key”: [“YSIgONIoHhLrHhZHbmVFR1==”], “Sec-Websocket-Version”: [“13”]}, “tls”: {“resumed”: false, “version”: 772, “ciphersuite”: 4867, “proto”: “”, “proto_mutual”: true, “server_name”: “mesh.mydomain.com”}}, “duration”: 0.001672582, “status”: 502, “err_id”: “kjap0eegz”, “err_trace”: “reverseproxy.(*Handler).ServeHTTP (reverseproxy.go:380)”}
2020/05/14 12:25:27 [INFO] Deactivating auth: https://acme-v02.api.letsencrypt.org/acme/authz-v3/4582566992
2020/05/14 12:25:27 [INFO] Unable to deactivate the authorization: https://acme-v02.api.letsencrypt.org/acme/authz-v3/4582566992
2020/05/14 12:25:27 [ERROR] error: one or more domains had a problem:

5. What I already tried:

Googled and forum searches for examples of other working Caddyfiles in V2 , but I’m pretty sure my syntax is wrong. I think this may have something to do with http transport or headers or something, but I’m confused on if I need that or not.

6. Links to relevant resources:

This scenario is to connect to a MeshCentral Server, where TLS offloading to Caddy was previously working in V1. It was difficult to get working, but the solution was documented here:

This just means that the client isn’t making a TLS connection; it’s not a problem with Caddy. Make sure the client is using HTTPS.

Is the mesh.domain.com responding correctly on 192.168.1.63:443?
What happens if you remove the tls block? (I would try both on 192.168.1.163:443 then from caddy).

So far, when I found that line in my log, it appeared that the service was indeed not running. Just a guess. (And I decided to handle that specific error with a 503 instead of 502)

Alex

@matt thanks for the response. Nothing has changed since my functioning V1 at the client… where the previous V1 Caddyfile worked. Perhaps you can help me shed some light on this…

#####Client’s Mesh JSON Config file, offloading TLS to caddy at 192.168.1.7#######
{
“settings”: {
“selfupdate”: true,
“Cert”: “mesh.mydomain.com”,
“Port”: 443,
“IgnoreAgentHashCheck”: true,
“TlsOffload”: “192.168.1.7”,#caddyserver
“Minify”: false,
“RedirPort”: 80,
“DesktopMultiplex”: true
},
“domains”: {
“”: {
“certurl”: “https://mesh.mydomain.com”,
“title”: “”,
“titlePicture”: “title-southeastern.png”,
“newAccounts”: 0,
“title2”: “”,
“footer”: “”
}
},
“smtp”: {
“host”: “westwind.mydomain.com”,
“port”: 25,
“from”: “mesh@mydomain.com”,
“user”: “notifications@mydomain.com”,
“pass”: “”,
“tls”: false
}
}
##################

@dukeart
Note the Client is responding currently responding at http://192.168.1.64:443, but not https://

Thanks to both of you for the help.
Shane

@dukeart so, removing the TLS block doesn’t seem to change anything and the service is def running as I can pull it up in the browser on “http://192.168.1.63:443

I realize that’s http on port 443, but for some reason that I can’t recall anymore, we needed to do it that way for tls offloading to caddy…
thanks

:exploding_head:

Have you tried specifying the scheme in your reverse_proxy so Caddy doesn’t assume :443 is HTTPS (like a good little web program)?

1 Like

Glad you got something out of my comment (it is terribly phrased…but you got the idea).
@Whitestrake is right, caddy expects standard scheme/port config unless you specify your own.

Good luck!

1 Like

Guys, so you’re saying just set it up like any other standard HTTP/80 scheme? And see if that will work?

OK, In short, @dukeart and @Whitestrake got this. It was playing badly with the destination port on 443. I moved that port to 4430 and it worked right away. Thanks so much!

1 Like

@cfm56 do you think you could show me your Caddyfile and docker-compose.yml? I’m having the same issue and tried what you said but it still didn’t work.

Hey PopeRigby,

Sure thing, I’m happy to help, however, since I’m self hosted, and not on Docker, I can’t help with the docker file. Here is my entire working Caddyfile. This allows me to have a number of servers all https secure on my LAN, all secured by caddy. Hope it helps
###########################################
control.mydomain.com {
reverse_proxy 192.168.1.230:80 {
}
}
cloud.mydomain.com {
reverse_proxy 192.168.1.95:80 {
}
}
mail.mydomain.com, secure.mydomain.com, backup.mydomain.com, helpdesk.mydomain.com, mobile.mydomain.com {
reverse_proxy 192.168.1.1:80 {
}
}
monitor.mydomain.com {
reverse_proxy 192.168.1.2:1443 {
}
}
community.mydomain.com {
reverse_proxy 192.168.1.17:80 {
}
}
backup2.mydomain.com {
reverse_proxy 192.168.1.21:4280 {
}
}
backup3.mydomain.com {
reverse_proxy 192.168.1.65:4280 {
}
}
wordpress.mydomain.com {
reverse_proxy 192.168.1.75:80 {
}
}
mesh.mydomain.com {
reverse_proxy 192.168.1.63:4430 {
}
tls {
protocols tls1.2
}
}
####################################