Auto HTTPS with Tailscale not working on 2.7.3

1. The problem I’m having:

In the new 2.7.3 update, SSL with Tailscale is not working. I’ve tested on 2.7.3 and then again on 2.6.4. It still does work on 2.6.4.

2. Error messages and/or full log output:

Here are the debug logs from 2.7.3:

{"level":"debug","ts":1691996068.5566437,"logger":"events","msg":"event","name":"tls_get_certificate","id":"bc161537-3252-4d91-a30b-101f18f7fb43","origin":"tls","data":{"client_hello":{"CipherSuites":[39578,4865,4866,4867,49195,49199,49196,49200,52393,52392,49171,49172,156,157,47,53],"ServerName":"machine-name.tailscale-host.ts.net","SupportedCurves":[64250,29,23,24],"SupportedPoints":"AA==","SignatureSchemes":[1027,2052,1025,1283,2053,1281,2054,1537],"SupportedProtos":["h2","http/1.1"],"SupportedVersions":[31354,772,771],"Conn":{}}}}

{"level":"debug","ts":1691996068.556993,"logger":"tls.handshake","msg":"no matching certificates and no custom selection logic","identifier":"*.*.ts.net"}

{"level":"debug","ts":1691996068.557011,"logger":"tls.handshake","msg":"no certificate matching TLS ClientHello","remote_ip":"100.*.*.*","remote_port":"49482","server_name":"machine-name.tailscale-host.ts.net","remote":"100.*.*.*:49482","identifier":"machine-name.tailscale-host.ts.net","cipher_suites":[39578,4865,4866,4867,49195,49199,49196,49200,52393,52392,49171,49172,156,157,47,53],"cert_cache_fill":0,"load_or_obtain_if_necessary":true,"on_demand":false}

3. Caddy version:

2.7.3

4. How I installed and ran Caddy:

a. System environment:

Debian, tested on both bullseye and bookworm.

b. Command:

start.sh

sysctl -w net.core.rmem_max=2048000

mkdir --parents /data/caddy/caddy /data/tailscale

tailscaled --state=/var/lib/tailscale/tailscaled.state --socket=/var/run/tailscale/tailscaled.sock &
sleep 5
if [ ! -S /var/run/tailscale/tailscaled.sock ]; then
    echo "tailscaled.sock does not exist. exit!"
    exit 1
fi

until tailscale up --authkey=${TS_AUTHKEY} --hostname=${TS_HOSTNAME}
do
  sleep 0.1
done

exec "$@"

command:

CMD ["caddy", "run", "--config", "/config/caddy/caddy.json"]

c. Service/unit/compose file:

This is for Fly.io

  services = [
    {
      ports = [
        {
          port     = 443
          handlers = ["tls", "http"]
        },
        {
          port     = 80
          handlers = ["http"]
        }
      ]
      "protocol" : "tcp",
      "internal_port" : 443
    }
  ]

d. My complete Caddy config:

{
  "admin": {
    "disabled": true
  },
  "logging": {
    "logs": {
      "default": {
        "level": "DEBUG"
      },
      "platform": {
        "level": "DEBUG",
        "writer": {
          "output": "stdout"
        },
        "include": [
          "http.log.access.platform"
        ]
      }
    }
  },
  "apps": {
    "http": {
      "http_port": 80,
      "https_port": 443,
      "servers": {
        "services": {
          "listen": [":80", ":443"],
          "logs": {
            "default_logger_name": "platform"
          },
          "routes": [
            {
              "handle": [
                {
                  "handler": "subroute",
                  "routes": [
                    {
                      "match": [
                        {
                          "host": [
                            "fly-app.tailnet-name.ts.net"
                          ]
                        }
                      ],
                      "handle": [
                        {
                          "handler": "reverse_proxy",
                          "upstreams": [
                            {
                              "dial": "fly-app.internal:9101"
                            }
                          ]
                        }
                      ]
                    }
                  ]
                }
              ],
              "terminal": true
            }
          ]
        }
      }
    }
  }
}

5. Links to relevant resources:

It works for me.

What are your full logs? No redactions please (this is a forum rule).

Please post the full logs as the help template requires so we know how to help you.