Can Caddy do TLS passthrough

Help
1

1. Caddy version (caddy version):

v2.3.0 h1:fnrqJLa3G5vfxcxmOH/+kJOcunPLhSBnjgIvjXV/QTA=

2. How I run Caddy:

a. System environment:

Linux, Ubuntu, systemd

c. Service/unit/compose file:

[Unit]
Description=Caddy
Documentation=https://caddyserver.com/docs/
After=network.target

[Service]
User=caddy
Group=caddy
Environment=CLOUDFLARE_API_TOKEN=REDACTED
ExecStart=/usr/bin/caddy run --environ --config /etc/caddy/Caddyfile
ExecReload=/usr/bin/caddy reload --config /etc/caddy/Caddyfile
TimeoutStopSec=5s
LimitNOFILE=1048576
LimitNPROC=512
PrivateTmp=true
ProtectSystem=full
AmbientCapabilities=CAP_NET_BIND_SERVICE

[Install]
WantedBy=multi-user.target

d. My complete Caddyfile or JSON config:

Main file:

{
    email <email>
}

(cf) {
    tls {
        dns cloudflare {env.CLOUDFLARE_API_TOKEN}
    }
}

(local_only) {
    @external {
        not remote_ip 10.0.0.0/8
    }
    respond @external 403
}

import /etc/caddy/Caddyfile.d/*

drone.fap.no {
   import cf

    reverse_proxy https://kubernetes.terra.fap.no {
        header_up Host "drone.fap.no"
        transport http {
            tls_insecure_skip_verify
            # tls_server_name "drone.fap.no"
        }
    }
}

3. The problem I’m having:

The problem, or what I would like to do:

My Kubernetes cluster has an Ingress (think reverse proxy) that already has a valid certificate for the domain in the example (drone.fap.no) and I would like to just use that certificate.

Currently my “public facing” HTTP server is Caddy, and I dont need it to create a Cert for the domain and then reverse proxy.

I would like to have Caddy just take in TLS packages which matches drone.fap.no and pass them on to the next host, if that isn’t supported, I would like to:

Have Caddy get a Cert and terminate “drone.fap.no”, then reverse_proxy it to my Kubernetes ingress, but over HTTPS. This will require it to pass the correct hostname and then validate the TLS cert presented.

I have tried different combinations of passing headers and using the SNI name (tls_server_name), but it does not seem to pass it correctly.

With the example in this post, I see a lot of the following errors:

Feb 06 17:39:09 core.terra caddy[2247810]: {"level":"error","ts":1612633149.7018335,"logger":"http.log.error","msg":"remote error: tls: internal error","request":{"remote_addr":"10.69.0.1:54822","proto":"HTTP/2.0","method":"GET","host":"drone.fap.no","uri":"/","headers":{"User-Agent":["Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0.3 Safari/605.1.15"],"Accept-Language":["en-us"],"Accept-Encoding":["gzip, deflate, br"],"Cookie":["_session_=<REDACTED>="],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"]},"tls":{"resumed":false,"version":772,"cipher_suite":4867,"proto":"h2","proto_mutual":true,"server_name":"drone.fap.no"}},"duration":0.006644412,"status":502,"err_id":"n8zar6gpz","err_trace":"reverseproxy.statusError (reverseproxy.go:783)"}

Thanks :slight_smile:

Let me know if you need more information.

Kristoffer

2

Unfortunately, no it’s not possible with vanilla Caddy. But you can use the caddy-l4 plugin for this. But it doesn’t support Caddyfile config at this time.

But it should be possible to do what you want, i.e. terminate TLS at Caddy then do a new handshake to your upstream.

This line should not be necessary, Caddy passes through the Host from the original request.

I think this is on the right track. If the upstream cert isn’t in Caddy’s trust store, then you may need to specify tls_trusted_ca_certs.

Alternatively, if you can have Caddy grab the certificate/key from your cluster, then it could reuse that instead of having one issued via ACME.

1 Like
3

Thanks for the tip @francislavoie

I have built a version containing l4 now and I am able to setup a proper TLS passthrough when configuring a caddy that uses l4 alone and not any other app, like http.

It looks like when I use a mix of l4 and http, the http app will snap up the connection before l4 can deal with it and it results in:

Feb 07 16:20:06 core.terra caddy[2288202]: {"level":"debug","ts":1612714806.5956323,"logger":"http.stdlib","msg":"http: TLS handshake error from 132.145.57.158:52132: no certificate available for 'drone.fap.no'"}

This might be expected behaviour though, I am not sure if I should be able to use both http and l4?

If it is possible, maybe someone can point me in the direction of what I might do wrong? There is currently no mention of drone.fap.no in the apps.http config.

Thanks
Kristoffer

4

What you probably want to do is bind l4 to your HTTPS port, then bind http to something else. In your l4 config, then, simply proxy to the http socket for all HTTP connections (that you’re not doing TLS passthru with).

1 Like
5

Hi @matt

I manage to get it working following your suggestion, not as elegant I suppose but it works.

Next thing to think about now is that it breaks my IP matchers :stuck_out_tongue:

Thanks
Kristoffer

6

This topic was automatically closed after 30 days. New replies are no longer allowed.