Client TLS Setup (iOS, macOS)

There’s a lot of detail here, but the short of it is that I’m pretty sure I’m not setting up my clients correctly or that Caddy isn’t requesting client certs in a way that my browsers can understand. If anyone has a working tutorial that works with browsers on macOS or iOS, I’ll try that.

1. Caddy version (caddy version): 2.2.1

2. How I run Caddy:

Docker Compose

a. System environment:

Raspberry Pi 2, Raspbian

b. Command:

caddy run

c. Service/unit/compose file:

---
version: '2.4'
services:
  caddy:
    image: caddy:2.2.1
    ports:
      # Local ports
      - "80:80"
      - "443:443"
      # Public ports forwarded from :80 and :443 on router
      - "8080:8080"
      - "8443:8443"
    volumes:
      - ./Caddyfile:/etc/caddy/Caddyfile
      - ./root_ca.crt:/etc/caddy/client_ca.crt
      - caddy_data:/data
    volumes_from:
      - certs
    environment:
      BASICAUTH_USER_HASH: ${BASICAUTH_USER_HASH}
    restart: always

  # Update star cert for internal domain (*.example.rocks and *.example.com)
  certs:
    build: ./lego
    volumes:
      - /srv/volumes/lego:/.lego
    labels:
      - 'dockron.schedule=0 0 * * 0'  # Weekly on Sunday
    environment:
      CLOUDFLARE_EMAIL: ${CLOUDFLARE_EMAIL}
      CLOUDFLARE_API_KEY: ${CLOUDFLARE_API_KEY}
    dns: 1.1.1.1

  # Update Cloudflare record for public home domain (home.example.com)
  ddns:
    image: iamthefij/cloudflare-ddns:linux-arm
    labels:
      - 'dockron.schedule=0 * * * *'  # hourly on the 0
    environment:
      DOMAIN: home.example.com
      CF_API_EMAIL: ${CLOUDFLARE_EMAIL}
      CF_API_KEY: ${CLOUDFLARE_API_KEY}
    dns: 1.1.1.1

  whoami:
    image: containous/whoami

volumes:
  caddy_data:

d. My complete Caddyfile or JSON config:

{
    debug
}

(add_basicauth) {
    basicauth {
        {$BASICAUTH_USER_HASH}
    }
}

# Adds basic auth and a simple reverse proxy to args.0
(proxy_auth) {
    import add_basicauth
    reverse_proxy {args.0} {
        header_up -Authorization
        header_up X-Webauth-User "{http.auth.user.id}"
    }
}

(lego_tls) {
    tls {
        load "/.lego"
    }
}

ok.example.rocks {
    # TLS certs must be loaded in at least one route
    # This route loads the TLS certs from file and acts as a healthcheck
    import lego_tls
    respond "OK"
}

pihole.example.rocks {
    reverse_proxy raspberrypi.example:9080
}

hass.example.rocks {
    reverse_proxy hapi.example:8123
}

# Public ports

# Public http port, redirect to https
# This is automatic for 80>443 but not custom ports
:8080 {
    redir https://{host}{uri}
}

auth_test.example.rocks:8443 {
    tls {
        client_auth {
            mode require_and_verify
            trusted_ca_cert_file "/etc/caddy/client_ca.crt"
        }
    }
    respond "Hi auth!"
}

3. The problem I’m having:

I’m encountering certificate errors when trying to access my auth_test via browsers on my desktop and mobile. I’m not totally sure if I’m setting up certificates correctly though, but Safari doesn’t even ask for a cert but instead says the connection is not private despite showing a trusted Lets Encrypt cert.

4. Error messages and/or full log output:

n/a

5. What I already tried:

When I try to curl the endpoint, it does appear to work:

# no certs presented
$ curl https://auth_test.example.rocks
curl: (56) OpenSSL SSL_read: error:14094412:SSL routines:ssl3_read_bytes:sslv3 alert bad certificate, errno 0
# certs presented
$ curl --cert ./user2.crt --key ./user2.key https://auth_test.example.rocks
Hi auth!

I’m not sure if the unauthenticated response is as expected either or if it should have instead returned a 401 or something.

I’ve also tried changing the directive a bit to verify_if_given

auth_test.example.rocks:8443 {
    tls {
        client_auth {
            mode verify_if_given
            trusted_ca_cert_file "/etc/caddy/client_ca.crt"
        }
    }
    respond "Hi {tls_client_subject}! I'm {system.hostname}"
}

With this curl shows:

# no certs presented
$ curl https://auth_test.example.rocks
Hi {http.request.tls.client.subject}! I'm 57f61d47b467
# certs presented
$ curl --cert ./user2.crt --key ./user2.key https://auth_test.example.rocks
Hi CN=user2! I'm 57f61d47b467

But still, no browsers ask for certificates.

6. Links to relevant resources:

Hmm, well you requested curl https://auth_test.example.rocks but you’re listening on port 8443. That doesn’t align.

The SSL error is correct though, because TLS verification happens when the TLS handshake is made, which is before HTTP. So any issues there won’t result in HTTP errors, but instead with a failed TLS handshake. That’s working as intended.

The verify_if_given mode does not require certs, so it will just keep going if the user never presented a cert. If you present a bad cert though (untrusted one), then it will fail.

What you can do is check the value of {http.request.tls.client.subject} with the expression matcher and return an HTTP error if it’s empty. Maybe something like this:

@nocert expression {http.request.tls.client.subject} == ''
respond @nocert "No cert provided" 403

I haven’t tested that, you’ll need to play around with it to see what works.

As an aside, why are you using a separate lego container for managing wildcard certs from cloudflare? You know you can use Caddy for this, right?

You just need to build Caddy with the cloudflare plugin. See the docs on docker hub for how to write the Dockerfile. It’s very easy. Docker Hub

Hmm, well you requested curl https://auth_test.example.rocks but you’re listening on port 8443. That doesn’t align.

Sorry. That’s unclear from my snippets above. I have my router forward public:80/public:443 > caddy:8080/caddy:8443. That’s why you’re seeing that. I do this to make it explicit what hosts I expose to the outside internet and which are reserved for the internal network only.

The SSL error is correct though, because TLS verification happens when the TLS handshake is made, which is before HTTP. So any issues there won’t result in HTTP errors, but instead with a failed TLS handshake. That’s working as intended.

Ah. Ok. So as I suspected, the issue must just be that I haven’t figured out how to get my browsers to present the client cert. Has anyone had luck with this yet?

As an aside, why are you using a separate lego container for managing wildcard certs from cloudflare? You know you can use Caddy for this, right?

It’s been on my to-do list for a while now. :slight_smile: The sidecar is left over from when I used to use Traefik v1 and wanted to generate one wildcard cert for example.rocks and a home.example.com cert, which wasn’t possible.

I’ve been hesitant to switch over because I don’t want to force my Pi to build the image since it’s rather slow to do so. I need to set up a pipeline to do that and push to a registry so my Pi can pull it down, however doing that means I’ll have to bump the version in two places every time I want to upgrade. On top of that, there’s a minor peace of mind in knowing that some breach in Caddy, however unlikely, won’t provide access to my Cloudflare API key.

@andrew-kennedy since you did a great guide on mTLS with Basic Auth fallback, I’m assuming you’ve managed to get your client browsers to present certs successfully. Any chance you could point me in the right direction?

Ok, so I think I’ve figured out what I was doing wrong. The user.crt and user.key files are not the right format for Safari or Firefox. I had to create a .p12 file and import that.

This was done with openssl pkcs12 -export -in user.crt -inkey user.key -name user > user.p12

Edit: Just found this can be done with step using step certificate p12 user.p12 user.crt user.key

For future travelers, here’s the steps I took using step and openssl.

Create CA and generate root cert

# initialize a local ca
$ step ca init

# locate root certs
$ ls ~/.step/certs/
intermediate_ca.crt  root_ca.crt

Configure Caddy

In Caddyfile example above, set trusted_ca_cert_file "/path/to/root_ca.crt"

Generate client cert

Since auto updating client certs on your phone is not an option, you’ll want a longer than 24h expiration. I set mine to 1 year. (365*24=8760)
Open .step/config/ca.json and add the following to the “authority” block

"claims": {
    "maxTLSCertDuration": "8760h"
},

Now you can generate the actual cert for a user

# generate user cert (uses password from CA initialization)
$ step ca certificate --offline --not-after=8760h "myuser" myuser.crt myuser.key

# this can be used directly by curl using
$ curl --cert ./myuser.crt --key ./myuser.key https://auth_test.caddy

# generate p12 for Safari and Firefox
$ step certificate p12 ./myuser.p12 ./myuser.crt ./myuser.key

Finally, manually import ./user.p12 into client

2 Likes

I’m going to jot down an idea here for someone (or me) to look into later…

It’s really cool that Caddy provides functionality like caddy trust and caddy hash-password to make installing root certs and hashing passwords for basic auth. It would be great to have something similarly simple to create and manage client certs using the built in CA rather than having to set up a new one using the steps above.

I’m picturing something like caddy client-create "user" to generate key and crt files with the provided user using a similar args as step above. I suspect this to be doable given Caddy is using the step libraries already. The advantage would be that Caddy would already have the root cert so Caddy would not have to import a new root CA. Revocation could be done with something like caddy client-revoke could be provided.

1 Like

Neat idea, can you open an issue to request that feature?

1 Like

Done! Feature: Caddy command to generate signed client certificates · Issue #3924 · caddyserver/caddy · GitHub

Oddly enough… The .p12 cert generated using step certificates p12 works just fine in Firefox on macOS, however it doesn’t work in Safari. I instead have to generate a p12 using openssl pkcs12 -export -in user.crt -inkey user.key -name user > user.p12. This then works in macOS Safari.

Unfortunately, neither of these appear to work in iOS Safari. So the hunt still continues.

I read that for Safari to present certificates that they have to be downloaded from Safari too. Not sure if this is true, but I created an internal route to allow users to download a client cert over the VPN so that they can use that to access from outside.

I did this using the following rule and it worked great.

client_setup.example.rocks {
    import add_basicauth
    file_server {
        root "/client_certs/{http.auth.user.id}"
        browse
    }
}

Afterwards I noticed two things. One, when I try to access my auth_test.example.rocks route (documented above), I get the following error. I’m not sure if I noticed it before, but I get the error even if I remove the cert from my phone.

caddy_1   | {"level":"debug","ts":1609616862.1972022,"logger":"http.stdlib","msg":"http: TLS handshake error from 73.152.34.112:58004: no certificate available for '172.28.0.2'"}
caddy_1   | {"level":"debug","ts":1609616862.4322636,"logger":"http.stdlib","msg":"http: TLS handshake error from 73.152.34.112:58005: no certificate available for '172.28.0.2'"}
caddy_1   | {"level":"debug","ts":1609616862.6423342,"logger":"http.stdlib","msg":"http: TLS handshake error from 73.152.34.112:58006: tls: client offered only unsupported versions: [301]"}

This seems odd because Safari should definitely support newer TLS versions.

Also, I think I may have uncovered an issue with the Caddy tls directive. As soon as I access my auth_test route, all of my other routes will respond with that error. If I visit my ok.example.rocks route where I also specify a tls directive, it fixes it.

It appears that setting the tls directive sets it globally, or at the very least, on an interface. Is this the case?

This topic was automatically closed after 30 days. New replies are no longer allowed.