Issues with the ZeroSSL TLS setup

Help
1

1. Caddy version (caddy version):

v2.2.1

2. How I run Caddy:

Using Caddy Alpine.

a. System environment:

Caddy is run in a pod, inside Kubernetes, inside Minikube.

b. Command:

Our Caddy pod Dockerfile:

ARG CADDY_VERSION=2.2.1
FROM caddy:${CADDY_VERSION}-builder-alpine AS builder

ARG ROUTE53_VERSION=1.0.2
RUN xcaddy build \
    --with github.com/caddy-dns/route53@v${ROUTE53_VERSION}

FROM caddy:${CADDY_VERSION}-alpine
COPY --from=builder /usr/bin/caddy /usr/bin/caddy

ENV XDG_DATA_HOME=/data \
    XDG_CONFIG_HOME=/config

COPY root /

c. Service/unit/compose file:

apiVersion: apps/v1
kind: Deployment
metadata:
    labels:
        svc: static
        type: deployment
    name: static
    namespace: 'development'
spec:
    replicas: 1
    selector:
        matchLabels:
            svc: static
    strategy:
        rollingUpdate: null
        type: Recreate
    template:
        metadata:
            labels:
                svc: static
        spec:
            containers:
                - image: 'local/static:latest'
                  name: static
                  ports:
                      - containerPort: 80
                      - containerPort: 443
                  volumeMounts:
                      - mountPath: '/data/caddy'
                        name: static-storage
                      - mountPath: '/config/caddy'
                        name: static-storage
            restartPolicy: Always
            volumes:
                - name: static-storage
                  persistentVolumeClaim:
                      claimName: 'development-static-volume-claim'

d. My complete Caddyfile or JSON config:

(tls-majority) {
    tls {$ACME_CA_EMAIL} {
        dns route53
    }

    header Strict-Transport-Security max-age=31536000;
}

(tls-alpha) {
    import tls-majority
}

(tls-beta) {
    import tls-majority
}

(tls-production) {
    import tls-majority
}

(tls-local) {
    tls {
        issuer zerossl {$ZERO_SSL_API_KEY} {
            dns route53
        }
    }
}

(default) {
    encode gzip

    log

    import tls-{$TARGET_ENV}

    header {
        Content-Security-Policy "frame-ancestors 'none'"
        X-Content-Type-Options "nosniff"
        X-Frame-Options "DENY"
        X-XSS-Protection "0"
    }
}

https://{$BASE_DOMAIN} {
    import default

    root * /www
    file_server

    @exceptProxy not path /css/* /fonts/* /images/* /js/* /favicon.ico /robots.txt /version.json
    reverse_proxy @exceptProxy http://app:{$APP_SERVICE_PORT_HTTP}
}

# For the Kubernetes liveness probe
:80 {
    respond /_status/ping 200
}

3. The problem I’m having:

Caddy seemed to get itself into a bind at some point with:

static-79d9d8fb99-x2lmr static {"level":"info","ts":1606267303.5151153,"logger":"tls.issuance.acme.acme_client","msg":"trying to solve challenge","identifier":"my.domain","challenge_type":"dns-01","ca":"https://acme.zerossl.com/v2/DV90"}
static-79d9d8fb99-x2lmr static {"level":"info","ts":1606267378.6840162,"logger":"tls.issuance.acme.acme_client","msg":"validations succeeded; finalizing order","order":"https://acme.zerossl.com/v2/DV90/order/0a7qMB_jt0fNhttgAdz2YA"}
static-79d9d8fb99-x2lmr static {"level":"warn","ts":1606267411.2951508,"logger":"tls.issuance.acme.acme_client","msg":"HTTP request failed; retrying","url":"https://acme.zerossl.com/v2/DV90/order/0a7qMB_jt0fNhttgAdz2YA","error":"performing request: Post \"https://acme.zerossl.com/v2/DV90/order/0a7qMB_jt0fNhttgAdz2YA\": http2: timeout awaiting response headers"}
static-79d9d8fb99-x2lmr static {"level":"error","ts":1606267415.1428266,"logger":"tls.obtain","msg":"will retry","error":"[my.domain] Obtain: [my.domain] finalizing order https://acme.zerossl.com/v2/DV90/order/0a7qMB_jt0fNhttgAdz2YA: polling order status: fetching new nonce from server: HTTP 504:  (ca=https://acme.zerossl.com/v2/DV90)","attempt":1,"retrying_in":60,"elapsed":116.224921962,"max_duration":2592000}
static-79d9d8fb99-x2lmr static {"level":"error","ts":1606267476.4744296,"logger":"tls.obtain","msg":"will retry","error":"[my.domain] Obtain: [my.domain] creating new order: fetching new nonce from server: HTTP 504:  (ca=https://acme.zerossl.com/v2/DV90)","attempt":2,"retrying_in":120,"elapsed":177.556524306,"max_duration":2592000}

It couldn’t get past fetching new nonce from server.

After clearing out the config directory, Caddy started again and it worked. Caddy acquired the certs and is now serving everything correctly, but I did receive the following error:

{"level":"warn","ts":1606268308.7926667,"logger":"tls","msg":"stapling OCSP","error":"no OCSP stapling for [my.domain]: parsing OCSP response: ocsp: error from server: unauthorized"}

I’d just like to point out that this was using a setup that was previously working with Let’s Encrypt. I simply switched to the new code without clearing any directories or anything.

2

A 504 error is a gateway timeout on their end. Probably something’s up with ZeroSSL infrastructure. An earlier message indicates a timeout awaiting headers, so indeed it looks like their servers weren’t writing a response in time.

(Fortunately, Caddy 2.3 – soon to be pre-released – supports mutli-issuer fallback, so it can try another CA if there’s an error with the first one! It’s the first web server to implement this.)

The OCSP stapling warning is just that, a warning, it means that there is no OCSP available for the certificate. This is also something the CA will have to fix. (Let’s Encrypt had this issue early on, years ago, but fixed it so it’s pretty rare now.)

3

Thanks for the information Matt. Do you have any tips on where I can report this to ZeroSSL? Otherwise, I’ll just strike up a support request as I’m a paying customer.

4

That’s a good question, let me find out and get back to you!

In the meantime I’ll also direct some attention to this thread. I’m sure it will be taken care of.

5

Hey Matt, fantastic, thank you!

6

This topic was automatically closed after 30 days. New replies are no longer allowed.