Certificate issuance only using ZeroSSL

Help
1

We’ve setup as described here and everything is working well, but we’ve noticed that only ZeroSSL certs are being acquired. I understood this would be the fall back and thus most certs should be from Letsencrypt

As you can see we have quite a number of certs
find certificates/ -type d | cut -d ‘/’ -f1-2 | wc -l
1123

find certificates/ -type d | cut -d ‘/’ -f1-2 | sort -u
certificates/
certificates/acme.zerossl.com-v2-dv90

2

I moved your comment to a new topic, because it wasn’t really relevant to that one.

Please fill out the help topic template (click New Topic, copy the template filled into the text box) into a new comment in this topic.

3

1. Caddy version (caddy version):

v2.5.1

2. How I run Caddy:

Systemd as caddy user

a. System environment:

AWS EC2 Instance pool
Linux 5.4.0-1072-aws #77~18.04.1-Ubuntu x86_64 x86_64 x86_64 GNU/Linux

b. Command:

service caddy start

c. Service/unit/compose file:

[Unit]
Description=Caddy HTTP/2 web server
Documentation=https://caddyserver.com/docs
After=network-online.target remote-fs.target
Wants=network-online.target systemd-networkd-wait-online.service

[Service]
Restart=on-abnormal

; User and group the process will run as.
User=root
Group=www-data

; Letsencrypt-issued certificates will be written to this directory.
Environment=CADDYPATH=/etc/ssl/caddy

; Always set "-root" to something safe in case it gets forgotten in the Caddyfile.
ExecStart=/usr/local/bin/caddy -log-timestamps=false -log /var/log/caddy/caddy.log -agree=true -conf=/etc/caddy/Caddyfile -root=/var/tmp -email=ssl@fzautomotive.com

ExecReload=/bin/kill -USR1 $MAINPID

; Use graceful shutdown with a reasonable timeout
KillMode=mixed
KillSignal=SIGQUIT
TimeoutStopSec=5s

; Limit the number of file descriptors; see `man systemd.exec` for more limit settings.
LimitNOFILE=1048576
; Unmodified caddy is not expected to use more than that.
LimitNPROC=2048

; Use private /tmp and /var/tmp, which are discarded after caddy stops.
PrivateTmp=true
; Use a minimal /dev (May bring additional security if switched to 'true', but it may not work on Raspberry Pi's or other devices, so it has been disabled in this dist.)
PrivateDevices=false
; Hide /home, /root, and /run/user. Nobody will steal your SSH-keys.
ProtectHome=true
; Make /usr, /boot, /etc and possibly some more folders read-only.
ProtectSystem=full
; … except /etc/ssl/caddy, because we want Letsencrypt-certificates there.
;   This merely retains r/w access rights, it does not add any new. Must still be writable on the host!
ReadWriteDirectories=/etc/ssl/caddy

; The following additional security directives only work with systemd v229 or later.
; They further restrict privileges that can be gained by caddy. Uncomment if you like.
; Note that you may have to add capabilities required by any plugins in use.
;CapabilityBoundingSet=CAP_NET_BIND_SERVICE
;AmbientCapabilities=CAP_NET_BIND_SERVICE
;NoNewPrivileges=true

[Install]
WantedBy=multi-user.target

d. My complete Caddyfile or JSON config:

# Global Options
# https://caddyserver.com/docs/caddyfile/options
{
        admin off

        #Order for rate limiting
        order rate_limit before basicauth

        # TLS Options
        email ssl@fzautomotive.com
        on_demand_tls {
                ask https://sites.example.com/allowed
                interval 2m
                burst 25
        }

        grace_period 10m

        #Set Certificate Location
        storage file_system /etc/ssl/caddy/v2
}

:443 {
        tls {
                #ca https://acme-staging-v02.api.letsencrypt.org/directory
                on_demand
        }

        reverse_proxy @geofilter http://127.0.0.1:7777

        #Set GeoBlocking
        #https://github.com/porech/caddy-maxmind-geolocation
        @geofilter {
                maxmind_geolocation {
                        db_path /etc/caddy/GeoLite2-Country.mmdb
                        #deny_countries CH SG RU RO NL BE AU
                        deny_countries RU RO VN
                }
        }

        #Set Rate Limiting
        #https://github.com/mholt/caddy-ratelimit
        rate_limit {
                zone dynamic {
                        key {remote_host}
                        events 60
                        window 120s
                }
        }
        #respond "I'm behind the rate limiter!"

        #CDN Setting
        uri replace s3.amazonaws.com/example static.example.com

        #Set Headers
        header {
                Strict-Transport-Security max-age=31536000;
                X-Content-Type-Options nosniff
                X-XSS-Protection "1; mode=block"
        }

        #Enable Compression
        encode {
                #gzip 9
                gzip 9

                #match {
                #header Content-Type text/*
                #}
        }

        #Define Logs
        #https://caddyserver.com/docs/caddyfile/directives/log
        log {
                output file /var/log/caddy/access.log {
                        roll_size 25
                        roll_keep 10
                        roll_keep_for 7d
                }
                format filter {
                        wrap console {
                                time_format rfc3339
                                time_key timestamp
                        }
                        fields {
                                request>headers>Accept delete
                        }
                }
        }
}

#Define Snippet for On-Demand TLS
(onDemand) {
        tls {
                on_demand
        }
}

#Include Redirs
import redir2/*
import redir2MultiDomain/*

3. The problem I’m having:

There is no real problem, but it appears that all our certificates are from ZeroSSL. It was understood this is the fallback, but only expected a few to be ZeroSSL and most from Letsencrypt.
Would just like to make sure there isn’t an issue with Letsencypt certificates being obtained,

4. Error messages and/or full log output:

/etc/ssl/caddy/v2# find certificates/ -type d | cut -d ‘/’ -f1-2 | wc -l
1123

find certificates/ -type d | cut -d ‘/’ -f1-2 | sort -u
certificates/
certificates/acme.zerossl.com-v2-dv90

5. What I already tried:

Tried deleting a certificate and stables, new certificate was generated but again from ZeroSSL

6. Links to relevant resources:

4

Do your logs (Caddy’s logs) show any indication of a problem issuing with Let’s Encrypt? You didn’t show those logs.

You can run journalctl -u caddy --no-pager | less +G to see Caddy’s logs.

Your config looks fine, I don’t think it should cause it to only use ZeroSSL, so I can only assume something is failing when it tries Let’s Encrypt causing the fallback, and the logs should show that.

5

There are 4 domains that for some reason keep trying to renew, these are all old clients who have since migrated sites away from our service, so the dns entries do not point at our cluster. I’m not even sure why or how those domains are being requested if the dns is “incorrect”.
These do try Letsencrypt, but obviously fail, would this cause valid domains to only use ZeroSSL?

6

That might be a known bug in ZeroSSL… they’ll try to validate domains even after authz is cleaned up, I’m not really sure what’s going on there.

This would make sense if ZeroSSL is holding onto the previously-resolved domain lookup from when it did point to your server, maybe.

If Caddy already has/had a certificate for a domain from a certain issuer, it will prefer that issuer again when renewing.

7

Ok so basically all our certs are ZeroSSL, so unless that renewal fails they will likely stay that way
Will test a cert deletion and see if it comes from Letsencryt

8

Sounds like a plan. The logs would also be helpful here, as they should indicate whether LE is being tried at all.

9

This topic was automatically closed after 29 days. New replies are no longer allowed.