V2: Automatic https certificate errors

HNRK · January 21, 2020, 6:24am

Continued from https://caddy.community/t/re-v2-config-doesnt-run-after-beta-9/6822.

Two more things to add before I create an issue on Github:

My automatic https setting for three domains is not working as expected.

All of the three dollowing sites have static htmls.

Caddyfile:

weather-frogs.de, www.weather-frogs.de {
        root * /etc/caddy/html/weather-frogs.de
        encode brotli zstd gzip
        php_fastcgi unix//run/php/php7.3-fpm.sock
        try_files {path} /index.php?{query}
        file_server
}
die-reikiquellen.de, www.die-reikiquellen.de {
        root * /etc/caddy/html/die-reikiquellen.de
        encode brotli zstd gzip
        php_fastcgi unix//run/php/php7.3-fpm.sock
        try_files {path} /index.php?{query}
        file_server
}
harmoniks.de, www.harmoniks.de {
        root * /etc/caddy/html/harmoniks.de
        encode brotli zstd gzip
        php_fastcgi unix//run/php/php7.3-fpm.sock
        try_files {path} /index.php?{query}
        file_server
}

Complete Caddy json config (extracted from autosave.json):

https://gist.github.com/Henrocker/26690bfe08b14eed337dae2c3d2a130b

will give me this message in chrome:

This site can’t provide a secure connection
www.die-reikiquellen.de sent an invalid response.
Try running Windows Network Diagnostics.
ERR_SSL_PROTOCOL_ERROR

and this one in logs:

â— v2caddy.service - Caddy v2
   Loaded: loaded (/lib/systemd/system/v2caddy.service; enabled; vendor preset: enabled)
   Active: active (running) since Mon 2020-01-20 15:06:42 CET; 19min ago
     Docs: https://caddyserver.com/docs/
 Main PID: 1432 (caddy)
    Tasks: 15 (limit: 4915)
   Memory: 20.5M
   CGroup: /system.slice/v2caddy.service
           â””â”€1432 /etc/v2caddy/caddy run --config /etc/v2caddy/Caddyfile --adapter caddyfile

Jan 20 15:06:42 hnrk.io caddy[1432]: 2020/01/20 14:06:42.269        INFO        http        skipping automatic certificate management because one or more matching certificates are already loaded        {"domain": "hnrk.io", "server_name": "srv1"}
Jan 20 15:06:42 hnrk.io caddy[1432]: 2020/01/20 14:06:42.269        INFO        http        skipping automatic certificate management because one or more matching certificates are already loaded        {"domain": "ip.hnrk.io", "server_name": "srv1"}
Jan 20 15:06:42 hnrk.io caddy[1432]: 2020/01/20 14:06:42.269        INFO        http        enabling automatic TLS certificate management        {"domains": ["die-reikiquellen.de", "www.harmoniks.de", "www.weather-frogs.de", "harmoniks.de", "www.die-reikiquellen.de", "weather-frogs.de"]}
Jan 20 15:06:42 hnrk.io caddy[1432]: 2020/01/20 14:06:42.270        INFO        http        enabling experimental HTTP/3 listener        {"addr": ":443"}
Jan 20 15:06:42 hnrk.io caddy[1432]: 2020/01/20 14:06:42.271        INFO        autosaved config        {"file": "/root/.config/caddy/autosave.json"}
Jan 20 15:06:42 hnrk.io caddy[1432]: 2020/01/20 14:06:42.271        INFO        serving initial configuration
Jan 20 15:06:43 hnrk.io caddy[1432]: 2020/01/20 15:06:43 http: TLS handshake error from 62.157.168.126:37494: no server TLS configuration available for ClientHello: &{CipherSuites:[14906 4865 4866 4867 49195 49199 49196 49200 52393 52392 49171 49172 156 157 47 53 10] ServerName:www.die-reikiquellen.de SupportedCurves:[10794 29 23 24] SupportedPoints:[0] SignatureSchemes:[1027 2052 1025 1283 2053 1281 2054 1537 513] SupportedProtos:[h2 http/1.1] SupportedVersions:[6682 772 771 770 769] Conn:0xc0000c6720}
Jan 20 15:06:44 hnrk.io caddy[1432]: 2020/01/20 15:06:44 http: TLS handshake error from 62.157.168.126:56285: no server TLS configuration available for ClientHello: &{CipherSuites:[2570 4865 4866 4867 49195 49199 49196 49200 52393 52392 49171 49172 156 157 47 53 10] ServerName:www.die-reikiquellen.de SupportedCurves:[35466 29 23 24] SupportedPoints:[0] SignatureSchemes:[1027 2052 1025 1283 2053 1281 2054 1537 513] SupportedProtos:[h2 http/1.1] SupportedVersions:[56026 772 771 770 769] Conn:0xc000652030}
Jan 20 15:06:45 hnrk.io caddy[1432]: 2020/01/20 15:06:45 http: TLS handshake error from 62.157.168.126:37524: no server TLS configuration available for ClientHello: &{CipherSuites:[51914 4865 4866 4867 49195 49199 49196 49200 52393 52392 49171 49172 156 157 47 53 10] ServerName:www.die-reikiquellen.de SupportedCurves:[19018 29 23 24] SupportedPoints:[0] SignatureSchemes:[1027 2052 1025 1283 2053 1281 2054 1537 513] SupportedProtos:[h2 http/1.1] SupportedVersions:[6682 772 771 770 769] Conn:0xc000652038}
Jan 20 15:20:22 hnrk.io caddy[1432]: 2020/01/20 15:20:22 http: TLS handshake error from 66.249.75.140:58356: no server TLS configuration available for ClientHello: &{CipherSuites:[60138 4865 4866 4867 49195 49199 52393 52392 49196 49200 49161 49171 49162 49172 156 157 47 53 10] ServerName:harmoniks.de SupportedCurves:[56026 29 23 24] SupportedPoints:[0] SignatureSchemes:[1027 2052 1025 1283 2053 1281 2054 1537 513] SupportedProtos:[] SupportedVersions:[14906 772 771 770 769] Conn:0xc00054e258}

Is there something missing when Caddy v2 adapts the config from Caddyfile style?

Question: Is there a directive to configure markdown via Caddyfile style configuration?

Thank you!

balloon · January 22, 2020, 5:37am

Hi HNRK,
Looking at your past posts, the actual Caddyfile appears to have more settings.

Some of them are left in .json, which seems to be bad.

What I checked was the status of the certificate:
harmoniks. de’s certificate has not been issued recently,
but it was only issued last month and has not been renewed since it has not been 30 days old.

Probably there is something wrong with such a long Caddyfile.
Or you need a mechanism to reset the json settings.

I know the implementation of markdown, but it leaves others to help.

HNRK · January 22, 2020, 6:43am

Hi,
thank you for your help!

~~Do you think it could help to flush the stored autosave.json and let Caddy generate a new one?~~
Tried this one, but it doesn’t change the behaviour.

With beta 9 it worked perfectly fine, or did I miss any breaking change for beta13?

balloon · January 22, 2020, 8:03am

I was able to refer to the file a bit more specifically.

From log file:

Jan 20 15:06:43 hnrk.io caddy[1432]: 2020/01/20 15:06:43 http: TLS handshake error from 62.157.168.126:37494: no server TLS configuration available for ClientHello: &{CipherSuites:[14906 4865 4866 4867 49195 49199 49196 49200 52393 52392 49171 49172 156 157 47 53 10] ServerName:www.die-reikiquellen.de SupportedCurves:[10794 29 23 24] SupportedPoints:[0] SignatureSchemes:[1027 2052 1025 1283 2053 1281 2054 1537 513] SupportedProtos:[h2 http/1.1] SupportedVersions:[6682 772 771 770 769] Conn:0xc0000c6720}
Jan 20 15:06:44 hnrk.io caddy[1432]: 2020/01/20 15:06:44 http: TLS handshake error from 62.157.168.126:56285: no server TLS configuration available for ClientHello: &{CipherSuites:[2570 4865 4866 4867 49195 49199 49196 49200 52393 52392 49171 49172 156 157 47 53 10] ServerName:www.die-reikiquellen.de SupportedCurves:[35466 29 23 24] SupportedPoints:[0] SignatureSchemes:[1027 2052 1025 1283 2053 1281 2054 1537 513] SupportedProtos:[h2 http/1.1] SupportedVersions:[56026 772 771 770 769] Conn:0xc000652030}
Jan 20 15:06:45 hnrk.io caddy[1432]: 2020/01/20 15:06:45 http: TLS handshake error from 62.157.168.126:37524: no server TLS configuration available for ClientHello: &{CipherSuites:[51914 4865 4866 4867 49195 49199 49196 49200 52393 52392 49171 49172 156 157 47 53 10] ServerName:www.die-reikiquellen.de SupportedCurves:[19018 29 23 24] SupportedPoints:[0] SignatureSchemes:[1027 2052 1025 1283 2053 1281 2054 1537 513] SupportedProtos:[h2 http/1.1] SupportedVersions:[6682 772 771 770 769] Conn:0xc000652038}
Jan 20 15:20:22 hnrk.io caddy[1432]: 2020/01/20 15:20:22 http: TLS handshake error from 66.249.75.140:58356: no server TLS configuration available for ClientHello: &{CipherSuites:[60138 4865 4866 4867 49195 49199 52393 52392 49196 49200 49161 49171 49162 49172 156 157 47 53 10] ServerName:harmoniks.de SupportedCurves:[56026 29 23 24] SupportedPoints:[0] SignatureSchemes:[1027 2052 1025 1283 2053 1281 2054 1537 513] SupportedProtos:[] SupportedVersions:[14906 772 771 770 769] Conn:0xc00054e258}

no server TLS configuration available was output there.

On the other hand, there is only tls here in the caddyv2.json you pasted.

      "tls":{
         "certificates":{
            "load_files":[
               {
                  "certificate":"/etc/caddy/hnrk.io.crt",
                  "key":"/etc/caddy/hnrk.io.key"
               }
            ]
         }

Could it be a solution hint?

balloon · January 22, 2020, 1:05pm

I happened to add a new domain to Caddyfile and restart Caddy.
Immediately after browsing the web browser, I got ERR_SSL_PROTOCOL_ERROR.
It appeared fine in a few seconds.
Meanwhile, Caddy should have issued a new Let’s Encrypt certificate

Perhaps something went wrong while trying to issue a new or renewed certificate.
For example, you may have reached the issuance limit because you have tried to issue many certificates in the (subdomain).hnrk.io in the past.

Rate Limits - Let's Encrypt

balloon · January 22, 2020, 1:28pm

@matt ,
I want to check one of the Caddy specifications.
Is it correct that Caddy issues / updates to Let’s Encrypt for each domain / subdomain listed in Caddyfile?

If it is correct, this rate limit will be reached when adding more than 11 new domains / subdomains simultaneously.

Rate Limits - Let's Encrypt :

You can create a maximum of 10 Accounts per IP Address per 3 hours.

To circumvent this rate limit, it is necessary to add a function so that multi-domain certificates (SANs) and wildcard certificates (*.example.com) can be issued.

matt · January 22, 2020, 3:06pm

@HNRK What is the full log output? It looks like you posted a few arbitrary lines but we need the full output. There’s important things missing.

What do you mean “configure markdown”? What do you want to do?

Caddy obtains certificates for each domain and subdomain listed in the Caddyfile, when they do not already exist or when they are expiring soon.

No, because only one ACME account is used for all domains (unless a separate email is used for each domain, which is not a good practice).

balloon · January 22, 2020, 3:54pm

@matt Sorry, and thank you. I was convinced that this could only be answered by you.

HNRK · January 22, 2020, 7:10pm

Okay, so first of all, thank you balloon for your time and help!

So let me do this step by step:

I am starting a fresh instance of Caddy v2 beta 13 (the amd64 one attached to the beta 13 release).

I am using the following Caddyfile style configuration:

{
        "experimental_http3": true
}
167.86.123.102:80, [2a02:c207:3004:1207:be:a:bad:babe]:80 {
        redir https://hnrk.io{uri} 301
}
hnrk.io, www.hnrk.io {
        root * /etc/caddy/html
        tls /etc/caddy/hnrk.io.crt /etc/caddy/hnrk.io.key
        encode brotli zstd gzip
        php_fastcgi unix//run/php/php7.3-fpm.sock
        try_files {path} /index.php?{query}
        file_server /md* browse
        file_server
}
weather-frogs.de, www.weather-frogs.de {
        tls samplemail@hnrk.io
        root * /etc/caddy/html/weather-frogs.de
        encode brotli zstd gzip
        php_fastcgi unix//run/php/php7.3-fpm.sock
        try_files {path} /index.php?{query}
        file_server
}
die-reikiquellen.de, www.die-reikiquellen.de {
        tls samplemail@hnrk.io
        root * /etc/caddy/html/die-reikiquellen.de
        encode brotli zstd gzip
        php_fastcgi unix//run/php/php7.3-fpm.sock
        try_files {path} /index.php?{query}
        file_server
}
harmoniks.de, www.harmoniks.de {
        tls samplemail@hnrk.io
        root * /etc/caddy/html/harmoniks.de
        encode brotli zstd gzip
        php_fastcgi unix//run/php/php7.3-fpm.sock
        try_files {path} /index.php?{query}
        file_server
}
ip.hnrk.io {
        root * /etc/caddy/html/myip
        tls /etc/caddy/hnrk.io.crt /etc/caddy/hnrk.io.key
        encode brotli zstd gzip
        php_fastcgi unix//run/php/php7.3-fpm.sock
        try_files {path} /index.php?{query}
        file_server
}
sql.hnrk.io {
        root * /etc/caddy/html/pma
        tls /etc/caddy/hnrk.io.crt /etc/caddy/hnrk.io.key
        encode brotli zstd gzip
        php_fastcgi unix//run/php/php7.3-fpm.sock
        try_files {path} /index.php?{query}
        file_server
}
postfix.hnrk.io {
        root * /etc/caddy/html/pfa/public
        tls /etc/caddy/hnrk.io.crt /etc/caddy/hnrk.io.key
        encode brotli zstd gzip
        php_fastcgi unix//run/php/php7.3-fpm.sock
        try_files {path} /index.php?{query}
        file_server
}
draw.hnrk.io {
        root * /etc/caddy/html/draw
        tls /etc/caddy/hnrk.io.crt /etc/caddy/hnrk.io.key
        encode brotli zstd gzip
        php_fastcgi unix//run/php/php7.3-fpm.sock
        try_files {path} /index.php?{query}
        file_server
}
mail.hnrk.io {
        root * /etc/caddy/html/rl
        tls /etc/caddy/hnrk.io.crt /etc/caddy/hnrk.io.key
        encode brotli zstd gzip
        php_fastcgi unix//run/php/php7.3-fpm.sock
        try_files {path} /index.php?{query}
        file_server * {
                        hide /data
        }
}
fotos.hnrk.io {
        root * /etc/caddy/html/fotos
        tls /etc/caddy/hnrk.io.crt /etc/caddy/hnrk.io.key
        encode brotli zstd gzip
        php_fastcgi unix//run/php/php7.3-fpm.sock
        try_files {path} /index.php?{query}
        file_server
}
proxy.hnrk.io {
        root * /etc/caddy/html/proxy
        tls /etc/caddy/hnrk.io.crt /etc/caddy/hnrk.io.key
        encode brotli zstd gzip
        php_fastcgi unix//run/php/php7.3-fpm.sock
        try_files {path} /index.php?{query}
        file_server
}
skip.hnrk.io {
        root * /etc/caddy/html/skip
        tls /etc/caddy/hnrk.io.crt /etc/caddy/hnrk.io.key
        encode brotli zstd gzip
        php_fastcgi unix//run/php/php7.3-fpm.sock
        try_files {path} /index.php?{query}
        file_server
}
zeit.hnrk.io {
        root * /etc/caddy/html/time
        tls /etc/caddy/hnrk.io.crt /etc/caddy/hnrk.io.key
        encode brotli zstd gzip
        php_fastcgi unix//run/php/php7.3-fpm.sock
        try_files {path} /index.php?{query}
        file_server
}
frequencies.hnrk.io {
        root * /etc/caddy/html/niviuk.free.fr/niviuk.free.fr
        tls /etc/caddy/hnrk.io.crt /etc/caddy/hnrk.io.key
        encode brotli zstd gzip
        php_fastcgi unix//run/php/php7.3-fpm.sock
        try_files {path} /index.php?{query}
        file_server
}
dns.hnrk.io {
        root * /etc/caddy/html/null
        reverse_proxy * localhost:10001
        tls /etc/caddy/hnrk.io.crt /etc/caddy/hnrk.io.key
        encode brotli zstd gzip
        php_fastcgi unix//run/php/php7.3-fpm.sock
        try_files {path} /index.php?{query}
        file_server
}

Notice the three domains:

weather-frogs.de (and www subdomain)
die-reikiquellen.de (and www subdomain)
harmoniks.de (and www subdomain)

These should receive LE Certificates automatically from Caddy v2, because the other domains have Cert + Keyfile already given as path (/etc/caddy/hnrk.io.[crt|key])

Now when I try to navigate to these three domains via my browser, unfortunately I get this error message in Brave:

These are the logs that Caddy v2 spits out when navigating to one of them:

Jan 22 19:46:10 hnrk.io systemd[1]: Started Caddy v2.
Jan 22 19:46:10 hnrk.io caddy[11773]: 2020/01/22 18:46:10.638        INFO        using provided configuration        {"config_file": "/etc/v2caddy/Caddyfile", "config_adapter": "caddyfile"}
Jan 22 19:46:10 hnrk.io caddy[11773]: 2020/01/22 18:46:10.657        INFO        admin        admin endpoint started        {"address": "localhost:2019", "enforce_origin": false, "origins": ["localhost:2019"]}
Jan 22 19:46:10 hnrk.io caddy[11773]: 2020/01/22 18:46:10.658        INFO        http        enabling automatic HTTP->HTTPS redirects        {"server_name": "srv0"}
Jan 22 19:46:10 hnrk.io caddy[11773]: 2020/01/22 18:46:10.659        INFO        http        server is listening only on the HTTP port, so no automatic HTTPS will be applied to this server        {"server_name": "srv1", "http_port": 80}
Jan 22 19:46:10 hnrk.io caddy[11773]: 2020/01/22 18:46:10.659        WARN        http        server is listening on same interface as redirects, so automatic HTTP->HTTPS redirects might be overridden by your own configuration        {"server_name": "srv1", "interface": ":80"}
Jan 22 19:46:10 hnrk.io caddy[11773]: 2020/01/22 19:46:10 [INFO][cache:0xc0005bc190] Started certificate maintenance routine
Jan 22 19:46:10 hnrk.io caddy[11773]: 2020/01/22 18:46:10.699        INFO        tls        cleaned up storage units
Jan 22 19:46:10 hnrk.io caddy[11773]: 2020/01/22 18:46:10.699        INFO        http        skipping automatic certificate management because one or more matching certificates are already loaded        {"domain": "draw.hnrk.io", "server_name": "srv0"}
Jan 22 19:46:10 hnrk.io caddy[11773]: 2020/01/22 18:46:10.699        INFO        http        skipping automatic certificate management because one or more matching certificates are already loaded        {"domain": "skip.hnrk.io", "server_name": "srv0"}
Jan 22 19:46:10 hnrk.io caddy[11773]: 2020/01/22 18:46:10.699        INFO        http        skipping automatic certificate management because one or more matching certificates are already loaded        {"domain": "sql.hnrk.io", "server_name": "srv0"}
Jan 22 19:46:10 hnrk.io caddy[11773]: 2020/01/22 18:46:10.699        INFO        http        skipping automatic certificate management because one or more matching certificates are already loaded        {"domain": "frequencies.hnrk.io", "server_name": "srv0"}
Jan 22 19:46:10 hnrk.io caddy[11773]: 2020/01/22 18:46:10.699        INFO        http        skipping automatic certificate management because one or more matching certificates are already loaded        {"domain": "fotos.hnrk.io", "server_name": "srv0"}
Jan 22 19:46:10 hnrk.io caddy[11773]: 2020/01/22 18:46:10.699        INFO        http        skipping automatic certificate management because one or more matching certificates are already loaded        {"domain": "proxy.hnrk.io", "server_name": "srv0"}
Jan 22 19:46:10 hnrk.io caddy[11773]: 2020/01/22 18:46:10.699        INFO        http        skipping automatic certificate management because one or more matching certificates are already loaded        {"domain": "ip.hnrk.io", "server_name": "srv0"}
Jan 22 19:46:10 hnrk.io caddy[11773]: 2020/01/22 18:46:10.699        INFO        http        skipping automatic certificate management because one or more matching certificates are already loaded        {"domain": "zeit.hnrk.io", "server_name": "srv0"}
Jan 22 19:46:10 hnrk.io caddy[11773]: 2020/01/22 18:46:10.699        INFO        http        skipping automatic certificate management because one or more matching certificates are already loaded        {"domain": "postfix.hnrk.io", "server_name": "srv0"}
Jan 22 19:46:10 hnrk.io caddy[11773]: 2020/01/22 18:46:10.699        INFO        http        skipping automatic certificate management because one or more matching certificates are already loaded        {"domain": "mail.hnrk.io", "server_name": "srv0"}
Jan 22 19:46:10 hnrk.io caddy[11773]: 2020/01/22 18:46:10.699        INFO        http        skipping automatic certificate management because one or more matching certificates are already loaded        {"domain": "hnrk.io", "server_name": "srv0"}
Jan 22 19:46:10 hnrk.io caddy[11773]: 2020/01/22 18:46:10.699        INFO        http        skipping automatic certificate management because one or more matching certificates are already loaded        {"domain": "www.hnrk.io", "server_name": "srv0"}
Jan 22 19:46:10 hnrk.io caddy[11773]: 2020/01/22 18:46:10.699        INFO        http        skipping automatic certificate management because one or more matching certificates are already loaded        {"domain": "dns.hnrk.io", "server_name": "srv0"}
Jan 22 19:46:10 hnrk.io caddy[11773]: 2020/01/22 18:46:10.699        INFO        http        enabling automatic TLS certificate management        {"domains": ["www.die-reikiquellen.de", "www.weather-frogs.de", "weather-frogs.de", "www.harmoniks.de", "die-reikiquellen.de", "harmoniks.de"]}
Jan 22 19:46:10 hnrk.io caddy[11773]: 2020/01/22 18:46:10.703        INFO        http        enabling experimental HTTP/3 listener        {"addr": ":443"}
Jan 22 19:46:10 hnrk.io caddy[11773]: 2020/01/22 18:46:10.704        INFO        autosaved config        {"file": "/root/.config/caddy/autosave.json"}
Jan 22 19:46:10 hnrk.io caddy[11773]: 2020/01/22 18:46:10.705        INFO        serving initial configuration
Jan 22 19:46:31 hnrk.io caddy[11773]: 2020/01/22 19:46:31 http: TLS handshake error from 2.202.73.167:17276: no server TLS configuration available for ClientHello: &{CipherSuites:[60138 4865 4866 4867 49195 49199 49196 49200 52393 52392 49171 49172 156 157 47 53 10] ServerName:harmoniks.de SupportedCurves:[2570 29 23 24] SupportedPoints:[0] SignatureSchemes:[1027 2052 1025 1283 2053 1281 2054 1537 513] SupportedProtos:[h2 http/1.1] SupportedVersions:[23130 772 771 770 769] Conn:0xc00028c0f8}
Jan 22 19:46:31 hnrk.io caddy[11773]: 2020/01/22 19:46:31 http: TLS handshake error from 2.202.73.167:17277: no server TLS configuration available for ClientHello: &{CipherSuites:[60138 4865 4866 4867 49195 49199 49196 49200 52393 52392 49171 49172 156 157 47 53 10] ServerName:harmoniks.de SupportedCurves:[47802 29 23 24] SupportedPoints:[0] SignatureSchemes:[1027 2052 1025 1283 2053 1281 2054 1537 513] SupportedProtos:[h2 http/1.1] SupportedVersions:[43690 772 771 770 769] Conn:0xc0000104a0}
Jan 22 19:46:35 hnrk.io caddy[11773]: 2020/01/22 19:46:35 http: TLS handshake error from 2.202.73.167:17279: no server TLS configuration available for ClientHello: &{CipherSuites:[23130 4865 4866 4867 49195 49199 49196 49200 52393 52392 49171 49172 156 157 47 53 10] ServerName:die-reikiquellen.de SupportedCurves:[6682 29 23 24] SupportedPoints:[0] SignatureSchemes:[1027 2052 1025 1283 2053 1281 2054 1537 513] SupportedProtos:[h2 http/1.1] SupportedVersions:[39578 772 771 770 769] Conn:0xc000122220}

So the question is: Did I make a configuration mistake or is this a bug in Caddy v2?
Again, thank you guys for your time!

matt · January 22, 2020, 7:56pm

@balloon No worries, thanks for your help! It’s great when others get involved.

@HNRK Thanks, that’s much clearer. Have you tried fiddling with anything (e.g. turning of HTTP/3) or removing sites from the config? What is the most minimal steps I would need to perform to reproduce the issue?

You might also try converting the Caddyfile to JSON, then set manage_sync in the automation policy to true: JSON Config Structure - Caddy Documentation – that should stop your server from continuing until it has something to say about certificates.

Let me know how it goes!

matt · January 22, 2020, 10:59pm

Hmm, I think your use case touches on a TODO in certmagic, let me see what I can do about it. In the meantime, any debugging updates from your end would be helpful! Especially the manage_sync stuff.

Edit: @HNRK would you be able to build Caddy beta 13 against this commit of CertMagic: Limit one new client at a time · caddyserver/certmagic@7f07778 · GitHub

balloon · January 23, 2020, 8:47am

The entry for tls looks strange:

HNRK · January 23, 2020, 10:00am

@balloon I cannot see any tls at your link, do you mean tls (Caddyfile directive) — Caddy Documentation?

matt · January 23, 2020, 3:12pm

Thanks for the details – can you also help by posting the full log output? From the very moment you start the process all the way until it is closed…

Also, are you using the API / caddy reload for config changes, or are you stopping and restarting the process entirely?

HNRK · January 23, 2020, 3:22pm

Log output from my last mentioned config setting where the LE domains fail (hnrk.io + three automatically LE certified domains):

Jan 23 16:17:32 hnrk.io systemd[1]: Stopped Caddy v2.
Jan 23 16:18:41 hnrk.io systemd[1]: Started Caddy v2.
Jan 23 16:18:41 hnrk.io caddy[16928]: 2020/01/23 15:18:41.128        INFO        using provided configuration        {"config_file": "/etc/v2caddy/Caddyfile", "config_adapter": "caddyfile"}
Jan 23 16:18:41 hnrk.io caddy[16928]: 2020/01/23 15:18:41.138        INFO        admin        admin endpoint started        {"address": "localhost:2019", "enforce_origin": false, "origins": ["localhost:2019"]}
Jan 23 16:18:41 hnrk.io caddy[16928]: 2020/01/23 15:18:41.138        INFO        http        enabling automatic HTTP->HTTPS redirects        {"server_name": "srv0"}
Jan 23 16:18:41 hnrk.io caddy[16928]: 2020/01/23 16:18:41 [INFO][cache:0xc0004cccd0] Started certificate maintenance routine
Jan 23 16:18:41 hnrk.io caddy[16928]: 2020/01/23 15:18:41.156        INFO        http        skipping automatic certificate management because one or more matching certificates are already loaded        {"domain": "hnrk.io", "server_name": "srv0"}
Jan 23 16:18:41 hnrk.io caddy[16928]: 2020/01/23 15:18:41.156        INFO        http        skipping automatic certificate management because one or more matching certificates are already loaded        {"domain": "www.hnrk.io", "server_name": "srv0"}
Jan 23 16:18:41 hnrk.io caddy[16928]: 2020/01/23 15:18:41.156        INFO        http        enabling automatic TLS certificate management        {"domains": ["www.weather-frogs.de", "harmoniks.de", "www.harmoniks.de", "die-reikiquellen.de", "www.die-reikiquellen.de", "weather-frogs.de"]}
Jan 23 16:18:41 hnrk.io caddy[16928]: 2020/01/23 15:18:41.177        INFO        tls        cleaned up storage units
Jan 23 16:18:41 hnrk.io caddy[16928]: 2020/01/23 15:18:41.177        INFO        autosaved config        {"file": "/root/.config/caddy/autosave.json"}
Jan 23 16:18:41 hnrk.io caddy[16928]: 2020/01/23 15:18:41.177        INFO        serving initial configuration
Jan 23 16:18:53 hnrk.io caddy[16928]: 2020/01/23 16:18:53 http: TLS handshake error from 2.202.73.167:18209: no server TLS configuration available for ClientHello: &{CipherSuites:[19018 4865 4866 4867 49195 49199 49196 49200 52393 52392 49171 49172 156 157 47 53 10] ServerName:harmoniks.de SupportedCurves:[51914 29 23 24] SupportedPoints:[0] SignatureSchemes:[1027 2052 1025 1283 2053 1281 2054 1537 513] SupportedProtos:[h2 http/1.1] SupportedVersions:[56026 772 771 770 769] Conn:0xc0008850a0}
Jan 23 16:18:54 hnrk.io caddy[16928]: 2020/01/23 16:18:54 http: TLS handshake error from 2.202.73.167:18210: no server TLS configuration available for ClientHello: &{CipherSuites:[31354 4865 4866 4867 49195 49199 49196 49200 52393 52392 49171 49172 156 157 47 53 10] ServerName:harmoniks.de SupportedCurves:[35466 29 23 24] SupportedPoints:[0] SignatureSchemes:[1027 2052 1025 1283 2053 1281 2054 1537 513] SupportedProtos:[h2 http/1.1] SupportedVersions:[43690 772 771 770 769] Conn:0xc00059a0a0}
Jan 23 16:18:54 hnrk.io caddy[16928]: 2020/01/23 16:18:54 http: TLS handshake error from 2.202.73.167:18211: no server TLS configuration available for ClientHello: &{CipherSuites:[6682 4865 4866 4867 49195 49199 49196 49200 52393 52392 49171 49172 156 157 47 53 10] ServerName:harmoniks.de SupportedCurves:[6682 29 23 24] SupportedPoints:[0] SignatureSchemes:[1027 2052 1025 1283 2053 1281 2054 1537 513] SupportedProtos:[h2 http/1.1] SupportedVersions:[47802 772 771 770 769] Conn:0xc00059a0a8}
Jan 23 16:18:55 hnrk.io caddy[16928]: 2020/01/23 16:18:55 http: TLS handshake error from 2.202.73.167:18212: no server TLS configuration available for ClientHello: &{CipherSuites:[14906 4865 4866 4867 49195 49199 49196 49200 52393 52392 49171 49172 156 157 47 53 10] ServerName:harmoniks.de SupportedCurves:[2570 29 23 24] SupportedPoints:[0] SignatureSchemes:[1027 2052 1025 1283 2053 1281 2054 1537 513] SupportedProtos:[h2 http/1.1] SupportedVersions:[14906 772 771 770 769] Conn:0xc0008850a8}
Jan 23 16:18:55 hnrk.io caddy[16928]: 2020/01/23 16:18:55 http: TLS handshake error from 2.202.73.167:18214: no server TLS configuration available for ClientHello: &{CipherSuites:[60138 4865 4866 4867 49195 49199 49196 49200 52393 52392 49171 49172 156 157 47 53 10] ServerName:harmoniks.de SupportedCurves:[10794 29 23 24] SupportedPoints:[0] SignatureSchemes:[1027 2052 1025 1283 2053 1281 2054 1537 513] SupportedProtos:[h2 http/1.1] SupportedVersions:[2570 772 771 770 769] Conn:0xc0008850b0}
Jan 23 16:18:55 hnrk.io caddy[16928]: 2020/01/23 16:18:55 http: TLS handshake error from 2.202.73.167:18213: no server TLS configuration available for ClientHello: &{CipherSuites:[39578 4865 4866 4867 49195 49199 49196 49200 52393 52392 49171 49172 156 157 47 53 10] ServerName:harmoniks.de SupportedCurves:[31354 29 23 24] SupportedPoints:[0] SignatureSchemes:[1027 2052 1025 1283 2053 1281 2054 1537 513] SupportedProtos:[h2 http/1.1] SupportedVersions:[10794 772 771 770 769] Conn:0xc00059a1e8}
Jan 23 16:18:55 hnrk.io caddy[16928]: 2020/01/23 16:18:55 http: TLS handshake error from 2.202.73.167:18216: no server TLS configuration available for ClientHello: &{CipherSuites:[35466 4865 4866 4867 49195 49199 49196 49200 52393 52392 49171 49172 156 157 47 53 10] ServerName:harmoniks.de SupportedCurves:[43690 29 23 24] SupportedPoints:[0] SignatureSchemes:[1027 2052 1025 1283 2053 1281 2054 1537 513] SupportedProtos:[h2 http/1.1] SupportedVersions:[14906 772 771 770 769] Conn:0xc0008850b8}

I completely stop and start the process via systemd file.

matt · January 23, 2020, 3:23pm

Thanks; do you have anything longer than 10 seconds worth of logs? At least a minute, maybe?

matt · January 23, 2020, 3:24pm

For troubleshooting, I think it’ll be helpful to eliminate as many variables as possible. Can you please try just running it on the command line directly without systemd?

HNRK · January 23, 2020, 3:24pm

EDIT: Yeah, sure: https://gist.github.com/Henrocker/172bc21eeb78dbc30785cf05fe9ce547

matt · January 23, 2020, 3:28pm

Thanks! I’ll try to look into this more. Might be a while though as I recover from being sick

HNRK · January 23, 2020, 3:30pm

Health is more important, have a rest !

Here is the logoutput when I start caddy v2 via command:

./caddy run --config /etc/v2caddy/Caddyfile --adapter caddyfile
2020/01/23 15:27:07.648 INFO    using provided configuration    {"config_file": "/etc/v2caddy/Caddyfile", "config_adapter": "caddyfile"}
2020/01/23 15:27:07.655 INFO    admin   admin endpoint started  {"address": "localhost:2019", "enforce_origin": false, "origins": ["localhost:2019"]}
2020/01/23 15:27:07.655 INFO    http    enabling automatic HTTP->HTTPS redirects        {"server_name": "srv0"}
2020/01/23 16:27:07 [INFO][cache:0xc00070a000] Started certificate maintenance routine
2020/01/23 15:27:07.670 INFO    http    skipping automatic certificate management because one or more matching certificates are already loaded  {"domain": "www.hnrk.io", "server_name": "srv0"}
2020/01/23 15:27:07.670 INFO    http    skipping automatic certificate management because one or more matching certificates are already loaded  {"domain": "hnrk.io", "server_name": "srv0"}
2020/01/23 15:27:07.670 INFO    http    enabling automatic TLS certificate management   {"domains": ["die-reikiquellen.de", "www.die-reikiquellen.de", "weather-frogs.de", "www.weather-frogs.de", "harmoniks.de", "www.harmoniks.de"]}
2020/01/23 15:27:07.675 INFO    tls     cleaned up storage units
2020/01/23 15:27:07.676 INFO    autosaved config        {"file": "/root/.config/caddy/autosave.json"}
2020/01/23 15:27:07.676 INFO    serving initial configuration
2020/01/23 16:27:46 http: TLS handshake error from 2.202.73.167:18223: no server TLS configuration available for ClientHello: &{CipherSuites:[2570 4865 4866 4867 49195 49199 49196 49200 52393 52392 49171 49172 156 157 47 53 10] ServerName:mail.hnrk.io SupportedCurves:[47802 29 23 24] SupportedPoints:[0] SignatureSchemes:[1027 2052 1025 1283 2053 1281 2054 1537 513] SupportedProtos:[h2 http/1.1] SupportedVersions:[2570 772 771 770 769] Conn:0xc0008fe158}
2020/01/23 16:28:46 http: TLS handshake error from 2.202.73.167:18226: no server TLS configuration available for ClientHello: &{CipherSuites:[51914 4865 4866 4867 49195 49199 49196 49200 52393 52392 49171 49172 156 157 47 53 10] ServerName:harmoniks.de SupportedCurves:[10794 29 23 24] SupportedPoints:[0] SignatureSchemes:[1027 2052 1025 1283 2053 1281 2054 1537 513] SupportedProtos:[h2 http/1.1] SupportedVersions:[23130 772 771 770 769] Conn:0xc000552010}
2020/01/23 16:28:53 http: TLS handshake error from 2.202.73.167:18227: no server TLS configuration available for ClientHello: &{CipherSuites:[47802 4865 4866 4867 49195 49199 49196 49200 52393 52392 49171 49172 156 157 47 53 10] ServerName:die-reikiquellen.de SupportedCurves:[10794 29 23 24] SupportedPoints:[0] SignatureSchemes:[1027 2052 1025 1283 2053 1281 2054 1537 513] SupportedProtos:[h2 http/1.1] SupportedVersions:[19018 772 771 770 769] Conn:0xc0000c8078}
2020/01/23 16:28:53 http: TLS handshake error from 2.202.73.167:18228: no server TLS configuration available for ClientHello: &{CipherSuites:[47802 4865 4866 4867 49195 49199 49196 49200 52393 52392 49171 49172 156 157 47 53 10] ServerName:die-reikiquellen.de SupportedCurves:[31354 29 23 24] SupportedPoints:[0] SignatureSchemes:[1027 2052 1025 1283 2053 1281 2054 1537 513] SupportedProtos:[h2 http/1.1] SupportedVersions:[56026 772 771 770 769] Conn:0xc000552020}
2020/01/23 16:28:59 http: TLS handshake error from 2.202.73.167:18231: no server TLS configuration available for ClientHello: &{CipherSuites:[47802 4865 4866 4867 49195 49199 49196 49200 52393 52392 49171 49172 156 157 47 53 10] ServerName:weather-frogs.de SupportedCurves:[47802 29 23 24] SupportedPoints:[0] SignatureSchemes:[1027 2052 1025 1283 2053 1281 2054 1537 513] SupportedProtos:[h2 http/1.1] SupportedVersions:[6682 772 771 770 769] Conn:0xc000552030}
2020/01/23 16:28:59 http: TLS handshake error from 2.202.73.167:18232: no server TLS configuration available for ClientHello: &{CipherSuites:[23130 4865 4866 4867 49195 49199 49196 49200 52393 52392 49171 49172 156 157 47 53 10] ServerName:weather-frogs.de SupportedCurves:[27242 29 23 24] SupportedPoints:[0] SignatureSchemes:[1027 2052 1025 1283 2053 1281 2054 1537 513] SupportedProtos:[h2 http/1.1] SupportedVersions:[14906 772 771 770 769] Conn:0xc000119038}