1. The problem I’m having:
with a restart of caddy using supervisord (ie. stop and then start), I’ve noticed that it takes quite awhile for caddy to be able to start serving HTTPS, as for a very noticeable period of time, I keep getting “internal error” from the clients, with “no certificate matching TLS ClientHello” in the Caddy logs.
Seems it’s relate to OSCP stapling that first need to happen, so, the question becomes: what is being done before Caddy will server first page succesfully after a restart (no new certificate, using already saved versions)
How could I make that faster?
Or how do I (in a HA/load balancer setup) confirm/check all the certificates is active before handing traffic to the caddy instance?
2. Error messages and/or full log output:
{"level":"debug","ts":1725446554.1754434,"logger":"tls.handshake","msg":"no certificate matching TLS ClientHello","remote_ip":"102.212.62.253","remote_port":"65478","server_name":"ae-tracs-multiplexor01.agileworkscloud.com","remote":"102.212.62.253:65478","identifier":"ae-tracs-multiplexor01.agileworkscloud.com","cipher_suites":[4867,4866,4865,52393,52392,52394,49200,49196,49192,49188,49172,49162,159,107,57,65413,196,136,129,157,61,53,192,132,49199,49195,49191,49187,49171,49161,158,103,51,190,69,156,60,47,186,65,49169,49159,5,4,49170,49160,22,10,255],"cert_cache_fill":0,"load_or_obtain_if_necessary":true,"on_demand":false}
* Connected to ae-tracs-multiplexor01.agileworkscloud.com (102.212.61.37) port 443
* ALPN: curl offers h2,http/1.1
* (304) (OUT), TLS handshake, Client hello (1):
* CAfile: /etc/ssl/cert.pem
* CApath: none
* LibreSSL/3.3.6: error:1404B438:SSL routines:ST_CONNECT:tlsv1 alert internal error
* Closing connection
curl: (35) LibreSSL/3.3.6: error:1404B438:SSL routines:ST_CONNECT:tlsv1 alert internal error
3. Caddy version:
v2.8.4 => github.com/caddyserver/caddy/v2@v2.8.4 h1:q3pe0wpBj1OcHFZ3n/1nl4V4bxBrYoSoab7rL9BMYNk=
Observed same with 2.7.4 before I upgrade now to 2.8.4
4. How I installed and ran Caddy:
xcaddy build with these modules:
Standard modules: 121
dns.providers.acmedns
dns.providers.lego_deprecated
exec
http.handlers.exec
http.handlers.named_route
http.handlers.s3browser
http.handlers.s3proxy
http.handlers.teapot
named_routes
Non-standard modules: 9
Unknown modules: 0
using supervisor on Devuan
a. System environment:
Devuan Chimera
LXC inside ProxMox
b. Command:
supervisorctl restart caddy
c. Service/unit/compose file:
d. My complete Caddy config:
{
log {
level debug
format json
output file /var/log/caddy/global.log {
roll_size 100M
roll_keep_for 60d
}
}
default_sni ae-tracs-multiplexor01.agileworkscloud.com
}
ae-tracs-multiplexor01.agileworkscloud.com localhost {
respond "Hello, world! PLease credit us alot :)<br>
"
}
localhost:2016 {
respond "Goodbye, world! Hello troubles this far\n"
}
##HH tracsdev02.agileworkscloud.com tracsdev.agileworkscloud.com {
##HH reverse_proxy {
##HH to https://agw-ae-tracs-mxdev02.qquark.in
##HH transport http {
##HH tls_insecure_skip_verify
##HH }
##HH header_up -X-Forwarded-For
##HH # header_up X-Forwarded-Proto {scheme}
##HH # header_up X-Forwarded-Host {host}
##HH header_up Host {host}
##HH }
##HH }
##h
crush-tracsdev02.agileworkscloud.com {
reverse_proxy {
to http://agw-ae-tracs-dev-crusher01.qquark.in:9090
}
}
##h
##h###
##h
##h
tracs-sam-dev02.agileworkscloud.com {
# reverse_proxy {
# to http://agw-ae-tracs-sea-dev01.qquark.in:8888
# to http://agw-ae-tracs-sea-dev02.qquark.in:8888
# to http://agw-ae-tracs-sea-dev03.qquark.in:8888
# to http://agw-ae-tracs-sea-dev04.qquark.in:8888
# to http://agw-ae-tracs-sea-dev05.qquark.in:8888
# to http://agw-ae-tracs-sea-dev06.qquark.in:8888
# health_uri /minio/health/live
# health_port 9000
# transport http {
# versions 1.1
# }
# header_up -X-Forwarded-For
# header_up X-Forwarded-Proto {scheme}
# header_up X-Forwarded-Host {host}
# header_up Host {host}
# }
# tls {
# dns acmedns /etc/acmedns/clientstorage.json
# propagation_timeout -1
# propagation_delay 5s
# }
log {
format json
output file /var/log/caddy/caddy-ae-dev02-sam.log {
}
level debug
}
}
##h
##hagw-ae-dev02-sa.qquark.in {
##h reverse_proxy {
##h to http://agw-ae-tracs-sea-dev01.qquark.in:9000
##h to http://agw-ae-tracs-sea-dev02.qquark.in:9000
##h to http://agw-ae-tracs-sea-dev03.qquark.in:9000
##h to http://agw-ae-tracs-sea-dev04.qquark.in:9000
##h to http://agw-ae-tracs-sea-dev05.qquark.in:9000
##h to http://agw-ae-tracs-sea-dev06.qquark.in:9000
##h health_uri /minio/health/live
##h transport http {
##h versions 1.1
##h }
##h header_up -X-Forwarded-For
##h header_up X-Forwarded-Proto {scheme}
##h header_up X-Forwarded-Host {host}
##h header_up Host {host}
##h }
##h tls {
##h dns acmedns /etc/acmedns/clientstorage.json
##h propagation_timeout -1
##h propagation_delay 5s
##h }
##h log {
##h format json
##h output file /var/log/caddy/caddy-ae-dev02-sa.log {
##h }
##h level debug
##h }
##h}