I don’t see any errors in the logs either. Did you add the site address back? Did you try to make some requests while Caddy was running and logging was enabled?
Hi.
Running
curl -v https://liberation.scot:1443 --resolve liberation.scot:1443:127.0.0.1
leads to NO additional log entries in the docker logs “xxxxxxx” from my Caddy container, like its not being connected to at all
What does curl output when you do that?
curl -v https://liberation.scot:1443 --resolve liberation.scot:1443:127.0.0.1
- Added liberation.scot:1443:127.0.0.1 to DNS cache
- Hostname liberation.scot was found in DNS cache
- Trying 127.0.0.1:1443…
- Connected to liberation.scot (127.0.0.1) port 1443 (#0)
- ALPN, offering h2
- ALPN, offering http/1.1
- CAfile: /etc/ssl/certs/ca-certificates.crt
- CApath: /etc/ssl/certs
- TLSv1.0 (OUT), TLS header, Certificate Status (22):
- TLSv1.3 (OUT), TLS handshake, Client hello (1):
- TLSv1.0 (OUT), TLS header, Unknown (21):
- TLSv1.3 (OUT), TLS alert, decode error (562):
- error:0A000126:SSL routines::unexpected eof while reading
- Closing connection 0
curl: (35) error:0A000126:SSL routines::unexpected eof while reading
root@liberation:~#
root@liberation:~#
ncat -v -C 127.0.0.1 1443
Ncat: Version 7.80 ( Ncat - Netcat for the 21st Century )
Ncat: Connected to 127.0.0.1:1443.
Like theres nobody home
I’m still concerned there is a generic issue with OpenSSL on ubuntu22.04.
Has Caddy been tested with this specific version of SSL being used by its calling clients?
openssl version -a
OpenSSL 3.0.2 15 Mar 2022 (Library: OpenSSL 3.0.2 15 Mar 2022)
built on: Tue Aug 20 17:27:32 2024 UTC
platform: debian-arm64
options: bn(64,64)
compiler: gcc -fPIC -pthread -Wa,–noexecstack -Wall -Wa,–noexecstack -g -O2 -ffile-prefix-map=/build/openssl-BW0rDL/openssl-3.0.2=. -flto=auto -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -DOPENSSL_TLS_SECURITY_LEVEL=2 -DOPENSSL_USE_NODELETE -DOPENSSL_PIC -DOPENSSL_BUILDING_OPENSSL -DNDEBUG -Wdate-time -D_FORTIFY_SOURCE=2
OPENSSLDIR: “/usr/lib/ssl”
ENGINESDIR: “/usr/lib/aarch64-linux-gnu/engines-3”
MODULESDIR: “/usr/lib/aarch64-linux-gnu/ossl-modules”
Seeding source: os-specific
CPUINFO: OPENSSL_armcap=0xbf
Are there known issues between this (OLD) OpenSSL client side that Ubuntu uses on EC2 and What TLS library is Caddy using?
It really looks like the “handshake” between GOs lib and OpenSSL (in this case) is erroring out.
It should be easy to reproduce.
OpenSSL 3.0 and Go:
- Go 1.18 and above: Go 1.18 introduced the
crypto/tlspackage that supports OpenSSL 3.0.
You’re running Caddy inside a container, right?
Is this still relevant?
Wouldn’t that mean connecting to localhost:1443 would be connecting to port 443 inside the container…? But if you’re running Caddy on port 1443 inside the container, you’d need a map 1443:1443 instead…?
MY FAULT.
docker-compose had 1443:443 not 1443:1443
THANKYOU!
Now I get lots of server side errors but at least its talking.
root@liberation:/data/invenio-rdm-starter# docker logs cf58f15645c6
{“level”:“info”,“ts”:1732385075.475556,“msg”:“using config from file”,“file”:“/etc/caddy/Caddyfile”}
{“level”:“info”,“ts”:1732385075.4858327,“msg”:“adapted config to JSON”,“adapter”:“caddyfile”}
{“level”:“warn”,“ts”:1732385075.4878175,“msg”:“Caddyfile input is not formatted; run ‘caddy fmt --overwrite’ to fix inconsistencies”,“adapter”:“caddyfile”,“file”:“/etc/caddy/Caddyfile”,“line”:2}
{“level”:“info”,“ts”:1732385075.49264,“logger”:“admin”,“msg”:“admin endpoint started”,“address”:“localhost:2019”,“enforce_origin”:false,“origins”:[“//localhost:2019”,“//[::1]:2019”,“//127.0.0.1:2019”]}
{“level”:“info”,“ts”:1732385075.4933097,“logger”:“tls.cache.maintenance”,“msg”:“started background certificate maintenance”,“cache”:“0x400021d900”}
{“level”:“debug”,“ts”:1732385075.5009623,“logger”:“events”,“msg”:“event”,“name”:“cached_unmanaged_cert”,“id”:“d77567e5-ae2e-4890-882d-df7fa9a0cda5”,“origin”:“tls”,“data”:{“sans”:[“.liberation.scot",“liberation.scot”]}}
{“level”:“debug”,“ts”:1732385075.501,“logger”:“tls.cache”,“msg”:“added certificate to cache”,“subjects”:[".liberation.scot”,“liberation.scot”],“expiration”:1763389967,“managed”:false,“issuer_key”:“”,“hash”:“11814135d9b36b3b021ce09dc68455486c1ba9b9e30e83998a7e69271d2fdb41”,“cache_size”:1,“cache_capacity”:10000}
{“level”:“info”,“ts”:1732385075.5010922,“logger”:“http.auto_https”,“msg”:“automatic HTTPS is completely disabled for server”,“server_name”:“srv0”}
{“level”:“debug”,“ts”:1732385075.501126,“logger”:“http.auto_https”,“msg”:“adjusted config”,“tls”:{“automation”:{“policies”:[{}]}},“http”:{“http_port”:7080,“https_port”:1443,“servers”:{“srv0”:{“listen”:[“:1443”],“read_timeout”:60000000000,“read_header_timeout”:60000000000,“routes”:[{“handle”:[{“handler”:“subroute”,“routes”:[{“handle”:[{“handler”:“vars”,“root”:“/opt/invenio/var/instance/static”}],“match”:[{“path”:[“/static/“]}]},{“handle”:[{“handler”:“rewrite”,“uri”:”{http.matchers.file.relative}“}],“match”:[{“file”:{“try_files”:[”{http.request.uri.path}“,”{http.request.uri.path}/“,”/index.html"]}}]},{“handle”:[{“encodings”:{“gzip”:{}},“handler”:“encode”,“prefer”:[“gzip”]},{“handler”:“reverse_proxy”,“upstreams”:[{“dial”:“web:5000”}]},{“handler”:“file_server”,“hide”:[“/etc/caddy/Caddyfile”]}]}]}],“terminal”:true}],“tls_connection_policies”:[{“match”:{“sni”:[“liberation.scot”]},“certificate_selection”:{“any_tag”:[“cert0”]}},{}],“automatic_https”:{“disable”:true}}}}}
{“level”:“info”,“ts”:1732385075.503,“logger”:“http”,“msg”:“enabling HTTP/3 listener”,“addr”:“:1443”}
{“level”:“debug”,“ts”:1732385075.5035672,“logger”:“http”,“msg”:“starting server loop”,“address”:“[::]:1443”,“tls”:true,“http3”:true}
{“level”:“info”,“ts”:1732385075.5041769,“logger”:“http.log”,“msg”:“server running”,“name”:“srv0”,“protocols”:[“h1”,“h2”,“h3”]}
{“level”:“info”,“ts”:1732385075.504684,“msg”:“autosaved config (load with --resume flag)”,“file”:“/config/caddy/autosave.json”}
{“level”:“info”,“ts”:1732385075.5047104,“msg”:“serving initial configuration”}
{“level”:“info”,“ts”:1732385075.5141025,“logger”:“tls”,“msg”:“storage cleaning happened too recently; skipping for now”,“storage”:“FileStorage:/data/caddy”,“instance”:“62804943-2f3d-4ee8-8fd5-77ad8e1e1906”,“try_again”:1732471475.5140991,“try_again_in”:86399.999999292}
{“level”:“info”,“ts”:1732385075.5144918,“logger”:“tls”,“msg”:“finished cleaning storage units”}
{“level”:“debug”,“ts”:1732385105.3336751,“logger”:“events”,“msg”:“event”,“name”:“tls_get_certificate”,“id”:“6d1fb5b3-84a1-46c6-9bd6-22bd1d5737f7”,“origin”:“tls”,“data”:{“client_hello”:{“CipherSuites”:[4866,4867,4865,49196,49200,159,52393,52392,52394,49195,49199,158,49188,49192,107,49187,49191,103,49162,49172,57,49161,49171,51,157,156,61,60,53,47,255],“ServerName”:“liberation.scot”,“SupportedCurves”:[29,23,30,25,24,256,257,258,259,260],“SupportedPoints”:“AAEC”,“SignatureSchemes”:[1027,1283,1539,2055,2056,2057,2058,2059,2052,2053,2054,1025,1281,1537,771,769,770,1026,1282,1538],“SupportedProtos”:[“h2”,“http/1.1”],“SupportedVersions”:[772,771],“RemoteAddr”:{“IP”:“172.18.0.1”,“Port”:59500,“Zone”:“”},“LocalAddr”:{“IP”:“172.18.0.2”,“Port”:1443,“Zone”:“”}}}}
{“level”:“debug”,“ts”:1732385105.3340967,“logger”:“tls.handshake”,“msg”:“choosing certificate”,“identifier”:“liberation.scot”,“num_choices”:1}
{“level”:“debug”,“ts”:1732385105.3356373,“logger”:“tls.handshake”,“msg”:“custom certificate selection results”,“identifier”:“liberation.scot”,“subjects”:[".liberation.scot”,“liberation.scot”],“managed”:false,“issuer_key”:“”,“hash”:“11814135d9b36b3b021ce09dc68455486c1ba9b9e30e83998a7e69271d2fdb41”}
{“level”:“debug”,“ts”:1732385105.3360643,“logger”:“tls.handshake”,“msg”:“matched certificate in cache”,“remote_ip”:“172.18.0.1”,“remote_port”:“59500”,“subjects”:[“.liberation.scot",“liberation.scot”],“managed”:false,“expiration”:1763389967,“hash”:“11814135d9b36b3b021ce09dc68455486c1ba9b9e30e83998a7e69271d2fdb41”}
{“level”:“debug”,“ts”:1732385105.353523,“logger”:“http.handlers.reverse_proxy”,“msg”:“selected upstream”,“dial”:“web:5000”,“total_upstreams”:1}
{“level”:“debug”,“ts”:1732385121.3470411,“logger”:“http.handlers.reverse_proxy”,“msg”:“upstream roundtrip”,“upstream”:“web:5000”,“duration”:15.993139954,“request”:{“remote_ip”:“172.18.0.1”,“remote_port”:“59500”,“client_ip”:“172.18.0.1”,“proto”:“HTTP/2.0”,“method”:“GET”,“host”:“liberation.scot:1443”,“uri”:“/”,“headers”:{“User-Agent”:[“curl/7.81.0”],“Accept”:["/“],“X-Forwarded-For”:[“172.18.0.1”],“X-Forwarded-Proto”:[“https”],“X-Forwarded-Host”:[“liberation.scot:1443”]},“tls”:{“resumed”:false,“version”:772,“cipher_suite”:4865,“proto”:“h2”,“server_name”:“liberation.scot”}},“headers”:{“Content-Type”:[“text/html”],“Content-Length”:[“141”]},“status”:500}
{“level”:“debug”,“ts”:1732385133.8846562,“logger”:“events”,“msg”:“event”,“name”:“tls_get_certificate”,“id”:“8009f3dc-316f-45eb-817d-ccf298117e85”,“origin”:“tls”,“data”:{“client_hello”:{“CipherSuites”:[47802,4865,4866,4867,49195,49199,49196,49200,52393,52392,49171,49172,156,157,47,53],“ServerName”:“liberation.scot”,“SupportedCurves”:[56026,25497,29,23,24],“SupportedPoints”:“AA==”,“SignatureSchemes”:[1027,2052,1025,1283,2053,1281,2054,1537],“SupportedProtos”:[“h2”,“http/1.1”],“SupportedVersions”:[23130,772,771],“RemoteAddr”:{“IP”:“98.33.117.86”,“Port”:46080,“Zone”:”“},“LocalAddr”:{“IP”:“172.18.0.2”,“Port”:1443,“Zone”:”“}}}}
{“level”:“debug”,“ts”:1732385133.8847044,“logger”:“tls.handshake”,“msg”:“choosing certificate”,“identifier”:“liberation.scot”,“num_choices”:1}
{“level”:“debug”,“ts”:1732385133.8847563,“logger”:“tls.handshake”,“msg”:“custom certificate selection results”,“identifier”:“liberation.scot”,“subjects”:[”.liberation.scot”,“liberation.scot”],“managed”:false,“issuer_key”:“”,“hash”:“11814135d9b36b3b021ce09dc68455486c1ba9b9e30e83998a7e69271d2fdb41”}
{“level”:“debug”,“ts”:1732385133.8847716,“logger”:“tls.handshake”,“msg”:“matched certificate in cache”,“remote_ip”:“98.33.117.86”,“remote_port”:“46080”,“subjects”:[“.liberation.scot",“liberation.scot”],“managed”:false,“expiration”:1763389967,“hash”:“11814135d9b36b3b021ce09dc68455486c1ba9b9e30e83998a7e69271d2fdb41”}
{“level”:“debug”,“ts”:1732385134.0767972,“logger”:“http.handlers.reverse_proxy”,“msg”:“selected upstream”,“dial”:“web:5000”,“total_upstreams”:1}
{“level”:“debug”,“ts”:1732385134.198753,“logger”:“http.handlers.reverse_proxy”,“msg”:“upstream roundtrip”,“upstream”:“web:5000”,“duration”:0.121867418,“request”:{“remote_ip”:“98.33.117.86”,“remote_port”:“46080”,“client_ip”:“98.33.117.86”,“proto”:“HTTP/2.0”,“method”:“GET”,“host”:“liberation.scot:1443”,“uri”:“/”,“headers”:{“Sec-Ch-Ua-Mobile”:[“?0”],“X-Forwarded-For”:[“98.33.117.86”],“Accept”:["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,/;q=0.8,application/signed-exchange;v=b3;q=0.7"],“Cookie”:[“REDACTED”],“Priority”:[“u=0, i”],“Sec-Fetch-User”:[“?1”],“Accept-Language”:[“en-GB,en-US;q=0.9,en;q=0.8”],“Sec-Fetch-Mode”:[“navigate”],“Upgrade-Insecure-Requests”:[“1”],“Sec-Ch-Ua-Platform”:[“"Linux"”],“Accept-Encoding”:[“gzip, deflate, br, zstd”],“Sec-Fetch-Site”:[“none”],“X-Forwarded-Proto”:[“https”],“Sec-Ch-Ua”:[“"Chromium";v="130", "Google Chrome";v="130", "Not?A_Brand";v="99"”],“User-Agent”:[“Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36”],“Sec-Fetch-Dest”:[“document”],“Cache-Control”:[“max-age=0”],“X-Forwarded-Host”:[“liberation.scot:1443”]},“tls”:{“resumed”:false,“version”:772,“cipher_suite”:4865,“proto”:“h2”,“server_name”:“liberation.scot”}},“headers”:{“Content-Length”:[“141”],“Content-Type”:[“text/html”]},“status”:500}
{“level”:“debug”,“ts”:1732385134.4590473,“logger”:“http.handlers.reverse_proxy”,“msg”:“selected upstream”,“dial”:“web:5000”,“total_upstreams”:1}
{“level”:“debug”,“ts”:1732385134.5075738,“logger”:“http.handlers.reverse_proxy”,“msg”:“upstream roundtrip”,“upstream”:“web:5000”,“duration”:0.048416396,“request”:{“remote_ip”:“98.33.117.86”,“remote_port”:“46080”,“client_ip”:“98.33.117.86”,“proto”:“HTTP/2.0”,“method”:“GET”,“host”:“liberation.scot:1443”,“uri”:“/favicon.ico”,“headers”:{“Sec-Fetch-Mode”:[“no-cors”],“Accept-Language”:[“en-GB,en-US;q=0.9,en;q=0.8”],“Sec-Fetch-Site”:[“same-origin”],“Sec-Ch-Ua”:[“"Chromium";v="130", "Google Chrome";v="130", "Not?A_Brand";v="99"”],“X-Forwarded-For”:[“98.33.117.86”],“User-Agent”:[“Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36”],“Sec-Ch-Ua-Mobile”:[“?0”],“Sec-Ch-Ua-Platform”:[“"Linux"”],“Cookie”:[“REDACTED”],“Sec-Fetch-Dest”:[“image”],“Priority”:[“u=1, i”],“X-Forwarded-Proto”:[“https”],“X-Forwarded-Host”:[“liberation.scot:1443”],“Accept”:["image/avif,image/webp,image/apng,image/svg+xml,image/,/;q=0.8”],“Referer”:[“https://liberation.scot:1443/"],“Accept-Encoding”:["gzip, deflate, br, zstd”]},“tls”:{“resumed”:false,“version”:772,“cipher_suite”:4865,“proto”:“h2”,“server_name”:“liberation.scot”}},“headers”:{“Content-Type”:[“text/html”],“Content-Length”:[“141”]},“status”:500}
{“level”:“debug”,“ts”:1732385136.6981754,“logger”:“http.handlers.reverse_proxy”,“msg”:“selected upstream”,“dial”:“web:5000”,“total_upstreams”:1}
{“level”:“debug”,“ts”:1732385136.7356613,“logger”:“http.handlers.reverse_proxy”,“msg”:“upstream roundtrip”,“upstream”:“web:5000”,“duration”:0.037350964,“request”:{“remote_ip”:“98.33.117.86”,“remote_port”:“46080”,“client_ip”:“98.33.117.86”,“proto”:“HTTP/2.0”,“method”:“GET”,“host”:“liberation.scot:1443”,“uri”:“/”,“headers”:{“Accept”:[“text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,/;q=0.8,application/signed-exchange;v=b3;q=0.7”],“Accept-Language”:[“en-GB,en-US;q=0.9,en;q=0.8”],“X-Forwarded-Proto”:[“https”],“Sec-Ch-Ua-Mobile”:[“?0”],“Sec-Fetch-User”:[“?1”],“Cookie”:[“REDACTED”],“Accept-Encoding”:[“gzip, deflate, br, zstd”],“X-Forwarded-For”:[“98.33.117.86”],“X-Forwarded-Host”:[“liberation.scot:1443”],“Sec-Fetch-Dest”:[“document”],“Priority”:[“u=0, i”],“Sec-Fetch-Site”:[“none”],“Cache-Control”:[“max-age=0”],“Sec-Fetch-Mode”:[“navigate”],“User-Agent”:[“Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36”],“Sec-Ch-Ua”:[“"Chromium";v="130", "Google Chrome";v="130", "Not?A_Brand";v="99"”],“Sec-Ch-Ua-Platform”:[“"Linux"”],“Upgrade-Insecure-Requests”:[“1”]},“tls”:{“resumed”:false,“version”:772,“cipher_suite”:4865,“proto”:“h2”,“server_name”:“liberation.scot”}},“headers”:{“Content-Type”:[“text/html”],“Content-Length”:[“141”]},“status”:500}
{“level”:“debug”,“ts”:1732385137.0349178,“logger”:“http.handlers.reverse_proxy”,“msg”:“selected upstream”,“dial”:“web:5000”,“total_upstreams”:1}
{“level”:“debug”,“ts”:1732385137.1461637,“logger”:“http.handlers.reverse_proxy”,“msg”:“upstream roundtrip”,“upstream”:“web:5000”,“duration”:0.111151031,“request”:{“remote_ip”:“98.33.117.86”,“remote_port”:“46080”,“client_ip”:“98.33.117.86”,“proto”:“HTTP/2.0”,“method”:“GET”,“host”:“liberation.scot:1443”,“uri”:“/favicon.ico”,“headers”:{“Sec-Ch-Ua”:[“"Chromium";v="130", "Google Chrome";v="130", "Not?A_Brand";v="99"”],“Sec-Fetch-Dest”:[“image”],“X-Forwarded-Host”:[“liberation.scot:1443”],“Accept”:[“image/avif,image/webp,image/apng,image/svg+xml,image/,/*;q=0.8”],“Accept-Encoding”:[“gzip, deflate, br, zstd”],“Sec-Ch-Ua-Platform”:[“"Linux"”],“Accept-Language”:[“en-GB,en-US;q=0.9,en;q=0.8”],“X-Forwarded-For”:[“98.33.117.86”],“Referer”:[“https://liberation.scot:1443/“],“Cookie”:[“REDACTED”],“X-Forwarded-Proto”:[“https”],“Sec-Ch-Ua-Mobile”:[”?0"],“Priority”:["u=1, i”],“User-Agent”:[“Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36”],“Sec-Fetch-Site”:[“same-origin”],“Sec-Fetch-Mode”:[“no-cors”]},“tls”:{“resumed”:false,“version”:772,“cipher_suite”:4865,“proto”:“h2”,“server_name”:“liberation.scot”}},“headers”:{“Content-Type”:[“text/html”],“Content-Length”:[“141”]},“status”:500}
root@liberation:/data/invenio-rdm-starter#
Thanks for Patience and help here (all Caddy team 
) .
The exceptions I see now are App side and not proxy side I believe.
So ,I’ll try and get help tracking them down.
THEN
I’ll write up an example how to use commercial certs on NEW ports with Caddy and a complex “web” app backend and publish HERE and the Invenio side.
1 Like
Would you like me to post an example file on leveraging commercial certs when Caddy is in a container?
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.