Caddy does not create certificate for one domain

Help
1

1. The problem I’m having:

I am starting caddy using caddy run with the following caddy file: k8s-infrastructure/Caddyfile at 7c58e73a340274f86c722dbd983b30ca8110ad41 · Androz2091/k8s-infrastructure · GitHub

I added a new entry, pdf.androz2091.fr.

2. Error messages and/or full log output:

the pdf certificate does not get created
Note that I included my whole log file but some errors come from /paypal/ipn failing because my manage-invite-api service is down

root@ns561436:~/.local/share/caddy/certificates/acme-v02.api.letsencrypt.org-directory# ls
10mans-adminjs.androz2091.fr  cloud.androz2091.fr                      harbor.androz2091.fr        photos.androz2091.fr                  radarr.androz2091.fr              tagger.androz2091.fr
analytics.androz2091.fr       dash.manage-invite.xyz                   haste.androz2091.fr         plex.androz2091.fr                    sabnzbd.androz2091.fr             tautulli.androz2091.fr
androz2091.fr                 ddpe.androz2091.fr                       kolc-adminjs.androz2091.fr  pokercode-quiz-adminjs.androz2091.fr  simonlefort.ch                    timetagger.androz2091.fr
api.manage-invite.xyz         diswho.androz2091.fr                     monica.androz2091.fr        prometheus-grafana.androz2091.fr      simonlefort.fr                    vault.androz2091.fr
argocd.androz2091.fr          evolution-markets-adminjs.androz2091.fr  pgadmin.androz2091.fr       qbt.androz2091.fr                     slash-commands-gui.androz2091.fr

ebian@ns561436:~/k8s-infrastructure$ sudo caddy run
2024/11/30 12:06:30.609 INFO    using adjacent Caddyfile
2024/11/30 12:06:30.612 INFO    adapted config to JSON  {"adapter": "caddyfile"}
2024/11/30 12:06:30.612 WARN    Caddyfile input is not formatted; run 'caddy fmt --overwrite' to fix inconsistencies    {"adapter": "caddyfile", "file": "Caddyfile", "line": 2}
2024/11/30 12:06:30.613 INFO    admin   admin endpoint started  {"address": "localhost:2019", "enforce_origin": false, "origins": ["//localhost:2019", "//[::1]:2019", "//127.0.0.1:2019"]}
2024/11/30 12:06:30.613 INFO    http.auto_https enabling automatic HTTP->HTTPS redirects        {"server_name": "srv0"}
2024/11/30 12:06:30.613 INFO    tls.cache.maintenance   started background certificate maintenance      {"cache": "0xc000319b00"}
2024/11/30 12:06:30.614 INFO    http.auto_https server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS {"server_name": "srv1", "https_port": 443}
2024/11/30 12:06:30.614 INFO    http.auto_https enabling automatic HTTP->HTTPS redirects        {"server_name": "srv1"}
2024/11/30 12:06:30.615 INFO    http    enabling HTTP/3 listener        {"addr": ":32400"}
2024/11/30 12:06:30.615 INFO    http.log        server running  {"name": "srv0", "protocols": ["h1", "h2", "h3"]}
2024/11/30 12:06:30.615 INFO    http    enabling HTTP/3 listener        {"addr": ":443"}
2024/11/30 12:06:30.615 INFO    http.log        server running  {"name": "srv1", "protocols": ["h1", "h2", "h3"]}
2024/11/30 12:06:30.615 INFO    http.log        server running  {"name": "remaining_auto_https_redirects", "protocols": ["h1", "h2", "h3"]}
2024/11/30 12:06:30.615 INFO    http    enabling automatic TLS certificate management   {"domains": ["timetagger.androz2091.fr", "androz2091.fr", "10mans-adminjs.androz2091.fr", "evolution-markets-adminjs.androz2091.fr", "api.manage-invite.xyz", "dash.manage-invite.xyz", "simonlefort.fr", "vault.androz2091.fr", "radarr.androz2091.fr", "harbor.androz2091.fr", "sabnzbd.androz2091.fr", "prometheus-grafana.androz2091.fr", "pokercode-quiz-adminjs.androz2091.fr", "tagger.androz2091.fr", "pgadmin.androz2091.fr", "cloud.androz2091.fr", "analytics.androz2091.fr", "ddpe.androz2091.fr", "simonlefort.ch", "diswho.androz2091.fr", "haste.androz2091.fr", "argocd.androz2091.fr", "plex.androz2091.fr", "tautulli.androz2091.fr", "photos.androz2091.fr", "kolc-adminjs.androz2091.fr", "qbt.androz2091.fr", "slash-commands-gui.androz2091.fr", "pdf.androz2091.fr"]}
2024/11/30 12:06:30.622 INFO    tls.obtain      acquiring lock  {"identifier": "pdf.androz2091.fr"}
2024/11/30 12:06:30.627 INFO    autosaved config (load with --resume flag)      {"file": "/root/.config/caddy/autosave.json"}
2024/11/30 12:06:30.627 INFO    serving initial configuration
2024/11/30 12:06:30.642 INFO    tls     storage cleaning happened too recently; skipping for now        {"storage": "FileStorage:/root/.local/share/caddy", "instance": "ea9c872e-2d5d-4c4f-a241-eedd310ff710", "try_again": "2024/12/01 12:06:30.642", "try_again_in": 86399.999998278}
2024/11/30 12:06:30.642 INFO    tls     finished cleaning storage units

3. Caddy version:

v2.8.4

4. How I installed and ran Caddy:

a. System environment:

Debian v12.8

b. Command:

sudo caddy run

d. My complete Caddy config:

See below.

5. Links to relevant resources:

2

Hi @Androz2091,

Using the online tool Let’s Debug yields these results
https://letsdebug.net/pdf.androz2091.fr/2296922

ANotWorking
Error
pdf.androz2091.fr has an A (IPv4) record (54.39.102.76) but a request to this address over port 80 did not succeed. Your web server must have at least one working IPv4 or IPv6 address.
Get "https://pdf.androz2091.fr/.well-known/acme-challenge/letsdebug-test": remote error: tls: internal error

Trace:
@0ms: Making a request to http://pdf.androz2091.fr/.well-known/acme-challenge/letsdebug-test (using initial IP 54.39.102.76)
@0ms: Dialing 54.39.102.76
@215ms: Server response: HTTP 308 Permanent Redirect
@215ms: Received redirect to https://pdf.androz2091.fr/.well-known/acme-challenge/letsdebug-test
@215ms: Dialing 54.39.102.76
@432ms: Experienced error: remote error: tls: internal error

image
image1001×433 6.38 KB

3

Thank you for your reply! I think it is linked to another issue I’ve got with Caddy, do you have any idea why? ssl - OpenSSL routines:ssl3_read_bytes:tlsv1 alert internal error with kubernetes and caddy - Server Fault

I’ve been having this issue for weeks and I cannot figure out what’s wrong

1 Like
4

HTTP on Port 80 ACME challenge gives a redirect, which is fine, to HTTPS.

$ curl -Ii http://pdf.androz2091.fr/.well-known/acme-challenge/sometestfile
HTTP/1.1 308 Permanent Redirect
Connection: close
Location: https://pdf.androz2091.fr/.well-known/acme-challenge/sometestfile
Server: Caddy
Date: Sat, 30 Nov 2024 17:18:23 GMT

Following the redirect to HTTPS.
HTTPS on Port 443 fails to connect.

$ curl -k -Ii https://pdf.androz2091.fr:443/.well-known/acme-challenge/sometestfile
curl: (35) error:0A000438:SSL routines::tlsv1 alert internal error

Checking what is on Port 443.
HTTP on Port 443 connects and get a response of HTTP/1.0 400 Bad Request.

$ curl -k -Ii http://pdf.androz2091.fr:443/.well-known/acme-challenge/sometestfile
HTTP/1.0 400 Bad Request
5

Thank you for your reply. What can I do for that? How can I debug the issue? I only ran caddy run.

1 Like
6

Kindly wait for more knowledgeable Caddy community volunteers to assist.

1 Like
7

https://pdf.androz2091.fr seems to load fine now.

8

Yes indeed, it now loads fine, but if you run openssl s_client -connect pdf.androz2091.fr:443 a few times… you’ll see some errors. I’m having a hard time figuring out why it doesn’t work.

9

You mean like this?

Failed this time.

$ echo ; openssl s_client -connect pdf.androz2091.fr:443 < /dev/null ; echo

CONNECTED(00000003)
4067E5A3397F0000:error:0A000438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error:../ssl/record/rec_layer_s3.c:1605:SSL alert number 80
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 7 bytes and written 323 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
---

Seem to work this time.

$ echo ; openssl s_client -connect pdf.androz2091.fr:443 < /dev/null ; echo

CONNECTED(00000003)
depth=2 C = US, O = Internet Security Research Group, CN = ISRG Root X1
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = E5
verify return:1
depth=0 CN = pdf.androz2091.fr
verify return:1
---
Certificate chain
 0 s:CN = pdf.androz2091.fr
   i:C = US, O = Let's Encrypt, CN = E5
   a:PKEY: id-ecPublicKey, 256 (bit); sigalg: ecdsa-with-SHA384
   v:NotBefore: Dec  3 06:05:58 2024 GMT; NotAfter: Mar  3 06:05:57 2025 GMT
 1 s:C = US, O = Let's Encrypt, CN = E5
   i:C = US, O = Internet Security Research Group, CN = ISRG Root X1
   a:PKEY: id-ecPublicKey, 384 (bit); sigalg: RSA-SHA256
   v:NotBefore: Mar 13 00:00:00 2024 GMT; NotAfter: Mar 12 23:59:59 2027 GMT
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIDgjCCAwigAwIBAgISAyR08BwjY5LKoBDWhYH38jVNMAoGCCqGSM49BAMDMDIx
CzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQswCQYDVQQDEwJF
NTAeFw0yNDEyMDMwNjA1NThaFw0yNTAzMDMwNjA1NTdaMBwxGjAYBgNVBAMTEXBk
Zi5hbmRyb3oyMDkxLmZyMFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEFEPugW5X
7zg6CIPdIuW0BfPeXdUE8ZAjIR7aNYI8AT0V7g73xFASHQ8QCcaHY/ICSymCL3XR
NGr0D/53mwd3JaOCAhIwggIOMA4GA1UdDwEB/wQEAwIHgDAdBgNVHSUEFjAUBggr
BgEFBQcDAQYIKwYBBQUHAwIwDAYDVR0TAQH/BAIwADAdBgNVHQ4EFgQU3rKGoPIg
urbHps+QZUklNqjakmAwHwYDVR0jBBgwFoAUnytfzzwhT50Et+0rLMTGcIvS1w0w
VQYIKwYBBQUHAQEESTBHMCEGCCsGAQUFBzABhhVodHRwOi8vZTUuby5sZW5jci5v
cmcwIgYIKwYBBQUHMAKGFmh0dHA6Ly9lNS5pLmxlbmNyLm9yZy8wHAYDVR0RBBUw
E4IRcGRmLmFuZHJvejIwOTEuZnIwEwYDVR0gBAwwCjAIBgZngQwBAgEwggEDBgor
BgEEAdZ5AgQCBIH0BIHxAO8AdQCi4wrkRe+9rZt+OO1HZ3dT14JbhJTXK14bLMS5
UKRH5wAAAZOLVbAFAAAEAwBGMEQCICEn3xEO58KK2bK9qk2JS5TtrlLCS40lN5ws
h3rW53UOAiBVNOvgEuCW6Hf0r5wUXIqlhPIBNGynXzc+0ywPjHOmtwB2AObSMWNA
d4zBEEEG13G5zsHSQPaWhIb7uocyHf0eN45QAAABk4tVt9oAAAQDAEcwRQIgPalF
+pZsXMYcK9H7JdPhCchfn1RwdR4bJFThPIi46HoCIQCuD/ac4fcIHTg+04QFSh9w
28jVKFwvSqo8nuk3wdIzwjAKBggqhkjOPQQDAwNoADBlAjEAy3VbQOXU53T2qVcW
SWI7fFtlc+BYr/3qZ0HozBB5MNjlJXYT+AWfZ2ygtc7H7XCdAjAoaqV3ZFbe7J/t
ljjqHkj0XRakq1d08phmX5pkU8LAuUj7q1WnyAWwnjlgMXGRYpI=
-----END CERTIFICATE-----
subject=CN = pdf.androz2091.fr
issuer=C = US, O = Let's Encrypt, CN = E5
---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: ECDSA
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 2378 bytes and written 387 bytes
Verification: OK
---
New, TLSv1.3, Cipher is TLS_AES_128_GCM_SHA256
Server public key is 256 bit
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
---
---
Post-Handshake New Session Ticket arrived:
SSL-Session:
    Protocol  : TLSv1.3
    Cipher    : TLS_AES_128_GCM_SHA256
    Session-ID: DF96BAC9F6964C2DD98110E47A63F30154DE6477E9B991C534602457CF01C0E4
    Session-ID-ctx:
    Resumption PSK: 7E5DAA67E82FF2102622A17DC19EB8D7D6B008A944E845D1AC026ECFF73AA093
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 604800 (seconds)
    TLS session ticket:
    0000 - b9 46 c1 53 5d 6c df 61-f7 de e6 2d 57 80 9a a5   .F.S]l.a...-W...
    0010 - 68 59 5d 14 37 63 f9 15-86 2e 27 52 73 3d cb f7   hY].7c....'Rs=..
    0020 - de e2 8f e7 9e 17 05 90-56 df 2e ba 1c 6a 38 5c   ........V....j8\
    0030 - 13 13 ad 2c 74 c1 b0 25-71 9a 4a 19 6e b8 27 42   ...,t..%q.J.n.'B
    0040 - 49 28 9b 58 e8 26 84 a6-b3 58 4e 29 f1 e6 c1 7e   I(.X.&...XN)...~
    0050 - ed 57 4d e7 3d 57 15 4c-af a8 0c df 32 2c c1 7d   .WM.=W.L....2,.}
    0060 - 1b 3e c0 eb e4 7d d2 01-09                        .>...}...

    Start Time: 1733589293
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: no
    Max Early Data: 0
---
read R BLOCK
DONE
10

Yes exactly! It randomly fails exactly like this. Do you have any idea about where I can have a look at? (I don’t have a load balancer, I just have Caddy on the host pointing to a kubernetes service)

1 Like
11

Not sure, but maybe the Cipher Suite is to limited
https://www.ssllabs.com/ssltest/analyze.html?d=pdf.androz2091.fr

1 Like
12

thank you, I don’t understand how these results would allow me to make progress (it’s because I’m still trying to learn, I understand that someone can know how to exploit this data).

13

@Bruce5051 thank you so much for your help! I had multiple instances of Caddy running on my server…!
Here is the complete explanation of my finding: ssl - OpenSSL routines:ssl3_read_bytes:tlsv1 alert internal error with kubernetes and caddy - Server Fault

1 Like
14

You are very welcome @Androz2091! :slight_smile:
Glady you got it all working.
Have pleasant day. :sun_with_face:

15

This

root     1325594       1  0 Oct16 ?        00:25:18 caddy run --pingback 127.0.0.1:46125

indicates you’ve run caddy start at one point. The caddy start command starts a Caddy process in the background, which is how you ended up with multiple instances. See the difference between start and run here:

1 Like