File or directory not found in TLS directive

Help
1

1. The problem I’m having:

I was trying to force the usage of https for local domain/access running Vaultwarden and Caddy via docker.

Starting up Caddy container will result in the subject error

2. Error messages and/or full log output:

Error: sending configuration to instance: caddy responded with error: HTTP 400: {"error":"loading config: loading new config: loading http app module: provision http: getting tls app: loading tls app module: provision tls: loading certificates: open /etc/ssl/certs/_wildcard.cccc.nord.pem: no such file or directory

3. Caddy version:

docker exec caddy caddy version

v2.6.4 h1:2hwYqiRwk1tf3VruhMpLcYTg+11fCdr8S3jhNAdnPy8=

4. How I installed and ran Caddy:

docker-compose.yml

a. System environment:

Ubuntu 20.4 LTS

b. Command:

docker compose up -d

c. Service/unit/compose file:

d. My complete Caddy config:

v.cccc.nord {
  encode gzip

  # Uncomment to improve security (WARNING: only use if you understand the implications!)
  # If you want to use FIDO2 WebAuthn, set X-Frame-Options to "SAMEORIGIN" or the Browser >
  header {
       # Enable HTTP Strict Transport Security (HSTS)
       Strict-Transport-Security "max-age=31536000;"
       # Enable cross-site filter (XSS) and tell browser to block detected attacks
       X-XSS-Protection "1; mode=block"
       # Disallow the site to be rendered within a frame (clickjacking protection)
       X-Frame-Options "DENY"
       # Prevent search engines from indexing (optional)
       X-Robots-Tag "none"
       # Server name removing
       -Server
  }
  tls  /etc/ssl/certs/_wildcard.cccc.nord.pem  /etc/ssl/certs/_wildcard.cccc.nord.key.pem

  
  # Notifications redirected to the websockets server
  #reverse_proxy /notifications/hub localhost:3012  

  # Proxy everything else to Rocket
  reverse_proxy localhost:8080  {                  
       # Send the true remote IP to Rocket, so that vaultwarden can put this in the
       # log, so that fail2ban can ban the correct IP.
       header_up X-Real-IP {remote_host}
  }
}

5. Links to relevant resources:

I have searched and found this topic SSL Configuration in Caddy . It appeared not different from what I was doing.

I have used ‘mkcert’ to generate the certificates for the local domain “cccc.nord” and copied them into /etc/ssl/certs directory

Could it be a permission issue? How does Caddy search for files in TLS directive?


ls -l /etc/ssl/certs/_w
-rw-r--r-- 1 root root 1704 Jun 18 03:14 /etc/ssl/certs/_wildcard.cccc.nord-key.pem
-rw-r--r-- 1 root root 1468 Jun 18 03:14 /etc/ssl/certs/_wildcard.cccc.nord.pem
2

Your Caddyfile is malformed (structure) – see the docs:

EDIT: Just kidding, my mobile screen + eyes playing tricks on me.

3

I don’t agree, looks fine to me.

Did you actually mount the certs into your Docker container? Processes inside containers can’t read files outside the container unless you mount it. You didn’t share your docker-compose.yml.

4

Oh, wow, you’re right – the indentation looked weird on mobile at a quick glance.

Sorry!

5

Thank you Francis, your answer is right on I think. I have tried to specify volume in caddy compose but they failed so far.

services:

  caddy:
    image: caddy
    container_name: caddy
    hostname: caddy
    restart: unless-stopped
    env_file: .env
    ports:
      - "80:80"
      - "443:443"
      - "443:443/udp"
    volumes:
      - ./Caddyfile:/etc/caddy/Caddyfile
      - ./caddy_config:/config
      - ./caddy_data:/data
      - /etc/ssl/certs:/etc/ssl/certs./:ro


networks:
  default:
    name: $DOCKER_MY_NETWORK
    external: true

I will keep searching for the volumes solution, but if you know the way can you help?

Thanks again.

1 Like
6

You have a ./ at the end there. Remove that.

1 Like
7

I have played with this volume parameters few times therefore this mistake. I have fixed this now and got it right in the past but same errors …

{"level":"info","ts":1687374023.1021485,"msg":"using adjacent Caddyfile"}
{"level":"warn","ts":1687374023.104558,"msg":"Caddyfile input is not formatted; run the 'caddy fmt' command to fix inconsistencies","adapter":"caddyfile","file":"Caddyfile","line":2}
Error: sending configuration to instance: caddy responded with error: HTTP 400: {"error":"loading config: loading new config: loading http app module: provision http: getting tls app: loading tls app module: provision tls: loading certificates: open /etc/ssl/certs/_wildcard.cccc.nord.pem: no such file or directory"}
8

Sorry my typo error again at somewhere else. Fixed it and now new errors …

{"level":"info","ts":1687375107.5364249,"logger":"http","msg":"done waiting on internal rate limiter","identifiers":["v.cccc.nord"],"ca":"https://acme-v02.api.letsencrypt.org/directory","account":"caddy@zerossl.com"}
{"level":"error","ts":1687375108.0732105,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"v.cccc.olympic.nord","issuer":"acme-v02.api.letsencrypt.org-directory","error":"HTTP 400 urn:ietf:params:acme:error:rejectedIdentifier - Error creating new order :: Cannot issue for \"v.cccc.nord\": Domain name does not end with a valid public suffix (TLD)"}

Does it mean the domain name must conform to something ending with .com, .org, etc …?
The name was given by Meshnet (or NordVPN) and I cannot change it.

9

That means that .nord is not in the PSL list and therefore doesn’t qualify for a public cert. You’d have to use a different domain. This isn’t a problem with Caddy, this is Let’s Encrypt denying it.

1 Like
10

If Let’s Encrypt does not allow it do you have other way to overcome my setup: using HTTPS with a local domain name not PSL conforming?

11

You can issue your own certificate but it won’t be publicly trusted, by adding this to your site block:

tls internal
12

I made use of the ‘tls internal’ and got this from Caddy log …

{"level":"info","ts":1687403536.5371983,"logger":"http.log","msg":"server running","name":"remaining_auto_https_redirects","protocols":["h1","h2","h3"]}
{"level":"info","ts":1687403536.537252,"logger":"http","msg":"enabling automatic TLS certificate management","domains":["zung102-olympic.nord"]}
{"level":"info","ts":1687403536.539026,"logger":"tls","msg":"finished cleaning storage units"}
{"level":"info","ts":1687403536.539404,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
{"level":"info","ts":1687403536.5394282,"msg":"serving initial configuration"}
{"level":"info","ts":1687403536.540136,"logger":"tls.obtain","msg":"acquiring lock","identifier":"zung102-olympic.nord"}
{"level":"info","ts":1687403536.5438573,"logger":"tls.obtain","msg":"lock acquired","identifier":"zung102-olympic.nord"}
{"level":"info","ts":1687403536.5440812,"logger":"tls.obtain","msg":"obtaining certificate","identifier":"zung102-olympic.nord"}
{"level":"info","ts":1687403536.545544,"logger":"tls.obtain","msg":"certificate obtained successfully","identifier":"zung102-olympic.nord"}
{"level":"info","ts":1687403536.5456305,"logger":"tls.obtain","msg":"releasing lock","identifier":"zung102-olympic.nord"}
{"level":"warn","ts":1687403536.5462217,"logger":"tls","msg":"stapling OCSP","error":"no OCSP stapling for [zung102-olympic.nord]: no OCSP server specified in certificate","identifiers":["zung102-olympic.nord"]}

However I got this error when using HTTPS …

zung102-olympic.nord sent an invalid response.

  • [Try running Windows Network Diagnostics](javascript:diagnoseErrors()).

ERR_SSL_PROTOCOL_ERROR

if using HTTP then Vaultwarden login page came up fine.

Why did HTTPS fail?

13

Don’t use browsers to troubleshoot.

What does a curl -v request for your domain tell you?

I’m guessing, 99%, that it’s because the certificate isn’t trusted. If so, you’ll need to make sure your host trusts the root Caddy generated in your container.

14

Firstly, thank you for the incredible support and help from you guys and for being patient with me.

I have restarted Caddy container again and got these errors …

{"level":"info","ts":1687447037.8745723,"logger":"http","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
{"level":"warn","ts":1687447038.0703156,"logger":"pki.ca.local","msg":"installing root certificate (you might be prompted for password)","path":"storage:pki/authorities/local/root.crt"}
{"level":"info","ts":1687447038.0743916,"msg":"warning: \"certutil\" is not available, install \"certutil\" with \"apt install libnss3-tools\" or \"yum install nss-tools\" and try again"}
{"level":"info","ts":1687447038.074617,"msg":"define JAVA_HOME environment variable to use the Java trust"}
{"level":"error","ts":1687447038.095018,"logger":"pki.ca.local","msg":"failed to install root certificate","error":"failed to execute update-ca-certificates: exit status 1","certificate_file":"storage:pki/authorities/local/root.crt"}
{"level":"info","ts":1687447038.095376,"logger":"http","msg":"enabling HTTP/3 listener","addr":":443"}
{"level":"info","ts":1687447038.095602,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2","h3"]}
{"level":"info","ts":1687447038.0956967,"logger":"tls","msg":"cleaning storage unit","description":"FileStorage:/data/caddy"}
{"level":"info","ts":1687447038.0957139,"logger":"http.log","msg":"server running","name":"remaining_auto_https_redirects","protocols":["h1","h2","h3"]}
{"level":"info","ts":1687447038.0957224,"logger":"http","msg":"enabling automatic TLS certificate management","domains":["zung102-olympic.nord"]}
{"level":"warn","ts":1687447038.101214,"logger":"tls","msg":"stapling OCSP","error":"no OCSP stapling for [zung102-olympic.nord]: no OCSP server specified in certificate","identifiers":["zung102-olympic.nord"]}
{"level":"info","ts":1687447038.104262,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
{"level":"info","ts":1687447038.1044781,"msg":"serving initial configuration"}
{"level":"info","ts":1687447038.1103811,"logger":"tls","msg":"finished cleaning storage units"}

It appeared there was an issue with permission since I can find “root.crt” here …

zung25@zung-1:~/docker/caddy$ sudo ls -l caddy_data/caddy/pki/authorities/local
total 16
-rw------- 1 root root 676 Jun 22 02:49 intermediate.crt
-rw------- 1 root root 227 Jun 22 02:49 intermediate.key
-rw------- 1 root root 631 Jun 22 02:49 root.crt
-rw------- 1 root root 227 Jun 22 02:49 root.key

Following Matt’s suggestion …

 curl -v zung102-olympic.nord
*   Trying 100.122.127.103:80...
* Connected to zung102-olympic.nord (100.122.127.103) port 80 (#0)
> GET / HTTP/1.1
> Host: zung102-olympic.nord
> User-Agent: curl/7.81.0
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 308 Permanent Redirect
< Connection: close
< Location: https://zung102-olympic.nord/
< Server: Caddy
< Date: Thu, 22 Jun 2023 15:51:46 GMT
< Content-Length: 0
< 
* Closing connection 0
1 Like
15

Those errors are saying that Caddy didn’t have permission to install the root CA cert (because it’s running as a non-root user without those permissions). To do that, run sudo caddy trust, which will perform the trust installation as root instead.

You only made an HTTP request, and Caddy served you a redirect to HTTPS. See the Location header, which instructs the client to try again, at a different address. You can use the -L flag to follow the Location header redirect, or you can just do curl -v https://zung102-olympic.nord instead.

1 Like
16

I do not have caddy binary to run sudo caddy trust …and

sudo docker exec caddy caddy trust
{"level":"warn","ts":1687451802.4255006,"msg":"installing root certificate (you might be prompted for password)","path":"localhost:2019/pki/ca/local"}
{"level":"info","ts":1687451802.426099,"msg":"warning: \"certutil\" is not available, install \"certutil\" with \"apt install libnss3-tools\" or \"yum install nss-tools\" and try again"}
{"level":"info","ts":1687451802.4261234,"msg":"define JAVA_HOME environment variable to use the Java trust"}
Error: failed to execute update-ca-certificates: exit status 1

what else can I try ?

curl -v https://zung102-olympic.nord
*   Trying 100.122.127.103:443...
* Connected to zung102-olympic.nord (100.122.127.103) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.0 (OUT), TLS header, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS header, Finished (20):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS header, Unknown (21):
* TLSv1.3 (OUT), TLS alert, unknown CA (560):
* SSL certificate problem: unable to get local issuer certificate
* Closing connection 0
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

I also tried to start caddy container with sudo (sudo docker compose up -d) but got same errors.

1 Like
17

Oh you’re in Docker, sorry. Instructions for that are here: Keep Caddy Running — Caddy Documentation

1 Like
18

tried this …

sudo docker compose cp     caddy:/data/caddy/pki/authorities/local/root.crt     /usr/local/share/ca-certificates/root.crt   && sudo update-ca-certificates
[+] Copying 1/0
 ✔ caddy copy caddy:/data/caddy/pki/authorities/local/root.crt to /usr/local/share/ca-certificates/root.crt Copied0.0s 
Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.

however got new errors …

curl -v https://zung102-olympic.nord:8080
*   Trying 100.122.127.103:8080...
* Connected to zung102-olympic.nord (100.122.127.103) port 8080 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.0 (OUT), TLS header, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* (5454) (IN), , Unknown (72):
* error:0A00010B:SSL routines::wrong version number
* Closing connection 0
curl: (35) error:0A00010B:SSL routines::wrong version number

I have searched around wrong version number solution but nothing exactly matching my case

19

Why are you making a request to port 8080? Your docker-compose.yml had Caddy bound to port 443 for HTTPS.

20

Sorry for the confusion. I am experimenting 2 instances of vaultwarden containers with Caddy container concurrently One instance has Caddy reverse proxy on public accessible domain name (mapped to public IP address) and it worked as expected with HTTPS. The other instance attempted to reverse proxy on localhost with TLS internal on local domain name -zung102-olympic.nord- (mapped to local IP address) and hopefully works with HTTPS. For the second instance I must use 8080:80 to avoid the collision with the 1st one.

Now I just ran one instance of vaultwarden (80:80) and use TLS internal for local HTPPS. Below are the results from curl command…

curl -v https://zung102-olympic.nord 
*   Trying 100.122.127.103:443...
* Connected to zung102-olympic.nord (100.122.127.103) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.0 (OUT), TLS header, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS header, Finished (20):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.2 (OUT), TLS header, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: [NONE]
*  start date: Jun 22 19:21:59 2023 GMT
*  expire date: Jun 23 07:21:59 2023 GMT
*  subjectAltName: host "zung102-olympic.nord" matched cert's "zung102-olympic.nord"
*  issuer: CN=Caddy Local Authority - ECC Intermediate
*  SSL certificate verify ok.
* Using HTTP2, server supports multiplexing
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* Using Stream ID: 1 (easy handle 0x561e0a27a560)
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
> GET / HTTP/2
> Host: zung102-olympic.nord
> user-agent: curl/7.81.0
> accept: */*
> 
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* Connection state changed (MAX_CONCURRENT_STREAMS == 250)!
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
< HTTP/2 308 
< alt-svc: h3=":443"; ma=2592000
< date: Fri, 23 Jun 2023 02:37:47 GMT
< location: https://zung102-olympic.nord/
< strict-transport-security: max-age=31536000;
< x-frame-options: DENY
< x-robots-tag: none
< x-xss-protection: 1; mode=block
< content-length: 0
< 
* Connection #0 to host zung102-olympic.nord left intact

Should I expect the contents of the web page?

From a browser I got this error ERR_TOO_MANY_REDIRECTS