Docker image built with xcaddy gives me error:14094438:SSL routines:ssl3_read_bytes:tlsv1

1. Caddy version (caddy version):

/srv # caddy version
v2.4.6 h1:HGkGICFGvyrodcqOOclHKfvJC0qTU7vny/7FhYp9hNw=

2. How I run Caddy:

with docker-composer and docker file:


FROM caddy:2-builder AS builder

RUN xcaddy build \
    --with github.com/caddy-dns/cloudflare \
    --with github.com/caddy-dns/duckdns

FROM caddy:2

COPY --from=builder /usr/bin/caddy /usr/bin/caddy

ENTRYPOINT ["/usr/bin/caddy"]
CMD ["docker-proxy"]
version: '3'

services:
  vaultwarden:
    image: vaultwarden/server:latest
    container_name: vw_vaultwarden
    restart: always
    environment:
      - SIGNUPS_ALLOWED=true
      - INVITATIONS_ALLOWED=false
      - WEBSOCKET_ENABLED=true
      - LOG_FILE=/var/log/docker/bitwarden.log
    volumes:
      - ./vaultwarden/data:/data

  caddy:
    image: mg/caddy:2
    build:
      context: .
    container_name: vw_caddy
    restart: always
    ports:
      - 80:80  # Needed for the ACME HTTP-01 challenge.
      - 443:443
    volumes:
      - ./Caddyfile:/etc/caddy/Caddyfile:ro
      - ./caddy/config:/config
      - ./caddy/data:/data
    environment:
      - DOMAIN=https://mgcl1n1.duckdns.org
      - DUCKDNS_TOKEN=<mytocken>
      - LOG_FILE=/data/access.log

a. System environment:

Running on Manjaro:
Linux mgcl1n32 5.15.19-1-MANJARO #1 SMP PREEMPT Tue Feb 1 16:58:17 UTC 2022 x86_64 GNU/Linux

docker version:

Client:
 Version:           20.10.12
 API version:       1.41
 Go version:        go1.17.5
 Git commit:        e91ed5707e
 Built:             Mon Dec 13 22:31:40 2021
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server:
 Engine:
  Version:          20.10.12
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.17.5
  Git commit:       459d0dfbbb
  Built:            Mon Dec 13 22:30:43 2021
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.5.9
  GitCommit:        1407cab509ff0d96baa4f0eb6ff9980270e6e620.m
 runc:
  Version:          1.1.0
  GitCommit:        v1.1.0-0-g067aaf85
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

b. Command:

docker exec -it vw_caddy ps -eaf
PID   USER     TIME  COMMAND
    1 root      0:00 caddy run --config /etc/caddy/Caddyfile --adapter caddyfil
   15 root      0:00 sh
   29 root      0:00 ps -eaf
Paste command here.

c. Service/unit/compose file:

Paste full file contents here.
Make sure backticks stay on their own lines,
and the post looks nice in the preview pane.

d. My complete Caddyfile or JSON config:

{$DOMAIN}:443 {
  log {
    level DEBUG
    output file {$LOG_FILE} {
      roll_size 10MB
      roll_keep 10
    }
  }

  # Use the ACME HTTP-01 challenge to get a cert for the configured domain.
  tls {
    dns duckdns {$DUCKDNS_TOKEN}
  }

  # This setting may have compatibility issues with some browsers
  # (e.g., attachment downloading on Firefox). Try disabling this
  # if you encounter issues.
  encode gzip

  # Notifications redirected to the WebSocket server
  reverse_proxy /notifications/hub vw_vaultwarden:3012

  # Proxy everything else to Rocket
  reverse_proxy vw_vaultwarden:80 {
       # Send the true remote IP to Rocket, so that vaultwarden can put this in the
       # log, so that fail2ban can ban the correct IP.
       header_up X-Real-IP {remote_host}
  }
}
Paste config here, replacing this text.
Use `caddy fmt` to make it readable.
DO NOT REDACT anything except credentials.
LEAVE DOMAIN NAMES INTACT.
Make sure the backticks stay on their own lines.

3. The problem I’m having:

curl command fails with the following error:

$curl -ikvL https://localhost
*   Trying 127.0.0.1:443...
* Connected to localhost (127.0.0.1) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS alert, internal error (592):
* error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error
* Closing connection 0
curl: (35) error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error

4. Error messages and/or full log output:

caddy container log:

{"level":"info","ts":1645178530.6221776,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":"caddyfile"}
{"level":"warn","ts":1645178530.6233735,"msg":"input is not formatted with 'caddy fmt'","adapter":"caddyfile","file":"/etc/caddy/Caddyfile","line":2}
{"level":"info","ts":1645178530.6240795,"logger":"admin","msg":"admin endpoint started","address":"tcp/localhost:2019","enforce_origin":false,"origins":["localhost:2019","[::1]:2019","127.0.0.1:2019"]}
{"level":"info","ts":1645178530.6242585,"logger":"http","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443}
{"level":"info","ts":1645178530.6242585,"logger":"http","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
{"level":"info","ts":1645178530.6244261,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc000302b60"}
{"level":"info","ts":1645178530.624526,"logger":"http","msg":"enabling automatic TLS certificate management","domains":["mgcl1n1.duckdns.org"]}
{"level":"info","ts":1645178530.6245608,"logger":"tls","msg":"cleaning storage unit","description":"FileStorage:/data/caddy"}
{"level":"info","ts":1645178530.6249325,"logger":"tls","msg":"finished cleaning storage units"}
{"level":"info","ts":1645178530.625241,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
{"level":"info","ts":1645178530.6252549,"msg":"serving initial configuration"}

5. What I already tried:

I have tried to user --insecure option with curl:

curl --trace - --insecure https://localhost/
== Info:   Trying 127.0.0.1:443...
== Info: Connected to localhost (127.0.0.1) port 443 (#0)
== Info: ALPN, offering h2
== Info: ALPN, offering http/1.1
=> Send SSL data, 5 bytes (0x5)
0000: 16 03 01 02 00                                  .....
== Info: TLSv1.3 (OUT), TLS handshake, Client hello (1):
=> Send SSL data, 512 bytes (0x200)
0000: 01 00 01 fc 03 03 97 06 72 be 3b ac 88 ab cf be ........r.;.....
0010: 5a 6c 96 14 87 6d c5 9b 60 51 d4 2b a4 1a 84 37 Zl...m..`Q.+...7
0020: b3 dc 88 90 4a 47 20 49 6d a9 ed cb 82 58 48 6c ....JG Im....XHl
0030: 58 7a dc 43 e8 4b fd 31 49 1d 14 49 2b 58 ad 5f Xz.C.K.1I..I+X._
0040: 1f 29 c7 5a fa e3 72 00 3e 13 02 13 03 13 01 c0 .).Z..r.>.......
0050: 2c c0 30 00 9f cc a9 cc a8 cc aa c0 2b c0 2f 00 ,.0.........+./.
0060: 9e c0 24 c0 28 00 6b c0 23 c0 27 00 67 c0 0a c0 ..$.(.k.#.'.g...
0070: 14 00 39 c0 09 c0 13 00 33 00 9d 00 9c 00 3d 00 ..9.....3.....=.
0080: 3c 00 35 00 2f 00 ff 01 00 01 75 00 00 00 0e 00 <.5./.....u.....
0090: 0c 00 00 09 6c 6f 63 61 6c 68 6f 73 74 00 0b 00 ....localhost...
00a0: 04 03 00 01 02 00 0a 00 0c 00 0a 00 1d 00 17 00 ................
00b0: 1e 00 19 00 18 33 74 00 00 00 10 00 0e 00 0c 02 .....3t.........
00c0: 68 32 08 68 74 74 70 2f 31 2e 31 00 16 00 00 00 h2.http/1.1.....
00d0: 17 00 00 00 31 00 00 00 0d 00 30 00 2e 04 03 05 ....1.....0.....
00e0: 03 06 03 08 07 08 08 08 09 08 0a 08 0b 08 04 08 ................
00f0: 05 08 06 04 01 05 01 06 01 03 03 02 03 03 01 02 ................
0100: 01 03 02 02 02 04 02 05 02 06 02 00 2b 00 09 08 ............+...
0110: 03 04 03 03 03 02 03 01 00 2d 00 02 01 01 00 33 .........-.....3
0120: 00 26 00 24 00 1d 00 20 dc 80 c0 81 fe f3 04 4b .&.$... .......K
0130: 75 5d 05 3b 2d 9b c5 8c 28 1a 9e 94 4a 43 e9 c6 u].;-...(...JC..
0140: 4a 69 43 20 10 4f 0d 0e 00 15 00 b4 00 00 00 00 JiC .O..........
0150: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0160: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0170: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0180: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0190: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
01a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
01b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
01c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
01d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
01e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
01f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
<= Recv SSL data, 5 bytes (0x5)
0000: 15 03 03 00 02                                  .....
== Info: TLSv1.3 (IN), TLS alert, internal error (592):
<= Recv SSL data, 2 bytes (0x2)
0000: 02 50                                           .P
== Info: error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error
== Info: Closing connection 0
curl: (35) error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error

I have also checked certificates in caddy container:

/data/caddy/certificates/acme-v02.api.letsencrypt.org-directory/mgcl1n1.duckdns.org # pwd
/data/caddy/certificates/acme-v02.api.letsencrypt.org-directory/mgcl1n1.duckdns.org
/data/caddy/certificates/acme-v02.api.letsencrypt.org-directory/mgcl1n1.duckdns.org # ls -ltr
total 16
-rw-------    1 root     root           227 Feb 18 08:00 mgcl1n1.duckdns.org.key
-rw-------    1 root     root           158 Feb 18 08:00 mgcl1n1.duckdns.org.json
-rw-------    1 root     root          5329 Feb 18 08:00 mgcl1n1.duckdns.org.crt

6. Links to relevant resources:

You configured Caddy with your DuckDNS domain, but made a request with localhost. That won’t work. You need to make a request with a domain name that Caddy knows about, because Caddy has to serve a TLS certificate that matches the domain in the request. This is how trust is established between the client and server - the client checks that the certificate was issued for the domain it made the request for, and that it was issued by a trusted certificate authority (i.e. Let’s Encrypt).

Thanks @francislavoie I have just put an entry in /etc/hosts and used the correct DuckDNS domain and it works.
I have just copied the same configuration in my RBPI4 which has aarch64 architecture.
Here the caddy container fails with the following error:

[ERROR] 'docker-proxy' is not a recognized subcommand; see 'caddy help'

What is the reason? Can this error be avoided?

You didn’t build Caddy with the caddy-docker-proxy plugin:

If you’re using a Caddyfile though, you don’t need to use docker-proxy at all. Just remove the CMD and ENTRYPOINT lines from your Dockerfile, as per the instructions on Docker, in the section titled “Adding custom Caddy modules”

Thanks @francislavoie I could make it working.
I have few more questions if you have a couple of minutes:

  1. Why the configuration is different between x86_64 and aarch64?
  2. Can you recommend me a easy solution for Android devices that I use in my home network?

I don’t understand the question. What do you mean?

You can set up a DNS server inside your home network to resolve your duckdns domain to your LAN IP of your server, implementing split-DNS (so devices outside the network use the WAN IP, devices inside the network use the LAN IP).

You could use CoreDNS for this, a DNS server also written in Go, a cousin of Caddy (CoreDNS was built on top of an older version of Caddy v1)

Regarding the first question, the following configruation works in x86_64:

FROM caddy:2-builder AS builder

RUN xcaddy build \
    --with github.com/caddy-dns/cloudflare \
    --with github.com/caddy-dns/duckdns

FROM caddy:2

COPY --from=builder /usr/bin/caddy /usr/bin/caddy

but in aarch64 needs to be changed to:

FROM caddy:2-builder AS builder

RUN xcaddy build \
    --with github.com/lucaslorentz/caddy-docker-proxy/plugin \
    --with github.com/caddy-dns/cloudflare \
    --with github.com/caddy-dns/duckdns

FROM caddy:2

COPY --from=builder /usr/bin/caddy /usr/bin/caddy

I would like to understand why.

You only need github.com/lucaslorentz/caddy-docker-proxy/plugin if you plan to use the caddy docker-proxy command, to configure Caddy dynamically via Docker labels. If you plan to use a Caddyfile, then you don’t need that plugin at all, and you can remove it. It’s not required at all.

This topic was automatically closed after 30 days. New replies are no longer allowed.