Cannot start Caddy via Docker

Help
20

zung@Dzungabc:/etc$ curl --insecure https://localhost -v

  • Trying 127.0.0.1:443…
  • TCP_NODELAY set
  • Connected to localhost (127.0.0.1) port 443 (#0)
  • ALPN, offering h2
  • ALPN, offering http/1.1
  • successfully set certificate verify locations:
  • CAfile: /etc/ssl/certs/ca-certificates.crt
    CApath: /etc/ssl/certs
  • TLSv1.3 (OUT), TLS handshake, Client hello (1):
  • TLSv1.3 (IN), TLS alert, internal error (592):
  • error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error
  • Closing connection 0
    curl: (35) error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error

zung@Dzungabc:/etc$ curl http://localhost:8080 -v

  • Trying 127.0.0.1:8080…
  • TCP_NODELAY set
  • Connected to localhost (127.0.0.1) port 8080 (#0)

GET / HTTP/1.1
Host: localhost:8080
User-Agent: curl/7.68.0
Accept: /

  • Mark bundle as not supporting multiuse
    < HTTP/1.1 200 OK
    < content-type: text/html; charset=utf-8
    < cache-control: public, max-age=600
    < expires: Fri, 06 Jan 2023 14:42:33 GMT
    < server: Rocket
    < x-frame-options: SAMEORIGIN
    < x-content-type-options: nosniff
    < permissions-policy: accelerometer=(), ambient-light-sensor=(), autoplay=(), battery=(), camera=(), display-capture=(), document-domain=(), encrypted-media=(), execution-while-not-rendered=(), execution-while-out-of-viewport=(), fullscreen=(), geolocation=(), gyroscope=(), keyboard-map=(), magnetometer=(), microphone=(), midi=(), payment=(), picture-in-picture=(), screen-wake-lock=(), sync-xhr=(), usb=(), web-share=(), xr-spatial-tracking=()
    < referrer-policy: same-origin
    < x-xss-protection: 0
    < content-security-policy: default-src ‘self’; base-uri ‘self’; form-action ‘self’; object-src ‘self’ blob:; script-src ‘self’; style-src ‘self’ ‘unsafe-inline’; child-src ‘self’ https://.duosecurity.com https://.duofederal.com; frame-src ‘self’ https://.duosecurity.com https://.duofederal.com; frame-ancestors ‘self’ chrome-extension://nngceckbapebfimnlniiiahkandclblb chrome-extension://jbkfoedolllekgbhcbcoahefnbanhhlh moz-extension://* ; img-src ‘self’ data: https://haveibeenpwned.com https://www.gravatar.com ; connect-src ‘self’ https://api.pwnedpasswords.com https://2fa.directory https://app.simplelogin.io/api/ https://app.anonaddy.com/api/ https://api.fastmail.com/ ;
    < content-length: 1240
    < date: Fri, 06 Jan 2023 14:32:33 GMT
    <
    <!doctype html>Vaultwarden Web Vault
    Bitwarden

by the way port forwarding as seen below …

PS C:\Users\user1> netsh interface portproxy show v4tov4

Listen on ipv4: Connect to ipv4:

Address Port Address Port

192.168.0.146 80 172.22.221.184 80
192.168.0.146 443 172.22.221.184 443
192.168.0.146 8080 172.22.221.184 8080
192.168.0.146 8443 172.22.221.184 8443

21

In attempt to resolve your issue, I install Caddy Docker on my WSL2 as well.
Strangely, I get the same ERR_SSL_PROTOCOL_ERROR.

Then I realize a mistake
If your Caddyfile is

dzung.duckdns.org {
    tls internal
    respond "Hello World!"
}

SSL certificate is issue to dzung.duckdns.org , not localhost or 127.0.0.1
Hence , if your browse to https://localhost or https://127.0.0.1 , It will product SSL error (duh!)

dzung.duckdns.org need to point it to your host IP locally within your LAN.
Then try browsing it to https://dzung.duckdns.org

You can add localhost in Caddyfile as well

localhost dzung.duckdns.org {
    tls internal
    respond "Hello World!"
}

If add 127.0.0.1 to host htts://127.0.0.1 ( localhost dzung.duckdns.org 127.0.0.1 { ) will cause SSL issue again due to Docker route back restriction

To reload the new Caddyfile config, run this docker exec -it CADDY caddy reload --config /etc/caddy/Caddyfile ( where CADDY is your container name, if is all UPPERCASE )

image
image928×649 140 KB

22

I didn’t perform any port forward on my host PC.
I am guessing newer Windows build resolve the issue.
Just giving you a head up.

23

The fact I can get to Vaultwarden sign-on screen by dzung.duckdns.org:8080 indicated that dzung.duckdns.org does use my public IP and was forwarded to Vaultwarden residing on local IP 192.168.0146:8080.

Looked like Caddy Docker failed various challenges (TLS-APN, HTTP-01,etc…) made me tried to do the following …
tls {
dns duckdns {
api_token 25c688fb-8422-480c-adc7-3c83a2ace566
}
}
reverse-proxy * 192.168.0.146:8080

Caddy log showed that module dns.provider.duckdns is not registered. I found this website Module dns.providers.duckdns - Caddy Documentation - https://caddyserver.com/ but note sure how to generate and add the module to Caddy Docker.

I then tried to generate certificates manually using acme.sh as below …
export DuckDNS_Token=“aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee”
acme.sh --insecure --issue --dns dns_duckdns -d dzung.duckdns.org

It appeared to generate cer and key OK. However when I specified them in Caddyfile …
tls internal {$HOME/.acme.sh/dzung.duckdns.org/dzung.duckdns.org.cer} {$HOME/.acme.sh/dzung.duckdns.org/dzung.duckdns.ord.key}
reverse_proxy * 192.168.0.146:8080

Yet https://dzung.duckdns.org still timed out.

24

Actually I removed “internal” keyword from Caddyfile but it did not improve the situation

25

@zung102 I hope that’s not your actual DuckDNS API token you posted above. Make sure to regenerate your API token if so. You can do that from the top-right of the DuckDNS website, next to your email address, there’s a button to let you “recreate token”.

Caddy doesn’t ship with DNS plugins. You need to download a build of Caddy with that plugin added to use it. See this article for details:

When posting your config and logs to the forums, please use code block syntax. There’s a </> button to help you do that; you need to use triple-backticks on their own lines, before and after the text. This is important so what you post isn’t misinterpreted as other content, which makes it very difficult to read.

26

I just realize your port forward for WSL2, actually forward both port 80 and 443, to your Vaultwarden, not Caddy.
This explain 443 error.

In your WSL2 distribution, you can perform docker network inspect bridge to find out, or if your caddy container name is CADDY , then in your WSL2 docker exec -it CADDY caddy ifconfig

27

Wow, I have been using duckdns so many year, and I never notice and realize API token regeneration is possible. Thank you Francis !

Picture attached for anyone who want to know

image
image1280×749 51.4 KB

28

Preformatted text Hi Francislavoie, thank you very much for pointing out the way to use DNS API…I need to investigate deeper to know how to install it in Caddy docker environment using method 1. After downloading it what do I do next?Preformatted text

29

I have fixed up the port forwarding for Caddy

Listen on ipv4:             Connect to ipv4:

Address         Port        Address         Port
--------------- ----------  --------------- ----------
192.168.0.146   80          172.17.0.3      80
192.168.0.146   443         172.17.0.3      443
192.168.0.146   8080        172.22.221.184  8080
192.168.0.146   8443        172.22.221.184  8443

However the issue still remains ...
2023-01-07 12:07:36 {"level":"error","ts":1673111256.4127765,"logger":"tls.obtain","msg":"will retry","error":"[dzung.duckdns.org] Obtain: [dzung.duckdns.org] solving challenges: [dzung.duckdns.org] authorization took too long (order=https://acme.zerossl.com/v2/DV90/order/HQCvzpxlA82Ztrol1I_iQg) (ca=https://acme.zerossl.com/v2/DV90)","attempt":4,"retrying_in":300,"elapsed":2162.33842893,"max_duration":2592000}
2023-01-07 12:07:45 {"level":"info","ts":1673111265.1946387,"logger":"admin.api","msg":"received request","method":"POST","host":"localhost:2019","uri":"/load","remote_ip":"127.0.0.1","remote_port":"59158","headers":{"Accept-Encoding":["gzip"],"Content-Length":["493"],"Content-Type":["application/json"],"Origin":["http://localhost:2019"],"User-Agent":["Go-http-client/1.1"]}}
2023-01-07 12:07:45 {"level":"info","ts":1673111265.2034545,"msg":"config is unchanged"}
2023-01-07 12:07:45 {"level":"info","ts":1673111265.2035413,"logger":"admin.api","msg":"load complete"}
2023-01-07 12:12:36 {"level":"info","ts":1673111556.4091594,"logger":"tls.obtain","msg":"obtaining certificate","identifier":"dzung.duckdns.org"}
2023-01-07 12:12:36 {"level":"info","ts":1673111556.906687,"logger":"http.acme_client","msg":"trying to solve challenge","identifier":"dzung.duckdns.org","challenge_type":"tls-alpn-01","ca":"https://acme-staging-v02.api.letsencrypt.org/directory"}
2023-01-07 12:12:47 {"level":"error","ts":1673111567.5187314,"logger":"http.acme_client","msg":"challenge failed","identifier":"dzung.duckdns.org","challenge_type":"tls-alpn-01","problem":{"type":"urn:ietf:params:acme:error:connection","title":"","detail":"72.140.211.179: Timeout during connect (likely firewall problem)","instance":"","subproblems":[]}}
2023-01-07 12:12:47 {"level":"error","ts":1673111567.518861,"logger":"http.acme_client","msg":"validating authorization","identifier":"dzung.duckdns.org","problem":{"type":"urn:ietf:params:acme:error:connection","title":"","detail":"72.140.211.179: Timeout during connect (likely firewall problem)","instance":"","subproblems":[]},"order":"https://acme-staging-v02.api.letsencrypt.org/acme/order/82178013/6387146193","attempt":1,"max_attempts":3}
2023-01-07 12:12:48 {"level":"info","ts":1673111568.6980333,"logger":"http.acme_client","msg":"trying to solve challenge","identifier":"dzung.duckdns.org","challenge_type":"http-01","ca":"https://acme-staging-v02.api.letsencrypt.org/directory"}
2023-01-07 12:12:58 {"level":"error","ts":1673111578.9793277,"logger":"http.acme_client","msg":"challenge failed","identifier":"dzung.duckdns.org","challenge_type":"http-01","problem":{"type":"urn:ietf:params:acme:error:connection","title":"","detail":"72.140.211.179: Fetching http://dzung

Thank you for being patient in helping me.

30

Did you try what I suggested ? Start something simple to rule out Firewall , IP, routing iport forward issue. If you can’t get Hello World, you
wouldn’t able SSL it. sort off

31

Use this Caddyfile

localhost dzung.duckdns.org {
    tls internal
    respond "Hello World!"
}

Where can I see the respond “Hello World”? from Caddy ip address (172.17.0.3) ? I have tried it and it showed ERR_CONNECTION_TIMED_OUT


zung@Dzungabc:/etc$ docker exec -it caddy ifconfig
eth0      Link encap:Ethernet  HWaddr 02:42:AC:11:00:03
          inet addr:172.17.0.3  Bcast:172.17.255.255  Mask:255.255.0.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:760 errors:0 dropped:0 overruns:0 frame:0

Tried https://localhost yielded ERR_SSL_PROTOCOL_ERROR

32

Can you do
Curl -v http://localhost
Curl -v --insecure https://localhost
ping dzung.duckdns.org
ping 172.17.0.3
ping caddy

33

You should get “Hello World” by browsing or curl to localhost and dzung.duckdns.org with SSL certificate being invalid (due to TLS Internal, which is normal)

The puzzle we trying to solve here:
Can we reach your Caddy inside Docker from LAN?
Answer need to be yes, before can proceed

34

I do not see Hello World anywhere …

zung@Dzungabc:/etc$ curl -v http://localhost
*   Trying 127.0.0.1:80...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 80 (#0)
> GET / HTTP/1.1
> Host: localhost
> User-Agent: curl/7.68.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 308 Permanent Redirect
< Connection: close
< Location: https://localhost/
< Server: Caddy
< Date: Sun, 08 Jan 2023 00:54:53 GMT
< Content-Length: 0
<
* Closing connection 0
zung@Dzungabc:/etc$ curl -v --insecure https://localhost
*   Trying 127.0.0.1:443...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS alert, internal error (592):
* error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error
* Closing connection 0
curl: (35) error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error
zung@Dzungabc:/etc$ ping dzung.duckdns.org
PING dzung.duckdns.org (72.140.211.179) 56(84) bytes of data.
--- dzung.duckdns.org ping statistics ---
143 packets transmitted, 0 received, 100% packet loss, time 147819ms
zung@Dzungabc:/etc$ ping 172.17.0.3
PING 172.17.0.3 (172.17.0.3) 56(84) bytes of data.
^C
--- 172.17.0.3 ping statistics ---
10 packets transmitted, 0 received, 100% packet loss, time 9389ms

zung@Dzungabc:/etc$ ping caddy
ping: caddy: No address associated with hostname
zung@Dzungabc:/etc$
35

We now know Caddy is running and working inside Docker, and is responding at Port80.
Communication 443 somehow break down

Which version of image are you running ? I suggest repull image again

36

I have repulled Caddy 2. With the simple Caddyfile (repsond “Hello World”) the Caddy log still showed errors as below …

2023-01-10 10:37:19 {"level":"info","ts":1673365039.9125206,"logger":"http.acme_client","msg":"trying to solve challenge","identifier":"dzung.duckdns.org","challenge_type":"http-01","ca":"https://acme-v02.api.letsencrypt.org/directory"}
2023-01-10 10:37:20 {"level":"error","ts":1673365040.3430007,"logger":"http.acme_client","msg":"challenge failed","identifier":"dzung.duckdns.org","challenge_type":"http-01","problem":{"type":"urn:ietf:params:acme:error:unauthorized","title":"","detail":"72.140.211.179: Invalid response from http://dzung.duckdns.org/.well-known/acme-challenge/k06_xLHX62drq-vt1J3TKnDyQqlePWUgFQGrhsvy8v8: 404","instance":"","subproblems":[]}}
2023-01-10 10:37:20 {"level":"error","ts":1673365040.3432078,"logger":"http.acme_client","msg":"validating authorization","identifier":"dzung.duckdns.org","problem":{"type":"urn:ietf:params:acme:error:unauthorized","title":"","detail":"72.140.211.179: Invalid response from http://dzung.duckdns.org/.well-known/acme-challenge/k06_xLHX62drq-vt1J3TKnDyQqlePWUgFQGrhsvy8v8: 404","instance":"","subproblems":[]},"order":"https://acme-v02.api.letsencrypt.org/acme/order/910415157/158016935497","attempt":1,"max_attempts":3}
2023-01-10 10:37:20 {"level":"error","ts":1673365040.34332,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"dzung.duckdns.org","issuer":"acme-v02.api.letsencrypt.org-directory","error":"HTTP 403 urn:ietf:params:acme:error:unauthorized - 72.140.211.179: Invalid response from http://dzung.duckdns.org/.well-known/acme-challenge/k06_xLHX62drq-vt1J3TKnDyQqlePWUgFQGrhsvy8v8: 404"}

I have searched around but could not find similar errors posted.
I hope you can guide me to some solution. Thanks

37

Open http://dzung.duckdns.org/.well-known/acme-challenge/k06_xLHX62drq-vt1J3TKnDyQqlePWUgFQGrhsvy8v8 in your browser – you’ll see that it reaches vaultwarden directly and Caddy doesn’t intercept it. That’s a problem. You need to make sure traffic on port 80 is reaching Caddy, not Vaultwarden.

38

Thank you for looking at my problem.

Here is my set up for port forwading …

dzung.duckdns.org uses my public IP , my router has port forwarding as this:

443 -----> 192.168.0.146:8080
80 ---->192.168.0.146:80

where 192.168.0.146 is local ip for Windows (WSL2+Vaultwarden docker)
8080 is the port for Vaultwarden

Also the port forwarding from Windows as below

Listen on ipv4:             Connect to ipv4:

Address         Port        Address         Port
--------------- ----------  --------------- ----------
192.168.0.146   80          172.17.0.3      80
192.168.0.146   443         172.22.221.184  443
192.168.0.146   8080        172.22.221.184  8080
192.168.0.146   8443        172.17.0.3      8443

where 172.22.221.184 is the IP of Ubuntu 22.04 (wsl) and 172.17.0.3 is the Caddy IP.
I tried to ping 172.17.0.3 but could not reach it. Is it normal?

Caddy log showed …

2023-01-10 14:02:31 {"level":"info","ts":1673377351.8486152,"logger":"http.acme_client","msg":"trying to solve challenge","identifier":"dzung.duckdns.org","challenge_type":"http-01","ca":"https://acme-v02.api.letsencrypt.org/directory"}
2023-01-10 14:02:53 {"level":"error","ts":1673377373.302111,"logger":"http.acme_client","msg":"challenge failed","identifier":"dzung.duckdns.org","challenge_type":"http-01","problem":{"type":"urn:ietf:params:acme:error:connection","title":"","detail":"72.140.211.179: Fetching http://dzung.duckdns.org/.well-known/acme-challenge/AUvvRDuqHy-E9d4zeHvmno_8o1h9v9-13rFMTkDnnJs: Connection reset by peer","instance":"","subproblems":[]}}
2023-01-10 14:02:53 {"level":"error","ts":1673377373.3024838,"logger":"http.acme_client","msg":"validating authorization","identifier":"dzung.duckdns.org","problem":{"type":"urn:ietf:params:acme:error:connection","title":"","detail":"72.140.211.179: Fetching http://dzung.duckdns.org/.well-known/acme-challenge/AUvvRDuqHy-E9d4zeHvmno_8o1h9v9-13rFMTkDnnJs: Connection reset by peer","instance":"","subproblems":[]},"order":"https://acme-v02.api.letsencrypt.org/acme/order/910657227/158044631967","attempt":1,"max_attempts":3}
2023-01-10 14:02:53 {"level":"error","ts":1673377373.3026953,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"dzung.duckdns.org","issuer":"acme-v02.api.letsencrypt.org-directory","error":"HTTP 400 urn:ietf:params:acme:error:connection - 72.140.211.179: Fetching http://dzung.duckdns.org/.well-known/acme-challenge/AUvvRDuqHy-E9d4zeHvmno_8o1h9v9-13rFMTkDnnJs: Connection reset by peer"}

they may be new errors
but I do not know how to interpret them.

39

That’s weird. Why not 80 → 80 and 443 → 443?

Both ports 80 and 443 should be routed to Caddy.

Caddy needs to be accessible on ports 80 and 443 for the ACME HTTP and/or TLS-ALPN challenges to properly succeed, and so that it can serve HTTP->HTTPS redirects, and serve your site over HTTPS.

Now it’s failing to connect altogether. Whatever you changed is causing traffic on port 80 to not make its way to Caddy.

Ping uses ICMP, not TCP/UDP so it doesn’t use ports. Pinging will probably not work because of how Docker sets up networking. So that’s probably fine.