Help with sub-subdomain with Docker

Help
1

Hello everyone!
I’m trying to get Caddy working on my little rock64 at home with OMV (arm64v8 and Docker).
I already have a Caddy configured on a Windows vps and it works wonder, I don’t know if this is because on my vps I configured only plain subdomains, btw.

I’m trying to follow this schema: subdomain.domain.com for my Windows vps and sub.home.domain.com for my OMV device.

I also have Cloudflare for my domain since I had Namecheap but it was way too slow to propagate DNS changes (sometimes when I created some TXT domain for SSL verification, Namecheap never propagated them). CF is configured in Full (Strict) SSL mode and I always try to non use its CDN.

Here’s a portion of the configuration on my Windows VPS that works:

subdomain.domain.com {
    gzip
    proxy / 127.0.0.1:5001 {
           transparent
    }
 }

This generates the certificate, and then correctly sets http://subdomain.domain.com to https://subdomain.domain.com

This, instead, is a portion of the configuration of a domain on my OMV device:

subdomain.home.domain.com {
    gzip
    proxy / 192.168.1.140:8089 {
        transparent
    }
}

If I leave as this is, it correctly verify the certificate, then the logs says that Caddy activated http://subdomain.home.domain.com:2015 and https://subdomain.home.domain.com:2015 (why 2015? Shouldn’t it be 80 and 443 as the other domain?) but, when I go to that domain: 404 Site subdomain.home.domain.com is not served on this interface when not using https, and ERR_SSL_PROTOCOL_ERROR when using it.

If, instead I put

tls {
	dns cloudflare
}

(if I activate the CDN on CF) on my Windows vps it gets correctly the certificate from CF, but instead if using it at home it gives ERR_SSL_PROTOCOL_ERROR, instead :smiley:

Before I didn’t care, so I used to prepone http:// in front of the domain and used tls off, so Caddy didn’t activate the SSL and it worked with the port 80 flawlessly.

Now, however, since I’m trying to host a little Telegram bot that needs https, I need that functionality.

Can someone of you help me?

What I’m trying to achieve is correct https and 80/443 ports.

Thank you!

(Sorry if I’m not clear, of course)

2

Something is very wrong if Caddy’s outputting that. Caddy can’t multiplex a HTTPS and a non-HTTPS listener on the same port - in fact, if you try to explicitly configure it to do so, it will throw an error about it!

Can you try start it up again and then copy+paste the logs that show this? Also, your caddy -version would be appreciated.

The syntax you’ve given is correct for both subdomain.domain.com and subdomain.home.domain.com.

Port 2015 is the default non-production port Caddy uses. It will fall back to this port for a site that is not eligible for Automatic HTTPS, unless you override the port.

Yeah, this is very strange, can’t say I’ve ever seen this particular issue (startup logs claiming to multiplex listeners, then neither listener actually serves the site in question).

If you can give us the actual logs, that might shed some light. Other than that, you might try to explicitly specify the HTTP and HTTPS ports:

http://subdomain.home.domain.com:80, https://subdomain.home.domain.com:443 {
    gzip
    proxy / 192.168.1.140:8089 {
        transparent
    }
}
3

Sure.
For sake of simplicity I’ll use only two domains:

subd-1.home.domain.com {
    gzip
    proxy / 192.168.1.140:8080 {
        transparent
    }
    tls myemail@gmail.com
}

http://subd-2.home.domain.com:80, https://subd-2.home.domain.com:443 {
    gzip
    proxy / 192.168.1.140:8081 {
        transparent
    }
    tls myemail@gmail.com
}

Here’s the log copypasted (of course I edited the domain info and email):

Activating privacy features... 
2018/09/24 17:59:40 [INFO] acme: Registering account for myemail@gmail.com
2018/09/24 17:59:41 [INFO][subd-1.home.domain.com] acme: Obtaining bundled SAN certificate
2018/09/24 17:59:41 [INFO][subd-1.home.domain.com] AuthURL: https://acme-v02.api.letsencrypt.org/acme/authz/HblayaVIfOJxgBiIBzq6Gsi4naxWsquluBQIsuARsiI
2018/09/24 17:59:41 [INFO][subd-1.home.domain.com] acme: Trying to solve HTTP-01
2018/09/24 17:59:42 [INFO][subd-1.home.domain.com] Served key authentication
2018/09/24 17:59:47 [INFO][subd-1.home.domain.com] The server validated our request
2018/09/24 17:59:47 [INFO][subd-1.home.domain.com] acme: Validations succeeded; requesting certificates
2018/09/24 17:59:59 [INFO][subd-1.home.domain.com] Server responded with a certificate.
2018/09/24 17:59:59 [INFO][subd-1.home.domain.com] Certificate written to disk: /root/.caddy/acme/acme-    v02.api.letsencrypt.org/sites/subd-1.home.domain.com/subd-1.home.domain.com.crt
2018/09/24 17:59:59 [INFO][subd-2.home.domain.com] acme: Obtaining bundled SAN certificate
2018/09/24 18:00:00 [INFO][subd-2.home.domain.com] AuthURL: https://acme-v02.api.letsencrypt.org/acme/authz/GecMQzd_A92O1SVAXC--    BvD0FKTiap_xHJ4pqXkXWSA
2018/09/24 18:00:00 [INFO][subd-2.home.domain.com] acme: Trying to solve HTTP-01
2018/09/24 18:00:00 [INFO][subd-2.home.domain.com] Served key authentication
2018/09/24 18:00:05 [INFO][subd-2.home.domain.com] The server validated our request
2018/09/24 18:00:05 [INFO][subd-2.home.domain.com] acme: Validations succeeded; requesting certificates
2018/09/24 18:00:20 [INFO][subd-2.home.domain.com] Server responded with a certificate.
2018/09/24 18:00:20 [INFO][subd-2.home.domain.com] Certificate written to disk: /root/.caddy/acme/acme-    v02.api.letsencrypt.org/sites/subd-2.home.domain.com/subd-2.home.domain.com.crt
done.
2018/09/24 18:00:20 [WARNING] TLS disabled for http://subd-2.home.domain.com
https://subd-1.home.domain.com
2018/09/24 18:00:20 https://subd-1.home.domain.com
https://subd-2.home.domain.com
2018/09/24 18:00:20 https://subd-2.home.domain.com
http://subd-2.home.domain.com
2018/09/24 18:00:20 http://subd-2.home.domain.com
http://subd-1.home.domain.com
2018/09/24 18:00:20 http://subd-1.home.domain.com

Here’s the output for caddy -version:

Caddy 0.11.0 (non-commercial use only)

When I visit subd-1.home.domain.com it correcly redirects me to HTTPS but I get ERR_SSL_PROTOCOL_ERROR
With subd-2.home.domain.com I can ge to the plain HTTP page (Chrome says “Not secure” but it works), instead going with HTTPS ERR_SSL_PROTOCOL_ERROR

Tell me if you need something more!

4

How is your Docker container configured?

Can you run curl -kIL subd-1.home.domain.com and tell us what you get?

5

Sure, I used various containers I found online but I ended creating one by myself because I always got the same problems.
Here’s my Dockerfile, if I recall correctly I edited one I found online:

FROM arm64v8/alpine:3.6
LABEL maintainer "Bonny1992"

LABEL caddy_version="0.11.0" architecture="arm64"

ARG plugins=dyndns,http.cache,http.cgi,http.cors,http.expires,http.git,http.minify,tls.dns.cloudflare,tls.dns.digitalocean,tls.dns.namecheap

RUN apk add --no-cache openssh-client git tar curl

RUN curl --silent --show-error --fail --location \
      --header "Accept: application/tar+gzip, application/x-gzip, application/octet-stream" -o - \
      "https://caddyserver.com/download/linux/arm64?plugins=${plugins}&license=personal&telemetry=off" \
    | tar --no-same-owner -C /usr/bin/ -xz caddy \
 && chmod 0755 /usr/bin/caddy \
 && mkdir /etc/caddyfile \
 && /usr/bin/caddy -version

EXPOSE 80 443 2015
VOLUME /root/.caddy /srv
WORKDIR /srv

ENTRYPOINT ["/usr/bin/caddy"]
CMD ["--conf", "/etc/caddyfile/Caddyfile", "--log", "stdout", "-agree"]

Its volumes are:

/etc/caddyfile/Caddyfile
/root/.caddy
/srv

The environment variables I set:

CLOUDFLARE_EMAIL
CLOUDFLARE_API_KEY

And here’s the output of the command:

curl -kIL subd-1.home.domain.com

HTTP/1.1 301 Moved Permanently
Connection: close
Content-Type: text/html; charset=utf-8
Location: https://subd-1.home.domain.com/
Server: Caddy
Date: Wed, 03 Oct 2018 11:16:43 GMT

curl: (35) error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol
6

Do you have the exact docker run command you used handy?

Try next: curl -kIL http://subd-1.home.domain.com:443/

7

Sadly not, I used a mix of the docker run command and Portainer.
However I remember setting the --network host argument.

Here’s the output:

curl -kIL https://subd-1.home.domain.com:443

curl: (35) error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol
8

No, sorry, you ran the wrong command.

I specified HTTP, not HTTPS, on port 443.

curl -kIL http://subd-1.home.domain.com:443/

If you ran --network host, then the exposed or published ports won’t matter.

9

Oh, sorry, didn’t notice.
Here’s the correct one:

curl -kIL http://subd-1.home.domain.com:443      
           
HTTP/1.1 301 Moved Permanently
Connection: close
Content-Type: text/html; charset=utf-8
Location: https://subd-1.home.domain.com/
Server: Caddy
Date: Wed, 03 Oct 2018 11:36:00 GMT

curl: (35) error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol
10

OK, I wondered, but that’s proof. A request on port 443 is actually being received by Caddy on the HTTP listener, not the HTTPS listener.

Most likely cause is your router’s external port 443 is forwarded to port 80 instead.

11

You were right! I thought that setting as a rule to forward ports 80, 443 to my internal ip’s ports 80, 443 would automatically decide what port to use.

I just separated the rules (one for port 80 and one for port 443) and it works!

However, this is only for domains with Let’s Encrypt automatic tls.
Using Cloudflare’s one, I still get ERR_SSL_VERSION_OR_CIPHER_MISMATCH, any ideas on why it happens?

12

When you say, “Using Cloudflare’s one”, what exactly are you referring to? tls dns or orange-cloud?

13

I refer to this:

tls { 	
   dns cloudflare
}

Here’s the Caddyfile portion for this domain:

 cloudflared.home.domain.com {
	gzip
	proxy / 192.168.1.140:8079 {
		transparent
	}
	tls {
		dns cloudflare
	}

}

And here’s the reply for curl -kIL cloudflared.home.domain.com:

curl -kIL cloudflared.home.domain.com

HTTP/1.1 301 Moved Permanently
Date: Wed, 03 Oct 2018 11:47:36 GMT
Content-Type: text/html; charset=utf-8
Connection: keep-alive
Set-Cookie: __cfduid=d8a9724d164bfc8232cd480bac1a417751538567256; expires=Thu, 03-Oct-19 11:47:36 GMT; path=/; domain=.domain.com; HttpOnly
Location: https://cloudflared.home.domain.com/
Server: cloudflare
CF-RAY: 463f2cc8d75fbe48-MXP

curl: (35) error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure
14

This is running on the same Caddy instance as the other sites above, behind the same router?

15

Yes.

Same Caddy instance, same Docker container, same device running it (a Rock64 board with OMV as OS), same IP address and same router.

16

OK, interesting.

This next command is a bit complicated, but can we get the output of:

echo | openssl s_client -showcerts -servername cloudflared.home.domain.com -connect cloudflared.home.domain.com:443 2>/dev/null | openssl x509 -inform pem -noout -text

17

Sure, here’s the output:

echo | openssl s_client -showcerts -servername cloudflared.home.domain.com -connect cloudflared.home.domain.com:443 2>/dev/null | openssl x509 -inform pem -noout -text

unable to load certificate
548496079312:error:0906D06C:PEM routines:PEM_read_bio:no start line:../crypto/pem/pem_lib.c:691:Expecting: TRUSTED CERTIFICATE

(I admire your command knowledge lol)

EDIT: I have to point out that Caddy on the Windows vps handles correctly the Cloudflare TLS option.

18

Hah, cheers. Comes with regularly troubleshooting networking and web servers.

That result is… weird. I’m inclined to blame Cloudflare itself:

Server: cloudflare
CF-RAY: 463f2cc8d75fbe48-MXP

The smoking gun, per se, them being in the middle. They’re not handing out a valid certificate to HTTPS requests (I’m not sure what they’re handing out, but it sure as heck aint standard).

Grey-cloud cloudflared.home.domain.com in your account control panel, double check the DNS has propagated, and try the curl again once it resolves directly to Caddy without CF in the way. We can try to rule out Caddy itself as the source of this problem.

19

Should I also remove the tls cloudflare definition from the Caddyfile?

EDIT: In this way:

cloudflared.home.domain.com {
	gzip
	proxy / 192.168.1.140:8079 {
		transparent
	}
	tls myemail@domain.com

}
20

No - that has nothing to do with Cloudflare as a revproxy and simply changes the method Caddy uses to verify itself to LetsEncrypt. Externally, a client should see no difference.

Leave it in, and if the issue persists, we might then infer that DNS validation has some part in the issue; again, though, my hunch is that the orange-cloud is the problem actor here.