Docker with `tls internal` only allows a single IP address (default container IP), all other IP addresses considered insecure certificates

{
  default_sni 192.168.1.42
}

192.168.1.8 {
  respond "Hello"
}

172.17.0.2 {
  respond "Bye"
}

No SNI:

192.168.1.8 (fail - SAN mismatch: 172.17.0.2)
$ echo | openssl s_client -showcerts -connect 192.168.1.8:443 2>/dev/null | openssl x509 -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            8b:4e:95:ff:2d:ce:e7:4b:ec:56:e7:47:ab:a3:f9:9a
        Signature Algorithm: ecdsa-with-SHA256
        Issuer: CN = Caddy Local Authority - ECC Intermediate
        Validity
            Not Before: Sep 13 06:07:31 2020 GMT
            Not After : Sep 13 18:08:31 2020 GMT
        Subject: 
        Subject Public Key Info:
            Public Key Algorithm: id-ecPublicKey
                Public-Key: (256 bit)
                pub:
                    04:96:4d:f1:e4:23:b1:81:d9:0f:ec:cf:d4:c1:de:
                    19:01:b8:13:bb:0b:35:8c:a4:7a:80:45:2d:3c:9f:
                    6a:99:ad:f1:aa:82:65:9b:3d:8b:40:77:7b:24:83:
                    37:3d:c4:72:40:84:80:a7:a2:1e:97:e6:a3:cc:6c:
                    f0:b2:ed:ee:b2
                ASN1 OID: prime256v1
                NIST CURVE: P-256
        X509v3 extensions:
            X509v3 Key Usage: critical
                Digital Signature, Key Encipherment
            X509v3 Extended Key Usage: 
                TLS Web Server Authentication, TLS Web Client Authentication
            X509v3 Subject Key Identifier: 
                95:4B:B7:FB:97:0C:CB:D3:01:E1:C8:A8:54:17:B7:14:6E:2E:67:CF
            X509v3 Authority Key Identifier: 
                keyid:8B:BE:D9:7C:3E:32:B2:A9:24:D8:38:42:5F:7B:89:DE:CA:87:58:88

            X509v3 Subject Alternative Name: 
                IP Address:172.17.0.2
    Signature Algorithm: ecdsa-with-SHA256
         30:45:02:20:55:53:59:f2:8d:50:41:2f:51:8e:6b:19:17:e9:
         fb:68:ea:8e:f9:07:8d:2d:e4:e1:b6:0b:77:7b:3a:de:79:9a:
         02:21:00:a4:b1:11:ec:55:c2:e3:e1:7f:77:01:47:dd:5e:b7:
         b3:f3:e8:d3:2e:e5:83:77:89:c1:bd:93:75:5c:01:0c:b4
-----BEGIN CERTIFICATE-----
MIIBtjCCAVygAwIBAgIRAItOlf8tzudL7FbnR6uj+ZowCgYIKoZIzj0EAwIwMzEx
MC8GA1UEAxMoQ2FkZHkgTG9jYWwgQXV0aG9yaXR5IC0gRUNDIEludGVybWVkaWF0
ZTAeFw0yMDA5MTMwNjA3MzFaFw0yMDA5MTMxODA4MzFaMAAwWTATBgcqhkjOPQIB
BggqhkjOPQMBBwNCAASWTfHkI7GB2Q/sz9TB3hkBuBO7CzWMpHqARS08n2qZrfGq
gmWbPYtAd3skgzc9xHJAhICnoh6X5qPMbPCy7e6yo4GDMIGAMA4GA1UdDwEB/wQE
AwIFoDAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwHQYDVR0OBBYEFJVL
t/uXDMvTAeHIqFQXtxRuLmfPMB8GA1UdIwQYMBaAFIu+2Xw+MrKpJNg4Ql97id7K
h1iIMA8GA1UdEQQIMAaHBKwRAAIwCgYIKoZIzj0EAwIDSAAwRQIgVVNZ8o1QQS9R
jmsZF+n7aOqO+QeNLeThtgt3ezreeZoCIQCksRHsVcLj4X93AUfdXrez8+jTLuWD
d4nBvZN1XAEMtA==
-----END CERTIFICATE-----

curl output:

192.168.1.8 (fail)
$ curl -v https://192.168.1.8
*   Trying 192.168.1.8:443...
* Connected to 192.168.1.8 (192.168.1.8) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: [NONE]
*  start date: Sep 13 06:07:31 2020 GMT
*  expire date: Sep 13 18:08:31 2020 GMT
*  subjectAltName does not match 192.168.1.8
* SSL: no alternative certificate subject name matches target host name '192.168.1.8'
* Closing connection 0
* TLSv1.3 (OUT), TLS alert, close notify (256):
curl: (60) SSL: no alternative certificate subject name matches target host name '192.168.1.8'
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

SNI 192.168.1.8:

172.17.0.2 (fail - SAN mismatch: 192.168.1.8)
$ echo | openssl s_client -showcerts -servername 192.168.1.8 -connect 172.17.0.2:443 2>/dev/null | openssl x509 -text    
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            ab:74:ba:60:11:17:0a:22:7d:80:a0:fa:aa:18:4d:2e
        Signature Algorithm: ecdsa-with-SHA256
        Issuer: CN = Caddy Local Authority - ECC Intermediate
        Validity
            Not Before: Sep 13 06:07:31 2020 GMT
            Not After : Sep 13 18:08:31 2020 GMT
        Subject: 
        Subject Public Key Info:
            Public Key Algorithm: id-ecPublicKey
                Public-Key: (256 bit)
                pub:
                    04:04:05:1f:12:dc:b7:91:36:4d:c5:3b:a6:97:1e:
                    1c:2f:e4:05:f6:28:df:9e:66:75:47:8d:1d:8f:58:
                    07:44:84:4a:99:a9:29:18:fa:d9:c7:c3:b8:cd:8a:
                    c8:26:c7:71:89:eb:90:95:1f:85:41:7c:b2:7a:58:
                    f3:4a:09:88:b8
                ASN1 OID: prime256v1
                NIST CURVE: P-256
        X509v3 extensions:
            X509v3 Key Usage: critical
                Digital Signature, Key Encipherment
            X509v3 Extended Key Usage: 
                TLS Web Server Authentication, TLS Web Client Authentication
            X509v3 Subject Key Identifier: 
                C5:FD:1A:C3:BC:88:A6:CB:65:2E:BD:C8:74:DF:14:F2:61:08:9A:A2
            X509v3 Authority Key Identifier: 
                keyid:8B:BE:D9:7C:3E:32:B2:A9:24:D8:38:42:5F:7B:89:DE:CA:87:58:88

            X509v3 Subject Alternative Name: 
                IP Address:192.168.1.8
    Signature Algorithm: ecdsa-with-SHA256
         30:46:02:21:00:bd:1e:ad:90:7e:91:70:cb:6e:5e:f6:69:cd:
         08:54:8d:fd:62:f2:30:85:a5:85:c8:92:b8:6a:7b:2e:fc:ca:
         a7:02:21:00:f7:6f:6a:25:48:a7:e8:f3:b8:24:a5:bf:20:a2:
         09:ba:9e:41:81:f3:94:f8:d6:e7:fe:2b:e6:81:bc:a1:2e:91
-----BEGIN CERTIFICATE-----
MIIBtzCCAVygAwIBAgIRAKt0umARFwoifYCg+qoYTS4wCgYIKoZIzj0EAwIwMzEx
MC8GA1UEAxMoQ2FkZHkgTG9jYWwgQXV0aG9yaXR5IC0gRUNDIEludGVybWVkaWF0
ZTAeFw0yMDA5MTMwNjA3MzFaFw0yMDA5MTMxODA4MzFaMAAwWTATBgcqhkjOPQIB
BggqhkjOPQMBBwNCAAQEBR8S3LeRNk3FO6aXHhwv5AX2KN+eZnVHjR2PWAdEhEqZ
qSkY+tnHw7jNisgmx3GJ65CVH4VBfLJ6WPNKCYi4o4GDMIGAMA4GA1UdDwEB/wQE
AwIFoDAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwHQYDVR0OBBYEFMX9
GsO8iKbLZS69yHTfFPJhCJqiMB8GA1UdIwQYMBaAFIu+2Xw+MrKpJNg4Ql97id7K
h1iIMA8GA1UdEQQIMAaHBMCoAQgwCgYIKoZIzj0EAwIDSQAwRgIhAL0erZB+kXDL
bl72ac0IVI39YvIwhaWFyJK4ansu/MqnAiEA929qJUin6PO4JKW/IKIJup5BgfOU
+Nbn/ivmgbyhLpE=
-----END CERTIFICATE-----

SNI 192.168.1.42:

172.17.0.2 (fail - Unable to load certificate)
$ echo | openssl s_client -showcerts -servername 192.168.1.42 -connect 172.17.0.2:443 2>/dev/null | openssl x509 -text
unable to load certificate
139637896070464:error:0909006C:PEM routines:get_name:no start line:crypto/pem/pem_lib.c:745:Expecting: TRUSTED CERTIFICATE

No SNI:

172.17.0.2 (success)
$ echo | openssl s_client -showcerts -connect 172.17.0.2:443 2>/dev/null | openssl x509 -text                          
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            8b:4e:95:ff:2d:ce:e7:4b:ec:56:e7:47:ab:a3:f9:9a
        Signature Algorithm: ecdsa-with-SHA256
        Issuer: CN = Caddy Local Authority - ECC Intermediate
        Validity
            Not Before: Sep 13 06:07:31 2020 GMT
            Not After : Sep 13 18:08:31 2020 GMT
        Subject: 
        Subject Public Key Info:
            Public Key Algorithm: id-ecPublicKey
                Public-Key: (256 bit)
                pub:
                    04:96:4d:f1:e4:23:b1:81:d9:0f:ec:cf:d4:c1:de:
                    19:01:b8:13:bb:0b:35:8c:a4:7a:80:45:2d:3c:9f:
                    6a:99:ad:f1:aa:82:65:9b:3d:8b:40:77:7b:24:83:
                    37:3d:c4:72:40:84:80:a7:a2:1e:97:e6:a3:cc:6c:
                    f0:b2:ed:ee:b2
                ASN1 OID: prime256v1
                NIST CURVE: P-256
        X509v3 extensions:
            X509v3 Key Usage: critical
                Digital Signature, Key Encipherment
            X509v3 Extended Key Usage: 
                TLS Web Server Authentication, TLS Web Client Authentication
            X509v3 Subject Key Identifier: 
                95:4B:B7:FB:97:0C:CB:D3:01:E1:C8:A8:54:17:B7:14:6E:2E:67:CF
            X509v3 Authority Key Identifier: 
                keyid:8B:BE:D9:7C:3E:32:B2:A9:24:D8:38:42:5F:7B:89:DE:CA:87:58:88

            X509v3 Subject Alternative Name: 
                IP Address:172.17.0.2
    Signature Algorithm: ecdsa-with-SHA256
         30:45:02:20:55:53:59:f2:8d:50:41:2f:51:8e:6b:19:17:e9:
         fb:68:ea:8e:f9:07:8d:2d:e4:e1:b6:0b:77:7b:3a:de:79:9a:
         02:21:00:a4:b1:11:ec:55:c2:e3:e1:7f:77:01:47:dd:5e:b7:
         b3:f3:e8:d3:2e:e5:83:77:89:c1:bd:93:75:5c:01:0c:b4
-----BEGIN CERTIFICATE-----
MIIBtjCCAVygAwIBAgIRAItOlf8tzudL7FbnR6uj+ZowCgYIKoZIzj0EAwIwMzEx
MC8GA1UEAxMoQ2FkZHkgTG9jYWwgQXV0aG9yaXR5IC0gRUNDIEludGVybWVkaWF0
ZTAeFw0yMDA5MTMwNjA3MzFaFw0yMDA5MTMxODA4MzFaMAAwWTATBgcqhkjOPQIB
BggqhkjOPQMBBwNCAASWTfHkI7GB2Q/sz9TB3hkBuBO7CzWMpHqARS08n2qZrfGq
gmWbPYtAd3skgzc9xHJAhICnoh6X5qPMbPCy7e6yo4GDMIGAMA4GA1UdDwEB/wQE
AwIFoDAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwHQYDVR0OBBYEFJVL
t/uXDMvTAeHIqFQXtxRuLmfPMB8GA1UdIwQYMBaAFIu+2Xw+MrKpJNg4Ql97id7K
h1iIMA8GA1UdEQQIMAaHBKwRAAIwCgYIKoZIzj0EAwIDSAAwRQIgVVNZ8o1QQS9R
jmsZF+n7aOqO+QeNLeThtgt3ezreeZoCIQCksRHsVcLj4X93AUfdXrez8+jTLuWD
d4nBvZN1XAEMtA==
-----END CERTIFICATE-----

curl output:

172.17.0.2
$ curl -v https://172.17.0.2
*   Trying 172.17.0.2:443...
* Connected to 172.17.0.2 (172.17.0.2) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: [NONE]
*  start date: Sep 13 06:07:31 2020 GMT
*  expire date: Sep 13 18:08:31 2020 GMT
*  subjectAltName: host "172.17.0.2" matched cert's IP address!
*  issuer: CN=Caddy Local Authority - ECC Intermediate
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x56396de0e9f0)
> GET / HTTP/2
> Host: 172.17.0.2
> user-agent: curl/7.71.1
> accept: */*
> 
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* Connection state changed (MAX_CONCURRENT_STREAMS == 250)!
< HTTP/2 200 
< server: Caddy
< content-length: 3
< date: Sun, 13 Sep 2020 06:37:09 GMT
< 
* Connection #0 to host 172.17.0.2 left intact
Bye

Ok… so that confirms that SNI looks up a certificate, and that default_sni has no relevance at least with IPs if that has no match within a site block’s addresses. I added a third site block for 192.168.1.42 to see what happens with the certificate for it, but continued to see 172.17.0.2 SAN, same when adding 192.168.1.42 to the 192.168.1.8 site block, 192.168.1.8 would continue to return the 172.17.0.2 SAN/cert.

EDIT: I thought that default_sni only didn’t work if the default SNI that setting overrides wasn’t in a site address for the same block, turns out that using the 172.17.0.2 for a different site block is what prevented default_sni from taking affect.

{
  default_sni 192.168.1.8
}

192.168.1.8 {
  respond "Hello"
}

172.17.0.2 {
  respond "Bye"
}

The above fails like earlier examples as the default_sni doesn’t override 172.17.0.2 which is returned as SAN still.

{
  default_sni 192.168.1.8
}

192.168.1.8 {
  respond "Hello"
}

Success with overriding, can use default_sni 192.168.1.42 and it will appear as the SAN now (provided it’s added as a site address for a site block too, otherwise will fail to load a certificate).


TL;DR:

  • Caddy will respond to IP addresses that are used as site addresses for site blocks correctly over HTTP.
  • Caddy relies on SNI to load TLS certificate, where each certificate has single SAN.
  • IP addresses will register individual certs with their IP as a SAN, but as SNI is typically not provided with IP address requests, the TLS cert loaded is dependent on the default SNI fallback value.

If Caddy responds to the correct site block over HTTP without SNI, why does it rely on SNI to load a TLS certificate? Is it possible to use that same information to look up a certificate as fallback, rather than whatever default_sni is set to? (which can only use a single address?)