"host" matcher more powerful than "listen" directive?

marcsal · July 10, 2023, 5:38am

1. The problem I’m having:

I am trying to run 2 servers:
Server 1 (acme_server): ACME Server, listens on localhost:14431
Server 2 (external_interfaces): generic Server (in this example answers “Hello World”), listens on 0.0.0.0 :443 and :80
Requests are sent via curl -vkL

On Server 1 I use a host matcher that results in unwanted behaviour:
HTTP requests from localhost directed at Server 2 get rerouted to Server 1, although the port does not match with the port from Server 1. Sending the same request via HTTPS results in the correct answer from Server 2. Also HTTP requests that are not coming from localhost are accepted.

I would expect that requests to Server 1 should match the Port 14431 to be accepted. But apparently the host matcher is more powerful than the “listen” directive. Is this behavior wanted?

When I remove the host matcher on Server 1 the server only accepts HTTP requests. Server 2 on the other hand now accepts both HTTP and HTTPS.

This is probably because a host matcher is needed to enable HTTPS
Automatic HTTPS — Caddy Documentation (caddyserver.com)

My goal is that HTTP and HTTPS requests from localhost to Server 2 result in the correct reply and also that Server 1 accepts HTTPS requests.

2. Error messages and/or full log output:

With host matcher:

HTTP Request to Server 1 :

-> % curl -v http://localhost:14431/acme/root_ca/directory -k -L
*   Trying 127.0.0.1:14431...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 14431 (#0)
> GET /acme/root_ca/directory HTTP/1.1
> Host: localhost:14431
> User-Agent: curl/7.68.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
* HTTP 1.0, assume close after body
< HTTP/1.0 400 Bad Request
<
Client sent an HTTP request to an HTTPS server.
* Closing connection 0

HTTPS Request to Server 1 :

-> % curl -v https://localhost:14431/acme/root_ca/directory -k -L
*   Trying 127.0.0.1:14431...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 14431 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: [NONE]
*  start date: Jul  5 13:22:37 2023 GMT
*  expire date: Jul  6 01:22:37 2023 GMT
*  issuer: CN=Caddy Local Authority - ECC Intermediate
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x556589aa5300)
> GET /acme/root_ca/directory HTTP/2
> Host: localhost:14431
> user-agent: curl/7.68.0
> accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* Connection state changed (MAX_CONCURRENT_STREAMS == 250)!
< HTTP/2 200
< alt-svc: h3=":14431"; ma=2592000
< content-type: application/json
< server: Caddy
< content-length: 312
< date: Wed, 05 Jul 2023 14:12:08 GMT
<
{"newNonce":"https://localhost:14431/acme/root_ca/new-nonce","newAccount":"https://localhost:14431/acme/root_ca/new-account","newOrder":"https://localhost:14431/acme/root_ca/new-order","revokeCert":"https://localhost:14431/acme/root_ca/revoke-cert","keyChange":"https://localhost:14431/acme/root_ca/key-change"}
* Connection #0 to host localhost left intact

HTTP Request to Server 2 :

-> % curl -v http://localhost/test -k -L
*   Trying 127.0.0.1:80...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 80 (#0)
> GET /test HTTP/1.1
> Host: localhost
> User-Agent: curl/7.68.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 308 Permanent Redirect
< Alt-Svc: h3=":443"; ma=2592000
< Connection: close
< Location: https://localhost:14431/test
< Server: Caddy
< Date: Wed, 05 Jul 2023 14:12:13 GMT
< Content-Length: 0
<
* Closing connection 0
* Clear auth, redirects to port from 80 to 14431Issue another request to this URL: 'https://localhost:14431/test'
*   Trying 127.0.0.1:14431...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 14431 (#1)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: [NONE]
*  start date: Jul  5 13:22:37 2023 GMT
*  expire date: Jul  6 01:22:37 2023 GMT
*  issuer: CN=Caddy Local Authority - ECC Intermediate
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x5601e49bd300)
> GET /test HTTP/2
> Host: localhost:14431
> user-agent: curl/7.68.0
> accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* Connection state changed (MAX_CONCURRENT_STREAMS == 250)!
< HTTP/2 200
< alt-svc: h3=":14431"; ma=2592000
< server: Caddy
< content-length: 0
< date: Wed, 05 Jul 2023 14:12:13 GMT
<
* Connection #1 to host localhost left intact

HTTPS Request to Server 2 :

-> % curl -v https://localhost/test -k -L
*   Trying 127.0.0.1:443...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: [NONE]
*  start date: Jul  5 13:22:37 2023 GMT
*  expire date: Jul  6 01:22:37 2023 GMT
*  issuer: CN=Caddy Local Authority - ECC Intermediate
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x564e80d37300)
> GET /test HTTP/2
> Host: localhost
> user-agent: curl/7.68.0
> accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* Connection state changed (MAX_CONCURRENT_STREAMS == 250)!
< HTTP/2 200
< alt-svc: h3=":443"; ma=2592000
< content-type: text/plain; charset=utf-8
< server: Caddy
< content-length: 12
< date: Wed, 05 Jul 2023 14:12:11 GMT
<
* Connection #0 to host localhost left intact
Hello, world%

Without host matcher:

HTTP Request to Server 1 :

-> % curl -v http://localhost:14431/acme/root_ca/directory -k -L
*   Trying 127.0.0.1:14431...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 14431 (#0)
> GET /acme/root_ca/directory HTTP/1.1
> Host: localhost:14431
> User-Agent: curl/7.68.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Content-Type: application/json
< Server: Caddy
< Date: Wed, 05 Jul 2023 14:11:06 GMT
< Content-Length: 312
<
{"newNonce":"https://localhost:14431/acme/root_ca/new-nonce","newAccount":"https://localhost:14431/acme/root_ca/new-account","newOrder":"https://localhost:14431/acme/root_ca/new-order","revokeCert":"https://localhost:14431/acme/root_ca/revoke-cert","keyChange":"https://localhost:14431/acme/root_ca/key-change"}
* Connection #0 to host localhost left intact

HTTPS Request to Server 1 :

-> % curl -v https://localhost:14431/acme/root_ca/directory -k -L
*   Trying 127.0.0.1:14431...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 14431 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* error:1408F10B:SSL routines:ssl3_get_record:wrong version number
* Closing connection 0
curl: (35) error:1408F10B:SSL routines:ssl3_get_record:wrong version number

HTTP Request to Server 2 :

-> % curl -v http://localhost/test -k -L
*   Trying 127.0.0.1:80...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 80 (#0)
> GET /test HTTP/1.1
> Host: localhost
> User-Agent: curl/7.68.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Alt-Svc: h3=":443"; ma=2592000
< Content-Type: text/plain; charset=utf-8
< Server: Caddy
< Date: Wed, 05 Jul 2023 14:11:22 GMT
< Content-Length: 12
<
* Connection #0 to host localhost left intact
Hello, world%

HTTPS Request to Server 2 :

-> % curl -v https://localhost/test -k -L
*   Trying 127.0.0.1:443...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: [NONE]
*  start date: Jul  5 11:52:01 2023 GMT
*  expire date: Jul  5 23:52:01 2023 GMT
*  issuer: CN=Caddy Root CA - ECC Intermediate
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x5564bcb10300)
> GET /test HTTP/2
> Host: localhost
> user-agent: curl/7.68.0
> accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* Connection state changed (MAX_CONCURRENT_STREAMS == 250)!
< HTTP/2 200
< alt-svc: h3=":443"; ma=2592000
< content-type: text/plain; charset=utf-8
< server: Caddy
< content-length: 12
< date: Wed, 05 Jul 2023 14:11:18 GMT
<
* Connection #0 to host localhost left intact
Hello, world%

3. Caddy version:

v2.6.4 h1:2hwYqiRwk1tf3VruhMpLcYTg+11fCdr8S3jhNAdnPy8=

4. How I installed and ran Caddy:

a. System environment:

WSL

b. Command:

sudo caddy run --config local_test_caddy_server_localhost_matcher.json

c. Service/unit/compose file:

d. My complete Caddy config:

{
  "apps": {
    "http": {
      "servers": {
        "acme_server": {
          "listen": [
            "127.0.0.1:14431"
          ],
          "routes": [
            {
              "handle": [
                {
                  "handler": "subroute",
                  "routes": [
                    {
                      "handle": [
                        {
                          "ca": "root_ca",
                          "handler": "acme_server"
                        }
                      ]
                    }
                  ]
                }
              ],
              "match": [
                {
                  "host": [
                    "localhost"
                  ]
                }
              ]
            }
          ]
        },
        "external-interfaces": {
          "automatic_https": {
            "disable_redirects": true
          },
          "listen": [
            "0.0.0.0:80",
            "0.0.0.0:443"
          ],
          "routes": [
            {
              "handle": [
                {
                  "handler": "subroute",
                  "routes": [
                    {
                      "handle": [
                        {
                          "body": "Hello, world",
                          "handler": "static_response",
                          "status_code": 200
                        }
                      ],
                      "match": [
                        {
                          "path": [
                            "/test"
                          ]
                        }
                      ]
                    }
                  ]
                }
              ]
            }
          ],
          "strict_sni_host": false,
          "tls_connection_policies": [
            {
              "cipher_suites": [
                "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256",
                "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
                "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384",
                "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",
                "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256",
                "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256"
              ],
              "protocol_max": "tls1.3",
              "protocol_min": "tls1.2"
            }
          ]
        }
      }
    },
    "pki": {
      "certificate_authorities": {
        "root_ca": {
          "name": "Caddy Root CA"
        }
      }
    },
    "tls": {
      "automation": {
        "on_demand": {
          "rate_limit": {
            "burst": 5,
            "interval": "1m"
          }
        },
        "policies": [
          {
            "issuers": [
              {
                "ca": "root_ca",
                "module": "internal"
              }
            ],
            "key_type": "rsa2048",
            "on_demand": true
          }
        ]
      }
    }
  }
}

5. Links to relevant resources:

matt · July 10, 2023, 6:49am

I’m mobile right now and about to go to sleep, so I can write a better answer later, but basically, the host matcher is what triggers auto HTTPS, which redirects HTTP to HTTPS. Will answer more later.

francislavoie · July 10, 2023, 6:53am

A server cannot serve multiple protocols simultaneously. Only HTTP or HTTPS at a time.

Remove "0.0.0.0:80" from server2 and it should work as expected, with HTTP->HTTPS redirects getting wired up automatically by Caddy’s “Automatic HTTPS” procedure. Or make a third site with port 80 if you don’t want redirects to happen.

marcsal · July 10, 2023, 3:22pm

Thank you! Removing "0.0.0.0:80" and "automatic_https": {"disable_redirects": true} from server2 has solved the issue. server2 now accepts both HTTP and HTTPS requests on the correct routes.

matt · July 10, 2023, 3:53pm

Ok well Francis did a better job than I would have anwyay Thanks @francislavoie !

system · August 9, 2023, 3:54pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.