How to abort or 403 sub-subdomains with a wildcard?

Help
1

1. The problem I’m having:

A customer of ours has wildly misconfigured their DNS, internal services, or… well something.

I want to block (abort or send a 403 back) all requests that match a specific host. I have so far been successful blocking one level deep subdomains (www), but not sub, subdomains (e.g. ‘www.auth.demo.shop.build.olympiadrtc.com’).

I have tried wildcard matchers, http matchers, and explicit sub subdomain matchers. The explicit ones seem to work, but these bots seem to be doing random variations of subdomains so I can’t keep up with hardcoding them in.

Here is the output of curl -vL www.auth.demo.shop.build.olympiadrtc.com, which I expected to 403 and not be reverse proxied.

curl -vL www.auth.demo.shop.build.olympiadrtc.com
*   Trying 34.210.1.245:80...
* Connected to www.auth.demo.shop.build.olympiadrtc.com (34.210.1.245) port 80 (#0)
> GET / HTTP/1.1
> Host: www.auth.demo.shop.build.olympiadrtc.com
> User-Agent: curl/8.1.2
> Accept: */*
> 
< HTTP/1.1 308 Permanent Redirect
< Connection: close
< Location: https://www.auth.demo.shop.build.olympiadrtc.com/
< Server: Caddy
< Date: Mon, 20 Nov 2023 03:26:37 GMT
< Content-Length: 0
< 
* Closing connection 0
* Clear auth, redirects to port from 80 to 443
* Issue another request to this URL: 'https://www.auth.demo.shop.build.olympiadrtc.com/'
*   Trying 34.210.1.245:443...
* Connected to www.auth.demo.shop.build.olympiadrtc.com (34.210.1.245) port 443 (#1)
* ALPN: offers h2,http/1.1
* (304) (OUT), TLS handshake, Client hello (1):
*  CAfile: /etc/ssl/cert.pem
*  CApath: none
* (304) (IN), TLS handshake, Server hello (2):
* (304) (IN), TLS handshake, Unknown (8):
* (304) (IN), TLS handshake, Certificate (11):
* (304) (IN), TLS handshake, CERT verify (15):
* (304) (IN), TLS handshake, Finished (20):
* (304) (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / AEAD-AES128-GCM-SHA256
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=www.auth.demo.shop.build.olympiadrtc.com
*  start date: Nov 18 00:00:00 2023 GMT
*  expire date: Feb 16 23:59:59 2024 GMT
*  subjectAltName: host "www.auth.demo.shop.build.olympiadrtc.com" matched cert's "www.auth.demo.shop.build.olympiadrtc.com"
*  issuer: C=AT; O=ZeroSSL; CN=ZeroSSL ECC Domain Secure Site CA
*  SSL certificate verify ok.
* using HTTP/2
* h2 [:method: GET]
* h2 [:scheme: https]
* h2 [:authority: www.auth.demo.shop.build.olympiadrtc.com]
* h2 [:path: /]
* h2 [user-agent: curl/8.1.2]
* h2 [accept: */*]
* Using Stream ID: 1 (easy handle 0x7fb752013000)
> GET / HTTP/2
> Host: www.auth.demo.shop.build.olympiadrtc.com
> User-Agent: curl/8.1.2
> Accept: */*
> 
< HTTP/2 404 
< alt-svc: h3=":443"; ma=2592000
< content-type: text/html; charset=UTF-8
< date: Mon, 20 Nov 2023 03:26:37 GMT
< nel: {"report_to":"heroku-nel","max_age":3600,"success_fraction":0.005,"failure_fraction":0.05,"response_headers":["Via"]}
< report-to: {"group":"heroku-nel","max_age":3600,"endpoints":[{"url":"https://nel.heroku.com/reports?ts=1700450797&sid=af571f24-03ee-46d1-9f90-ab9030c2c74c&s=0ID8lCQ0xEHMOS9yP0jvdfyHibV4TNflOPJwuf%2F0tBk%3D"}]}
< reporting-endpoints: heroku-nel=https://nel.heroku.com/reports?ts=1700450797&sid=af571f24-03ee-46d1-9f90-ab9030c2c74c&s=0ID8lCQ0xEHMOS9yP0jvdfyHibV4TNflOPJwuf%2F0tBk%3D
< server: Caddy
< server: Cowboy
< strict-transport-security: max-age=63072000; includeSubDomains
< via: 1.1 vegur
< x-request-id: 2de2e8ec-a91d-401e-84e4-8593b339c336
< x-runtime: 0.016930
< content-length: 1564
< 
<!DOCTYPE html>
<html>
<head>
  <title>The page you were looking for doesn't exist (404)</title>
  <meta name="viewport" content="width=device-width,initial-scale=1">
  <style>
  body {
    background-color: #EFEFEF;
    color: #2E2F30;
    text-align: center;
    font-family: arial, sans-serif;
    margin: 0;
  }

  div.dialog {
    width: 95%;
    max-width: 33em;
    margin: 4em auto 0;
  }

  div.dialog > div {
    border: 1px solid #CCC;
    border-right-color: #999;
    border-left-color: #999;
    border-bottom-color: #BBB;
    border-top: #48647F solid 4px;
    border-top-left-radius: 9px;
    border-top-right-radius: 9px;
    background-color: white;
    padding: 7px 12% 0;
    box-shadow: 0 3px 8px rgba(50, 50, 50, 0.17);
  }

  h1 {
    font-size: 100%;
    color: #102A42;
    line-height: 1.5em;
  }

  div.dialog > p {
    margin: 0 0 1em;
    padding: 1em;
    background-color: #F7F7F7;
    border: 1px solid #CCC;
    border-right-color: #999;
    border-left-color: #999;
    border-bottom-color: #999;
    border-bottom-left-radius: 4px;
    border-bottom-right-radius: 4px;
    border-top-color: #DADADA;
    color: #666;
    box-shadow: 0 3px 8px rgba(50, 50, 50, 0.17);
  }
  </style>
</head>

<body>
  <!-- This file lives in public/404.html -->
  <div class="dialog">
    <div>
      <h1>The page you were looking for doesn't exist.</h1>
      <p>You may have mistyped the address or the page may have moved.</p>
    </div>
    <p>If you are the application owner check the logs for more information.</p>
  </div>
</body>
</html>
* Connection #1 to host www.auth.demo.shop.build.olympiadrtc.com left intact

2. Error messages and/or full log output:

No error messages. Here is a line from the logs I would have expected to 403, but instead reverse proxied.

{"level":"error","ts":1700450969.11674,"logger":"http.log.access.log0","msg":"handled request","request":{"remote_ip":"87.236.176.41","remote_port":"54137","client_ip":"87.236.176.41","proto":"HTTP/1.1","method":"GET","host":"www.autodiscover.ww1.new.build.olympiadrtc.com","uri":"/","headers":{"Connection":["close"],"Accept":["*/*"],"Accept-Encoding":["gzip"],"User-Agent":["Mozilla/5.0 (compatible; InternetMeasurement/1.0; +https://internet-measurement.com/)"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"","server_name":"www.autodiscover.ww1.new.build.olympiadrtc.com"}},"bytes_read":0,"user_id":"","duration":0.077113841,"size":1564,"status":404,"resp_headers":{"Content-Length":["1564"],"Server":["Caddy","Cowboy"],"Report-To":["{\"group\":\"heroku-nel\",\"max_age\":3600,\"endpoints\":[{\"url\":\"https://nel.heroku.com/reports?ts=1700450969&sid=af571f24-03ee-46d1-9f90-ab9030c2c74c&s=98eWutDZGRpA3Xk3wwGtASVffVFgqylqFUybBiYeHzw%3D\"}]}"],"Reporting-Endpoints":["heroku-nel=https://nel.heroku.com/reports?ts=1700450969&sid=af571f24-03ee-46d1-9f90-ab9030c2c74c&s=98eWutDZGRpA3Xk3wwGtASVffVFgqylqFUybBiYeHzw%3D"],"Nel":["{\"report_to\":\"heroku-nel\",\"max_age\":3600,\"success_fraction\":0.005,\"failure_fraction\":0.05,\"response_headers\":[\"Via\"]}"],"X-Request-Id":["4de173e1-7f40-4d85-aeff-2533ee3b56d7"],"X-Runtime":["0.014560"],"Strict-Transport-Security":["max-age=63072000; includeSubDomains"],"Alt-Svc":["h3=\":443\"; ma=2592000"],"Date":["Mon, 20 Nov 2023 03:29:28 GMT"],"Via":["1.1 vegur"],"Content-Type":["text/html; charset=UTF-8"]}}

3. Caddy version:

v2.7.5

4. How I installed and ran Caddy:

Followed official debian instructions this evening to re-install caddy.

a. System environment:

Ubuntu 20.04.2 LTS

b. Command:

systemctl caddy start

c. Service/unit/compose file:

N/A

d. My complete Caddy config:

{
	on_demand_tls {
		ask http://localhost:8001/
	}
}

*.olympiadrtc.com {
	respond "Access Denied" 403
}

http://*.olympiadrtc.com {
	respond "Access Denied" 403
}

*.olympiadwrestling.com {
	respond "Access Denied" 403
}

http://*.olympiadwrestling.com {
	respond "Access Denied" 403
}

www.auth.secure.proton.olympiadwrestling.com {
	respond "Access Denied" 403
}

gitlab.git.git.gitlab.git.2022.olympiadwrestling.com {
	respond "Access Denied" 403
}

https:// {
	log {
		output file /var/log/caddy/access.log
	}

	# Very basic ip address ban functionality
	@blockedip remote_ip forwarded 159.223.78.147 159.223.81.241 159.223.89.50
	abort @blockedip

	# TODO block specific paths .env /wp-admin, etc
	@forbidden {
		path /wp-login.php
		path /wp-login*
		path /wp-config*
		path *wp-includes*
		path *.env*
		host *olympiadrtc*
		host *olympiadwrestling*
	}
	respond @forbidden "<h1>Access Denied</h1>" 403

	# A custom matcher to pick out any custom domains to redirect naked to www
	# @customhost {
	#    host not_starts_with www.
	#    host not *wrestlingiq*
	#}
	@customhost not header_regexp Host (www\..*|(.*wrestlingiq.*|.georgeschool.*))
	redir @customhost https://www.{host}{uri}

	reverse_proxy https://production-wrestlingiq.herokuapp.com {
		header_up Host {http.reverse_proxy.upstream.hostport}
		header_up X-Real-IP {http.reverse-proxy.upstream.address}
		header_up X-Forwarded-Port {http.request.port}
		header_up X-Forwarded-Host {http.request.host}
	}

	tls {
		on_demand
	}
}

:8001 {
	bind 127.0.0.1 ::1
        # TODO - invert this tomorrow to be explicit list of custom domains we serve
	map {query.domain} {allowed} {
		olympiadrtc.com 0
		olympiadwrestling.com 0
		default 1
	}

	@allowed `{allowed} == "1"`
	respond @allowed 200
	respond 400
}

5. Links to relevant resources:

2

You’re supposed to reject those domains on your ask endpoint. Are you not doing that? You should only allow known domains, you should never blanket allow domains.

The whole point of ask is to avoid a DDoS vulnerability. If you allow all domains (or most including unknown domains), then you’re issuing certificates for each of those. That means doing cert issuance with an ACME issuer. That’s relatively expensive in terms of time and storage, and you may hit rate limits, or run out of storage entirely if an attacker decides to make requests enumerating domains ad infinitum.

After you fix that, make sure to delete the certificates for domains you don’t want from Caddy’s storage then restart Caddy; otherwise it’ll continue to serve requests from those domains because the TLS handshake would succeed.

That’s the way to do it. Just write a regexp matching Host to reject those requests.

But like I said that’s the wrong place to solve this, because the TLS handshake should have failed before even reaching HTTP routes.

3

You’re supposed to reject those domains on your ask endpoint. Are you not doing that? You should only allow known domains, you should never blanket allow domains.

Agreed! I had set this infrastructure up before the ask endpoint was required and am now seeing that exact situation. The file based solution was taken from this post:

In a panic last night to get something up to shield my caddy servers. I’ll be implementing the ask endpoint in my real backend today to protect moving forward. I appreciate the tip on deleting the domains as well.

Part of the problem is I do need to serve the certificate for the www version of this domain (for their website), but not any of the other subdomain stuff that their webmaster misconfigured—I understand what you are saying though, that should just be part of the ask endpoint.

That’s the way to do it. Just write a regexp matching Host to reject those requests.

:man_facepalming: Thank you

4

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.