Restricting access to only private IPs, IPv6 & CGNAT subnets

Help
1

1. The problem I’m having:

I am trying to deny access to some proxy sites using the @denied not client_ip ... string in my caddy file(s). When using just RFC1918 addresses via private_ranges everything works as expected. Adding more than just private_ranges does not appear to work correctly. I would like to restrict access to only RFC1918 addresses, the IPv6 block assigned by my ISP, and (now that I have added tailscale to the mix) the 100.64.0.0/10 CGNAT block.

# private_ranges               RFC1918
# 100.64.0.0/10                tailscale
# 2601:601:600:7a40:0:0:0:0/60 Camp: comcast delegated supernet.

@denied not client_ip private_ranges 100.64.0.0/10 2601:601:600:7a40:0:0:0:0/60

# OR

}
@denied {
  not client_ip private_ranges
  not client_ip 100.64.0.0/10
  # not client_ip 2601:601:600:7a40:0:0:0:0/60 #Same result if I remove IPv6
}

I even went as far as trying this. 100.95.184.67 is my current tailscale IP.
@denied not client_ip 100.95.184.67

2. Error messages and/or full log output:

$ curl -vL sonarr.cozzo.net

* Host sonarr.cozzo.net:80 was resolved.
* IPv6: (none)
* IPv4: 10.10.0.2
*   Trying 10.10.0.2:80...
* Connected to sonarr.cozzo.net (10.10.0.2) port 80
> GET / HTTP/1.1
> Host: sonarr.cozzo.net
> User-Agent: curl/8.6.0
> Accept: */*
>
< HTTP/1.1 308 Permanent Redirect
< Connection: close
< Location: https://sonarr.cozzo.net/
< Server: Caddy
< Date: Mon, 09 Dec 2024 23:26:25 GMT
< Content-Length: 0
<
* Closing connection
* Clear auth, redirects to port from 80 to 443
* Issue another request to this URL: 'https://sonarr.cozzo.net/'
* Host sonarr.cozzo.net:443 was resolved.
* IPv6: (none)
* IPv4: 10.10.0.2
*   Trying 10.10.0.2:443...
* Connected to sonarr.cozzo.net (10.10.0.2) port 443
* ALPN: curl offers h2,http/1.1
* (304) (OUT), TLS handshake, Client hello (1):
*  CAfile: /etc/ssl/cert.pem
*  CApath: none
* (304) (IN), TLS handshake, Server hello (2):
* (304) (IN), TLS handshake, Unknown (8):
* (304) (IN), TLS handshake, Certificate (11):
* (304) (IN), TLS handshake, CERT verify (15):
* (304) (IN), TLS handshake, Finished (20):
* (304) (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / AEAD-AES128-GCM-SHA256 / [blank] / UNDEF
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=sonarr.cozzo.net
*  start date: Dec  8 17:13:15 2024 GMT
*  expire date: Mar  8 17:13:14 2025 GMT
*  subjectAltName: host "sonarr.cozzo.net" matched cert's "sonarr.cozzo.net"
*  issuer: C=US; O=Let's Encrypt; CN=E6
*  SSL certificate verify ok.
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://sonarr.cozzo.net/
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: sonarr.cozzo.net]
* [HTTP/2] [1] [:path: /]
* [HTTP/2] [1] [user-agent: curl/8.6.0]
* [HTTP/2] [1] [accept: */*]
> GET / HTTP/2
> Host: sonarr.cozzo.net
> User-Agent: curl/8.6.0
> Accept: */*
>
< HTTP/2 200
< alt-svc: h3=":443"; ma=2592000
< content-type: text/plain; charset=utf-8
< referrer-policy: same-origin
< server: Caddy
< strict-transport-security: max-age=31536000; includeSubdomains
< x-content-type-options: nosniff
< x-frame-options: SAMEORIGIN
< x-xss-protection: 1; mode=block
< content-length: 59
< date: Mon, 09 Dec 2024 23:26:25 GMT
<
* Connection #1 to host sonarr.cozzo.net left intact
Your request from 100.95.184.67 was dropped.%

3. Caddy version:

$ sudo docker exec 62338e8af401 caddy version
v2.8.4 h1:q3pe0wpBj1OcHFZ3n/1nl4V4bxBrYoSoab7rL9BMYNk=

4. How I installed and ran Caddy:

Via a docker compose file:

/mnt/git/docker/apps/caddy/docker-compose.yml

---
services:
  caddy:
    container_name: caddy
    image: caddy:latest
    restart: unless-stopped
    build: .
    env_file:
      - /mnt/docker/secrets/caddy.env
    cap_add:
      - NET_ADMIN
    networks:
      - frontend
    dns:
      - 1.1.1.1
      - 1.0.0.1
    ports:
      - "80:80"
      - "80:80/udp"
      - "443:443"
      - "443:443/udp"
    volumes:
      - /mnt/docker/configs/caddy/caddy:/etc/caddy       # Custom caddy file - root
      - /mnt/docker/configs/caddy/site:/srv              # Default site
      - /mnt/docker/configs/caddy/data:/data
      - /mnt/docker/configs/caddy/config:/config
      - /mnt/docker/configs/caddy/caddyfiles:/caddyfiles # Site config files, rproxy redirects, etc.

networks:
  frontend:
    external: true

a. System environment:

  • Digital ocean droplet: Ubuntu 24.10
  • eth0 - Public
  • eth1 - Internal (VPC) 10.10.0.0/16
  • Tailscale: Proxy server is acting as an exit node and advertizing the 10.10.0.0/16 subnet.
    ** Other nodes are just member nodes.

b. Command:

$ sudo docker compose -f /mnt/git/docker/apps/caddy.do.cozzo.net/docker-compose.yml up -d

c. Service/unit/compose file:

N/A

d. My complete Caddy config:

/mnt/docker/configs/caddy/caddy/Caddyfile

:80, :443 {
  redir https://proxy.do.cozzo.net
}

proxy.do.cozzo.net {
  root * /srv/default
  file_server
}

# Import caddy files so each site can be unique
import /caddyfiles/*.caddy

Site specific BROKEN:
/mnt/docker/configs/caddy/caddyfiles/sonarr.do.cozzo.net.caddy

sonarr.cozzo.net, do-sonarr.cozzo.net {
  reverse_proxy p-docker01.do.cozzo.net:8989

  header {
    Strict-Transport-Security "max-age=31536000; includeSubdomains"
    X-XSS-Protection "1; mode=block"
    X-Content-Type-Options "nosniff"
    X-Frame-Options "SAMEORIGIN"
    Referrer-Policy "same-origin"
  }

  tls {
    dns cloudflare {env.CF_API_TOKEN}
    resolvers 1.1.1.1
  }

  # Only respond to RFC1918
  # private_ranges               RFC1918
  # 100.64.0.0/10                tailscale
  # 2601:601:600:7a40:0:0:0:0/60 Camp: comcast delegated supernet.

  # @denied not client_ip private_ranges 100.64.0.0/10 2601:601:600:7a40:0:0:0:0/60
  }
  @denied {
    not client_ip private_ranges
    not client_ip 100.64.0.0/10
    # not client_ip 2601:601:600:7a40:0:0:0:0/60 #Same result if I remove IPv6
  }

  abort @denied

  respond "Your request from {client_ip} was dropped."
}

Working config:
/mnt/docker/configs/caddy/caddyfiles/kuma.do.cozzo.net.caddy

kuma.cozzo.net, do-kuma.cozzo.net, uptime.cozzo.net, status.cozzo.net {
  reverse_proxy p-proxy.do.cozzo.net:3001

  header {
    Strict-Transport-Security "max-age=31536000; includeSubdomains"
    X-XSS-Protection "1; mode=block"
    X-Content-Type-Options "nosniff"
    X-Frame-Options "SAMEORIGIN"
    Referrer-Policy "same-origin"
  }

  #tls {
  # dns cloudflare {env.CF_API_TOKEN}
  # resolvers 1.1.1.1
  #}
}

5. Links to relevant resources:

Public

dig proxy.do.cozzo.net +noall +answer
proxy.do.cozzo.net. 300 IN  A 165.232.153.168

dig docker01.do.cozzo.net +noall +answer
docker01.do.cozzo.net.  300 IN  A 164.92.74.175

Internal

dig p-proxy.do.cozzo.net +noall +answer
p-proxy.do.cozzo.net. 300 IN  A 10.10.0.2

dig p-docker01.do.cozzo.net +noall +answer
p-docker01.do.cozzo.net. 300  IN  A 10.10.0.4

Apps proxied

https://sonarr.cozzo.net

dig sonarr.cozzo.net +noall +answer
sonarr.cozzo.net. 300 IN  CNAME p-proxy.do.cozzo.net.
p-proxy.do.cozzo.net. 300 IN  A 10.10.0.2

dig kuma.cozzo.net +noall +answer
kuma.cozzo.net.   300 IN  CNAME proxy.do.cozzo.net.
proxy.do.cozzo.net. 300 IN  A 165.232.153.168

Apps direct

https://sonarr.do.cozzo.net:8989

dig sonarr.do.cozzo.net +noall +answer
sonarr.do.cozzo.net.  104 IN  CNAME p-docker01.do.cozzo.net.
p-docker01.do.cozzo.net. 104  IN  A 10.10.0.4

dig kuma.do.cozzo.net +noall +answer
kuma.do.cozzo.net.  300 IN  CNAME p-proxy.do.cozzo.net.
p-proxy.do.cozzo.net. 300 IN  A 10.10.0.2

From within the caddy container:

/srv # ping p-docker01.do.cozzo.net
PING p-docker01.do.cozzo.net (10.10.0.4): 56 data bytes
64 bytes from 10.10.0.4: seq=0 ttl=63 time=6.348 ms
64 bytes from 10.10.0.4: seq=1 ttl=63 time=3.873 ms
^C

/srv # ping sonarr.do.cozzo.net
PING sonarr.do.cozzo.net (10.10.0.4): 56 data bytes
64 bytes from 10.10.0.4: seq=0 ttl=63 time=5.042 ms
64 bytes from 10.10.0.4: seq=1 ttl=63 time=1.897 ms
^C

/srv # wget sonarr.do.cozzo.net:8989/
Connecting to sonarr.do.cozzo.net:8989 (10.10.0.4:8989)
Connecting to sonarr.do.cozzo.net:8989 (10.10.0.4:8989)
saving to 'index.html'
index.html           100% |*****************************************************************************************************************************************************************************************************************************************|  9654  0:00:00 ETA
'index.html' saved

/srv # rm index.html

/srv # wget p-docker01.do.cozzo.net:8989/
Connecting to p-docker01.do.cozzo.net:8989 (10.10.0.4:8989)
Connecting to p-docker01.do.cozzo.net:8989 (10.10.0.4:8989)
saving to 'index.html'
index.html           100% |*****************************************************************************************************************************************************************************************************************************************|  9654  0:00:00 ETA
'index.html' saved
1 Like
2

Howdy @mcozzo, welcome to the Caddy community.

What exactly do you mean by “does not appear to work correctly”?

Is it blocking when it shouldn’t block? Is it allowing requests that shouldn’t be allowed? What, exactly, is going wrong?

3

If I have just @denied not client_ip private_ranges I’m able to view the site from inside the network and it is blocked from public IP addresses.

When I add @denied not client_ip private_ranges 100.64.0.0/10 I can’t access the site from the new subnet. You can see the error that is returned with curl.

4

The error returned with curl is expected behaviour per your configuration:

Requests outside the configured IPs get aborted (expected: curl: (52) Empty reply from server).

All other requests fall through to respond (expected: Your request from {client_ip} was dropped.)

Your reverse_proxy doesn’t matter because respond has a higher priority. See: https://caddyserver.com/docs/caddyfile/directives#directive-order

5

By my accounts it should be showing me the proxied page.

@denied not client_ip private_ranges 100.64.0.0/10 
respond "Your request from {client_ip} was dropped."

I added the respond line with {client_ip} so that I can see where the connections are coming from.

My source address, and what caddy shows as the {client_ip} address is 100.95.184.67 That is firmly inside the 100.64.0.0/10 subnet.

The allocated address block is 100.64.0.0/10, i.e. IP addresses from 100.64.0.0 to 100.127.255.255.
https://en.wikipedia.org/wiki/Carrier-grade_NAT

You can see that with this CURL command.

curl -vL sonarr.cozzo.net
...
* Connection #1 to host sonarr.cozzo.net left intact
Your request from 100.95.184.67 was dropped.

So, how do I list multiple subnets?
Ultimatly, I need the RFC1918 subnets, CGNAT, and my IPv6 block to all be listed?

@denied {
  not client_ip private_ranges               # RFC1918
  not client_ip 100.64.0.0/10                # CGNAT
  not client_ip 2601:601:600:7a40:0:0:0:0/60 # Assigned IPv6 subnet
}
6

I see, the respond take precedent over the proxy line. I added that as I was trying to debug the issue. I commented that out and it appears to be working now.

How should I respond with a more detailed error page on connection drop?

	not client_ip private_ranges 
	not client_ip 100.64.0.0/10 
	not client_ip 2601:601:600:7a40:0:0:0:0/60 
}
	
abort @denied "Your request from {client_ip} was dropped."
# ??

#respond "Your request from {client_ip} was dropped."
7

respond is what we refer to as a terminal handler; it writes a full, complete response and ends the processing chain.

If you wanted to respond to denials, you would need to actually add the named matcher to the directive, e.g. respond @denied "..." - refer: https://caddyserver.com/docs/caddyfile/directives/respond#syntax

But, then you would also need to get rid of abort @denied because abort takes precedence over respond, but it does not allow you to specify a response body like that. This is by design - you are configuring Caddy to abort processing without returning any response.

If you wanted to add additional info to another route, you might consider writing a header instead of a response body.

That said, as this is for debugging purposes, you’re better off just adding debug to the global options and inspecting the request / upstream roundtrip logs. Those will tell you almost everything Caddy knows and uses to make decisions about routing and handling when it comes to the client as well as the backend servers if that’s necessary.

Remove abort. Use respond instead.

If you want to reply with a HTML error page, you might consider using a handle @denied block, and within, rewriting to some error.html with a file_server. Alternatively you could put your HTML document in your Caddyfile with a heredoc if it’s simple enough, which lets you relatively easily insert placeholders for end user reference. Here’s an example from our docs:

example.com {
	respond <<HTML
		<html>
		  <head><title>Foo</title></head>
		  <body>Foo</body>
		</html>
		HTML 200
}

(Refer: https://caddyserver.com/docs/caddyfile/concepts#tokens-and-quotes)

1 Like
8

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.