Best solution for mixing LAN and Cloudflare Authenticated Origin Pulls?

[xnaas]

1. Caddy version (caddy version):

N/A

2. How I run Caddy:

Docker/Docker Compose.

a. System environment:

Docker/Docker Compose on Debian.

b. Command:

docker-compose up -d --no-deps --build --remove-orphans

c. Service/unit/compose file:

xnaas/docker-compose.yml/blob/master/docker-compose.yml

d. My complete Caddyfile or JSON config:

xnaas/Caddyfile/blob/master/Caddyfile

3. The problem I’m having:

tls {
    dns cloudflare {env.CLOUDFLARE_API_TOKEN}
    client_auth {
      mode require_and_verify
      trusted_ca_cert_file /data/origin-pull-ca.pem
    }
  }

I have this section in my main snippet. The client_auth is how I leverage Authenticated Origin Pulls from Cloudflare. This is great for forcing connections to come through Cloudflare and prevent direct-IP access to my sites, among other things.

Unfortunately, this also means that I can’t access any of my sites via localhost or LAN without first going out through the entire Internet and through Cloudflare. I’m wondering what the best solution or approach is to have client_auth for external requests but not for localhost and LAN.

4. Error messages and/or full log output:

Don’t have anything to provide at the moment, but if you try to bypass client_auth, you get invalid or internal SSL errors.

5. What I already tried:

N/A — I’m mostly looking for ideas/strategies and haven’t really committed to any approach at the moment.

6. Links to relevant resources:

N/A

[francislavoie]

Just make a second site that has doesn’t have that tls config in it, and use an address like <lan-ip>:<some-port> so it only accepts connections with that hostname (IP).

[xnaas]

Sorry, I probably should’ve clarified further.

The end goal would be: if I type xnaas.info on my phone or laptop at home, I would load the resource on LAN, but typing the same domain off LAN would load over the internet, through Cloudflare.

Edit: I control a DNS server on LAN, so DNS rewrites are a possible thing.

Edit 2: Additionally, Caddy would need to be able to “reach out to itself” so to speak. If irc.xnaas.info is trying to load a resource from itself or xnaas.info/tmp/, it should be reaching out to localhost (like what would be accomplished by adding 127.0.0.1 irc.xnaas.info to /etc/hosts), not over the Internet.

[francislavoie]

I don’t think that’s possible right now, because the TLS handshake happens too early, i.e. before request matchers are invoked.

It might be possible to implement a feature where some limited set of request matchers could be configured to decide whether client_auth should apply… I’m imagining something like this:

tls {
	client_auth {
		mode ...
		trusted_ca_cert_file ...
		match not remote_ip 127.0.0.1
	}
}

WDYT @matt, is this sort of thing viable? Do we have a http.Request at that point, or is it still too early in the pipeline?

[xnaas]

For fun, I was thinking of some janky ways one could handle this. Multiple Caddy instances would be one way. One Caddy instance could have all sites on :4443 or something (but :443 on the actual router or something to map the port) and then run the LAN/localhost Caddy instance on :443.

[matt (Matt Holt)]

Wait, I’m not entirely clear on what is needed. If you want to access properties of an HTTP request before a TLS connection, that’s not possible because TLS handshakes have to happen first. (That’s not a Caddy or Go restriction, that’s just how the Internet works.)

Or do you need to match on remote IP of the connection instead?

[james]

I believe what @xnaas wants here is to require the client certificate when the request is coming from any IP that isn’t their LAN/local connection.
That way they can have Cloudflare origin pulls authenticated by their certificate but LAN traffic be permitted since it wont have it.

[matt (Matt Holt)]

Ok, so we just need a way to match connection policies by remote IP. Currently they can only be matched by ServerName (SNI).

Will need JSON config to use it, though.

Edit: See this PR:

caddytls: Implement remote IP connection matcher

[xnaas]

Thank you for helping explain, @james!

---

@matt

Will need JSON config to use it, though.

I guess this further adds to “things I need to move from Caddyfile to JSON for” in my life.

---

Edit: Errors with Caddy 2.2.1 · Issue #2 · abiosoft/caddy-json-schema · GitHub — it’s a shame this tool doesn’t work anymore.

[xnaas]

For some additional perspective, this serverfault answer has a working solution for the same concept in nginx using its “geo” module. It looks like that is what, more or less, caddytls: Implement remote IP connection matcher by mholt · Pull Request #4123 · caddyserver/caddy · GitHub would add, so that’s cool.