Using a public domain name to enable HTTPS on a public server and LAN-only server

I would like to use Cloudflare for ACME validation over LAN using caddy-cloudflare-ddns.

I have A Records pointing server1.mydomain.tld to my public IP and would add server2.mydomain.tld to a LAN IP.

I have port 443 open for server1 but I don’t want anyone outside my LAN accessing server2.

This would be my Caddyfile:

https://server1.mydomain.tld:443 {
  reverse_proxy localhost:1234
  }

server2.mydomain.tld {
  tls {
    dns cloudflare CLOUDFLARE_API_TOKEN
  }
  reverse_proxy localhost:5678
}

Would this let someone access server2 without physically being on my LAN? It feels like it might and I could have a guess at how but I’d probably be wrong. I am extremely new to this so I really appreciate any help and insight.

I’ve gone with:

https://server1.mydomain.tld:443 {
  reverse_proxy localhost:1234
  }

server2.mydomain.tld {
  @denied not client_ip private_ranges
  abort @denied
  tls {
    dns cloudflare CLOUDFLARE_API_TOKEN
  }
  reverse_proxy localhost:5678
}

As I understand it, IP will only respond to the IP address it thinks it’s connected to, so if I can block all public IPs, a spoofed private IP will inherently keep the connection inside my LAN.