Caddy in Docker - Understanding headers with Cloudflare

1. Caddy version (caddy version):

Latest release on Docker hub.

2. How I run Caddy:

a. System environment:

I run this in Docker on a Linux (Debian 10) host. Compose file below.

b. Command:

docker-compose up --build

c. Service/unit/compose file:

version: "3.7"

services:
  caddy:
    image: caddy:latest
    ports:
      - 443:443/tcp
      - 80:80/tcp
    volumes:
      - ./Caddyfile:/etc/caddy/Caddyfile
      - /home/penumbra/images/:/etc/images/:ro
    network_mode: host
    container_name: "caddy"

  fastapi:
    image: python:3.9.0-buster
    build: .
    ports:
      - 9000:9000/tcp
    volumes:
      - /home/penumbra/images/:/etc/images/:rw
    network_mode: host
    container_name: "fastapi"

d. My complete Caddyfile or JSON config:

umbra-is.gay:80, might-be.gay:80 {
    header remote_addr "{header.X-Forwarded-For}"
    root * /etc/images/
    try_files {path} /404.html
    file_server
}

umbra-is.gay:80/image, might-be.gay:80/image {
    header remote_addr "{header.X-Forwarded-For}"
    reverse_proxy localhost:9000
}

maybe.umbra-is.gay:80, maybe.might-be.gay:80 {
    redir https://github.com/AbstractUmbra
}

3. The problem I’m having:

The issue I’m having is that the traffic routed through Caddy to my Python application (running uvicorn) is being passed the Docker internal IP addresses, not the external ones. By setting it to network-mode: host I am able to pass the address that comes through CloudFlare, but instead I want to set the remote_addr header (I believe?) to the value of the X-Forwarded-For header and what I have currently does not achieve this.

4. Error messages and/or full log output:

No errors as such, all output is successful except the logging from the uvicorn app is showing the CloudFlare address(es) as it should.

5. What I already tried:

I have tried using the header directive in Caddy to rewrite the remote_addr header, albeit incorrectly so far.

6. Links to relevant resources:

N/A

remote_addr isn’t really a header, but actually the IP in the TCP packets. That’s not something you can rewrite.

There is this PROXY protocol plugin that can set RemoteAddr on the request early in the request lifecycle, but it requires the downstream proxy to also support that protocol (I don’t know if Cloudflare does?)

I haven’t had trouble with running Caddy in docker in bridge mode, but I don’t run with Cloudflare. I do know that running in swam mode will bring about these kinds of issues because swarm mode uses its own ingress proxy which causes the IP address to change – but it doesn’t look like you’re using swarm mode here. Something else must be going on.

Thank you very much for the explanation. This might be more of a uvicorn issue in reading the header i desire, or some other method of reading the correct header.