Caddy + Cloudflare tunnel

[avirut]

I just got a particular Caddy setup working after hours of trying out various things, and wanted to share. I was looking to have a fully Dockerized setup that puts Caddy behind a Cloudflare tunnel, while also allowing the cloudflared browser-based SSH to work.

First up is cloudflared - I’m using msnelling/cloudflared as the official image lacks ARM support. Most of my setup is based off of this tutorial, so you’ll have to follow that for the initial steps so far as creating a tunnel goes. Here are some of the files I’m using as well as some notes on what makes them work:

cloudflared

cloudflare/docker-compose.yml:

version: '3.4'
services:
  cloudflared:
    image: msnelling/cloudflared
    container_name: cloudflared
    volumes:
      - ./config:/etc/cloudflared
    # interpolate environment vars into config template to make config and then run
    command: [sh, -c, ( echo "cat <<EOF" ; cat /etc/cloudflared/config.yml.template ; echo EOF ) | sh > /etc/cloudflared/config.yml && /usr/local/bin/cloudflared tunnel --no-autoupdate run]
    env_file:
      - .env
    restart: always
    extra_hosts:
      - "host.docker.internal:host-gateway"
    networks:
     - cloudflared

networks:
  cloudflared:
    external: true

• Although cloudflared requires a config.yml of its own, it does not use environment variables for specified values at runtime. Since I want to be able to easily share/change my setup, I added a step to the command that takes a config.yml.template, and interpolates in environment variables to create a final config.yml before cloudflared is run within the container.
• The extra_hosts is required for SSH to work through cloudflared, as a container binding to port 22 cannot otherwise be started while SSH’d into the server. The only alternatives I am aware of are:
  • network_mode: host, which keeps you from putting the cloudflared container on any other networks and therefore makes it more difficult to send traffic to your caddy container
  • running cloudflared as a system service without Docker, which both makes sending traffic to caddy harder and makes the system harder to transport

cloudflared/config/config.yml.template

tunnel: ${TUNNEL_UUID}

ingress:
  # hostname for SSH, best as tname.domain.tld/ssh
  - hostname: "${HOST_HOSTNAME}.${DOMAIN}"
    service: ssh://host.docker.internal:22
  # hostname for root domain in form domain.tld
  - hostname: "${DOMAIN}"
    service: https://caddy:443
    originRequest:
      originServerName: "${DOMAIN}"
  # hostname for everything else in form *.domain.tld
  - hostname: "*.${DOMAIN}"
    service: https://caddy:443
    originRequest:
      originServerName: "${DOMAIN}"
  - service: http_status:404

• The first ingress allows you to use SSH from a browser at host_hostname.domain.tld as long as you’ve followed the other instructions in the aforementioned tutorial. This is where having the extra_hosts from our Compose file was necessary.
• The second ingress proxies the root domain to our caddy container.
• The third ingress proxies all subdomains to our caddy container. Note that the originServerName is still just domain.tld rather than including the subdomain (this one took me like an hour to figure out, since I know next to nothing about networking).

cloudflared/.env.template

TUNNEL_UUID=
DOMAIN=
HOST_HOSTNAME=

• These are the environment variables that will be interpolated into config.yml.template to make a config.yml.

Caddy

caddy/Dockerfile

ARG CADDY_VERSION=2.5.0

FROM caddy:${CADDY_VERSION}-builder AS builder

RUN xcaddy build \
	--with github.com/lucaslorentz/caddy-docker-proxy/plugin \
	--with github.com/caddy-dns/cloudflare \
	--with github.com/greenpau/caddy-security

FROM caddy:${CADDY_VERSION}-alpine

COPY --from=builder /usr/bin/caddy /usr/bin/caddy

CMD ["caddy", "docker-proxy"]

• Haven’t yet tagged the plugin versions but will do that soon.

caddy/docker-compose.yml

version: "3.9"

services:
  caddy:
    build:
      context: .
      dockerfile: Dockerfile
    image: caddy:v2.5.0
    container_name: caddy
    labels:
      caddy.acme_dns: "cloudflare {env.CF_API_TOKEN}"
      caddy.email: "{env.EMAIL}"
    environment:
      - CADDY_INGRESS_NETWORKS=caddy
    env_file:
      - .env
    networks:
      - caddy
      - cloudflared
    volumes:
      # for caddy-docker-proxy to work
      - /var/run/docker.sock:/var/run/docker.sock
      # for caddy itself
      - ~/data/caddy/data:/data
      - ~/data/caddy/config:/config
    restart: unless-stopped

networks:
  caddy:
    external: true
  cloudflared:
    external: true

• Goes on the cloudflared network to recieve traffic through the tunnel, while all of your application containers can go on just the caddy network.
• This also assumes that your DNS is through Cloudflare… which, if you’re using Cloudflare tunnels, it must be.

caddy/.env.template

CF_API_TOKEN=
EMAIL=

And that’s it! With caddy-docker-proxy as part of the image, every container on the caddy Docker network will get picked up and should “just work”.

[zzz (B Murphy)]

Thanks for the post… excellent work. It deserves to be highlighted in it’s own post which the moderator has done.