Difference between CLI and Caddyfile for reverse-proxy

Help
1

Hello!

1. The problem I’m having:

I’m using Caddy as a reverse proxy in front of an application serving HTTP and GRPC APIs.

I tried using the CLI, but GRPC didn’t work. I then tried to use a Caddyfile and it worked. What’s the difference between the two configuration methods?

2. Error messages and/or full log output:

With the CLI:

caddy-1    | {"level":"debug","ts":1730823380.2811732,"logger":"events","msg":"event","name":"tls_get_certificate","id":"170e4e6d-2193-4873-94e9-22037da0f9e4","origin":"tls","data":{"client_hello":{"CipherSuites":[49195,49199,49196,49200,52393,52392,4865,4866,4867],"ServerName":"localhost","SupportedCurves":[29,23,24,25],"SupportedPoints":"AA==","SignatureSchemes":[2052,1027,2055,2053,2054,1025,1281,1537,1283,1539,513,515],"SupportedProtos":["h2"],"SupportedVersions":[772,771],"RemoteAddr":{"IP":"10.89.0.32","Port":33628,"Zone":""},"LocalAddr":{"IP":"10.89.0.32","Port":8443,"Zone":""}}}}
caddy-1    | {"level":"debug","ts":1730823380.2814481,"logger":"tls.handshake","msg":"choosing certificate","identifier":"localhost","num_choices":1}
caddy-1    | {"level":"debug","ts":1730823380.2814696,"logger":"tls.handshake","msg":"default certificate selection results","identifier":"localhost","subjects":["localhost"],"managed":true,"issuer_key":"local","hash":"863ccab3bfda2b546430f0046f0486b1a5177add34621adfcd350aae5ee0d9f5"}
caddy-1    | {"level":"debug","ts":1730823380.2814777,"logger":"tls.handshake","msg":"matched certificate in cache","remote_ip":"10.89.0.32","remote_port":"33628","subjects":["localhost"],"managed":true,"expiration":1730866558,"hash":"863ccab3bfda2b546430f0046f0486b1a5177add34621adfcd350aae5ee0d9f5"}
caddy-1    | {"level":"debug","ts":1730823380.283846,"logger":"http.handlers.reverse_proxy","msg":"selected upstream","dial":"zitadel:8080","total_upstreams":1}
zitadel-1  | 2024/11/05 16:16:20 ERROR: Failed to extract ServerMetadata from context

With Caddyfile:

caddy-1    | {"level":"debug","ts":1730823667.8643844,"logger":"events","msg":"event","name":"tls_get_certificate","id":"ae770161-1da8-400c-9746-3fa5be50264d","origin":"tls","data":{"client_hello":{"CipherSuites":[49195,49199,49196,49200,52393,52392,4865,4866,4867],"ServerName":"localhost","SupportedCurves":[29,23,24,25],"SupportedPoints":"AA==","SignatureSchemes":[2052,1027,2055,2053,2054,1025,1281,1537,1283,1539,513,515],"SupportedProtos":["h2"],"SupportedVersions":[772,771],"RemoteAddr":{"IP":"10.89.0.41","Port":39032,"Zone":""},"LocalAddr":{"IP":"10.89.0.41","Port":8443,"Zone":""}}}}
caddy-1    | {"level":"debug","ts":1730823667.8644094,"logger":"tls.handshake","msg":"choosing certificate","identifier":"localhost","num_choices":1}
caddy-1    | {"level":"debug","ts":1730823667.8644223,"logger":"tls.handshake","msg":"default certificate selection results","identifier":"localhost","subjects":["localhost"],"managed":true,"issuer_key":"local","hash":"dbf33036f79f0b3085611b859715ffc4ecd15e6ce0f9f2fb72cecf63e782758c"}
caddy-1    | {"level":"debug","ts":1730823667.8644297,"logger":"tls.handshake","msg":"matched certificate in cache","remote_ip":"10.89.0.41","remote_port":"39032","subjects":["localhost"],"managed":true,"expiration":1730866753,"hash":"dbf33036f79f0b3085611b859715ffc4ecd15e6ce0f9f2fb72cecf63e782758c"}
caddy-1    | {"level":"debug","ts":1730823667.8665938,"logger":"http.handlers.reverse_proxy","msg":"selected upstream","dial":"zitadel:8080","total_upstreams":1}
caddy-1    | {"level":"debug","ts":1730823667.8687286,"logger":"http.handlers.reverse_proxy","msg":"upstream roundtrip","upstream":"zitadel:8080","duration":0.002089246,"request":{"remote_ip":"10.89.0.41","remote_port":"39032","client_ip":"10.89.0.41","proto":"HTTP/2.0","method":"POST","host":"localhost:8443","uri":"/grpc.reflection.v1.ServerReflection/ServerReflectionInfo","headers":{"Content-Type":["application/grpc"],"User-Agent":["grpcurl/v1.9.1 grpc-go/1.61.0"],"Te":["trailers"],"Grpc-Accept-Encoding":["gzip"],"X-Forwarded-For":["10.89.0.41"],"X-Forwarded-Proto":["https"],"X-Forwarded-Host":["localhost:8443"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","server_name":"localhost"}},"headers":{"Content-Type":["application/grpc"],"X-Robots-Tag":["none"]},"status":200}

3. Caddy version:

v2.8.4 h1:q3pe0wpBj1OcHFZ3n/1nl4V4bxBrYoSoab7rL9BMYNk=

4. How I installed and ran Caddy:

Podman Compose

a. System environment:

Linux (Fedora Silverblue 41), x86_64, Podman 5.2.5

b. Command:

podman compose up

c. Service/unit/compose file:

docker-compose.yaml:

version: '3.8'

services:
  caddy:
    image: "caddy:latest"
    volumes:
      - './Caddyfile:/etc/caddy/Caddyfile:ro,z'
    #command: 'caddy reverse-proxy --debug --access-log --to h2c://zitadel:8080 --from localhost:8443'
    ports:
      - "8443:8443"
    networks:
      - 'zitadel'

  zitadel:
    restart: 'always'
    networks:
      - 'zitadel'
    image: 'ghcr.io/zitadel/zitadel:latest'
    command: 'start-from-init --masterkey "MasterkeyNeedsToHave32Characters" --tlsMode disabled'
    environment:
      - 'ZITADEL_DATABASE_POSTGRES_HOST=db'
      - 'ZITADEL_DATABASE_POSTGRES_PORT=5432'
      - 'ZITADEL_DATABASE_POSTGRES_DATABASE=zitadel'
      - 'ZITADEL_DATABASE_POSTGRES_USER_USERNAME=zitadel'
      - 'ZITADEL_DATABASE_POSTGRES_USER_PASSWORD=zitadel'
      - 'ZITADEL_DATABASE_POSTGRES_USER_SSL_MODE=disable'
      - 'ZITADEL_DATABASE_POSTGRES_ADMIN_USERNAME=postgres'
      - 'ZITADEL_DATABASE_POSTGRES_ADMIN_PASSWORD=postgres'
      - 'ZITADEL_DATABASE_POSTGRES_ADMIN_SSL_MODE=disable'
      - 'ZITADEL_EXTERNALSECURE=false'
      - 'ZITADEL_FIRSTINSTANCE_MACHINEKEYPATH=/machinekey/zitadel-admin-sa.json'
      - 'ZITADEL_FIRSTINSTANCE_PATPATH=/machinekey/pat.json'
      - 'ZITADEL_FIRSTINSTANCE_ORG_MACHINE_MACHINE_USERNAME=zitadel-admin-sa'
      - 'ZITADEL_FIRSTINSTANCE_ORG_MACHINE_MACHINE_NAME=Admin'
      - 'ZITADEL_FIRSTINSTANCE_ORG_MACHINE_MACHINEKEY_TYPE=1'
      - 'ZITADEL_FIRSTINSTANCE_ORG_MACHINE_PAT_EXPIRATIONDATE=2025-01-01T00:00:00Z'
    depends_on:
      db:
        condition: 'service_healthy'
    ports:
      - '8080:8080'
    volumes:
      - ./machinekey:/machinekey:z

  db:
    restart: 'always'
    image: postgres:16-alpine
    environment:
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=postgres
    networks:
      - 'zitadel'
    healthcheck:
      test: ["CMD-SHELL", "pg_isready", "-d", "db_prod"]
      interval: '10s'
      timeout: '30s'
      retries: 5
      start_period: '20s'

networks:
  zitadel:

d. My complete Caddy config:

CLI:

caddy reverse-proxy --debug --access-log --to h2c://zitadel:8080 --from localhost:8443

Caddyfile:

{
	debug
}

localhost:8443 {
	reverse_proxy h2c://zitadel:8080
}

5. Links to relevant resources:

2

I don’t think CLI supports the h2c:// shortcut. That’s exclusive to the Caddyfile. As the docs here explain, it’s short for setting the versions on the HTTP transport.

PRs welcome to add support for that, I guess. But it can get complicated to introduce.

Either way, using a Caddyfile is strongly recommended. Since you’re running Caddy in Docker anyway, you should prefer a config file. Much more power and control. CLI is only meant for quick-and-dirty local servers when developing and testing something, not for production usecases.

1 Like
3

Hi @francislavoie,

The problem is that my context makes it hard to pass a Caddyfile to the Caddy container, whereas I can easily use command line or environment variables.

Anyway, thanks for your answer.

4

Thanks a lot for explain.

5

I don’t understand. Does this make it infeasible to supply a bind mounted file?

If so, have you considered using a config in your Compose file itself? e.g.:

configs:
  Caddyfile:
    content: |
      example.com {
        # ...
      }
services:
  caddy:
    image: caddy:latest
    configs:
      - source: Caddyfile
        target: /etc/caddy/Caddyfile
1 Like