Caddy overwrites Content-Type headers from upstream

Xanazf · March 21, 2025, 10:17am

1. The problem I’m having:

I’m trying to setup M/Monit inside a docker-compose configuration together with Caddy.

When i set everything up, the Content-Type for application files (.js, .css) gets set to text/plain.

I’d contacted M/Monit to confirm if it was M/Monit doing that or not, and they’d said that M/Monit sends correct headers, they’d tested it on Nginx and Apache and everything worked, so it must be Caddy.

As you can see the request succeeds, but because of the combination of X-Content-Type-Options nosniff and Content-Type text/plain the console returns a MIME-type mismatch error.

path: {domain}/lib/js/application.js

HTTP/2 200 
accept-ranges: bytes
alt-svc: h3=":443"; ma=2592000
content-security-policy: frame-ancestors 'self'
content-type: text/plain
date: Fri, 21 Mar 2025 09:24:07 GMT
etag: "1738060188-461"
last-modified: Tue, 28 Jan 2025 10:29:48 GMT
permissions-policy: geolocation=(),microphone=()
referrer-policy: same-origin
server: Caddy
server: mmonit/4.3.4
strict-transport-security: max-age=63072000; includeSubDomains
x-content-type-options: nosniff
x-frame-options: SAMEORIGIN
x-xss-protection: 1; mode=block
content-length: 461
X-Firefox-Spdy: h2

2. Error messages and/or full log output:

there are no errors or logs since the request succeeds with 200 as far as Caddy is concerned

3. Caddy version:

v2.9.1 h1:OEYiZ7DbCzAWVb6TNEkjRcSCRGHVoZsJinoDR/n9oaY=

4. How I installed and ran Caddy:

Dockerfile

FROM caddy:latest

a. System environment:

Linux 6.8.0-31-generic #31-Ubuntu SMP PREEMPT_DYNAMIC Sat Apr 20 00:40:06 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Docker version 26.1.3, build 26.1.3-0ubuntu1~24.04.1

docker-compose version 1.29.2, build unknown

b. Command:

docker-compose --env_file ./.env up

c. Service/unit/compose file:

docker-compose.yml

services:
  caddy:
    build:
      context: "${CADDY_BIN}"
    container_name: "${COMPOSE_PROJECT_NAME}_caddy"
    restart: "unless-stopped"
    depends_on:
      - synapse
      - keycloak
    ports:
      - "${HOST_MACHINE_UNSECURE_HOST_PORT}:80"
      - "${HOST_MACHINE_SECURE_HOST_PORT}:443"
      - 2019:2019
    volumes:
      - ${CADDY_CONFIG}:/etc/caddy/Caddyfile
      - ${CADDY_LOG_DIR}:/var/log/caddy
      - "./data/caddy/:/caddydata:rw"
    environment:
      DOMAIN_NAME: ${DOMAIN_NAME}
      ALLOWED_IP: ${ALLOWED_IP}
      XDG_DATA_HOME: "/caddydata"
    extra_hosts:
      - "host.docker.internal:host-gateway"
  keycloak:
    user: "0:1001"  
    build:
      context: "${KC_BIN}"
    container_name: "${COMPOSE_PROJECT_NAME}_keycloak"
    restart: "unless-stopped"
    depends_on:
      - kc-db
    expose:
      - 8080
      - 8448
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8080/health/ready"]
      interval: 15s 
      timeout: 2s   
      retries: 15   
    volumes:
      - "./config/certs/kc.${DOMAIN_NAME}:/certs"
    environment:
      KC_BOOTSTRAP_ADMIN_USERNAME: "${KC_USERNAME}"
      KC_BOOTSTRAP_ADMIN_PASSWORD: "${KC_PASSWORD}"
      KC_HOSTNAME: "kc.${DOMAIN_NAME}"
      KC_HOSTNAME_PORT: "${KC_HTTP_PORT}"
      KC_HOSTNAME_STRICT_BACKCHANNEL: "true"
      KC_HEALTH_ENABLED: "true"
      KC_LOG_LEVEL: info
      KC_HTTPS_CERTIFICATE_FILE: "/certs/kc.${DOMAIN_NAME}.crt"
      KC_HTTPS_CERTIFICATE_KEY_FILE: "/certs/kc.${DOMAIN_NAME}.key"
      POSTGRES_PASSWORD: "${POSTGRES_PASSWORD}"
      POSTGRES_USER: "${KC_POSTGRES_USER}"
      POSTGRES_DB: "${KC_POSTGRES_DATABASE}"
    command: ["start-dev", "--http-port", "${KC_HTTP_PORT}", "--https-port", "${KC_HTTPS_PORT}"]

  coturn:
    user: "0:1001"  
    build:
      network: host 
      context: "${COTURN_BIN}"
    container_name: "${COMPOSE_PROJECT_NAME}_coturn"
    restart: "unless-stopped"
    volumes:
      - "${COTURN_CONFIG}:/etc/turnserver.conf:ro"
      - "${COTURN_DATA}:/srv/coturn"
      - "./config/certs/turn.${DOMAIN_NAME}/turn.${DOMAIN_NAME}.key:/etc/ssl/private/turnkey.key:ro"
      - "./config/certs/turn.${DOMAIN_NAME}/turn.${DOMAIN_NAME}.crt:/etc/ssl/certs/turncert.crt:ro"
    environment:
      COTURN_USERNAME: ${COTURN_USER}
      COTURN_PASSWORD: ${COTURN_PASSWORD}
    network_mode: host
  synapse:
    build:
      context: "${SYNAPSE_BIN}"
    container_name: "${COMPOSE_PROJECT_NAME}_synapse"
    restart: "unless-stopped"
    expose:
      - 8008
    depends_on:
      - syn-db
      - keycloak
      - coturn
    volumes:
      - ${SYNAPSE_DATA}:/data:rw
    environment:
      POSTGRES_PASSWORD: "${SYNAPSE_POSTGRES_PASSWORD}"
      POSTGRES_USER: "${SYNAPSE_POSTGRES_USER}"
      POSTGRES_DATABASE: "${SYNAPSE_POSTGRES_DATABASE}"
      SYNAPSE_SERVER_NAME: "${SYNAPSE_SERVER_NAME}"
      SYNAPSE_REPORT_STATS: "yes"
      #SYNAPSE_HTTP_PORT: 8008
      #SYNAPSE_CONFIG_DIR: /data
      #SYNAPSE_CONFIG_PATH: /data/homeserver.yaml
      #SYNAPSE_DATA_DIR: /data
      SYNAPSE_VOIP_TURN_MAIN_URL: "stun:turn.${DOMAIN_NAME}:5349"
      SYNAPSE_VOIP_TURN_USERNAME: ${COTURN_USER}
      SYNAPSE_VOIP_TURN_PASSWORD: ${COTURN_PASSWORD}
    extra_hosts:
      - "host.docker.internal:host-gateway"

  syn-db:
    image: postgres:latest
    container_name: "${COMPOSE_PROJECT_NAME}_${DATABASE}-synapse"
    restart: "unless-stopped"
    expose:
      - 5432
    volumes:
      - "${SYNAPSE_POSTGRES_INITDB_DIR}:/docker-entrypoint-initdb.d"
      - "${SYNAPSE_POSTGRES_DATA_DIR}:/var/lib/postgres/"
      - "${SYNAPSE_POSTGRES_LOG_DIR}:/var/log/postgres/"
    environment:
      POSTGRES_USER: "${SYNAPSE_POSTGRES_USER}"
      POSTGRES_PASSWORD: "${POSTGRES_PASSWORD}"
      POSTGRES_DB: "${SYNAPSE_POSTGRES_DATABASE}"
      POSTGRES_INITDB_ARGS: "--encoding=UTF-8 --lc-collate=C --lc-ctype=C"
        #healthcheck:
        #  test: ["CMD-SHELL", "pg_isready", "-U", "${SYNAPSE_POSTGRES_USER}"]
        #  interval: 1s
        #  timeout: 5s
        #  retries: 10
  kc-db:
    image: postgres:latest
    container_name: "${COMPOSE_PROJECT_NAME}_${DATABASE}-kc"
    restart: "unless-stopped"
    expose:
      - 5432
    volumes:
      - "${KC_POSTGRES_INITDB_DIR}:/docker-entrypoint-initdb.d"
      - "${KC_POSTGRES_DATA_DIR}:/var/lib/postgres/"
      - "${KC_POSTGRES_LOG_DIR}:/var/log/postgres/"
    environment:
      POSTGRES_USER: "${KC_POSTGRES_USER}"
      POSTGRES_PASSWORD: "${POSTGRES_PASSWORD}"
      POSTGRES_DB: "${KC_POSTGRES_DATABASE}"
      POSTGRES_INITDB_ARGS: "--encoding=UTF-8 --lc-collate=C --lc-ctype=C"
        #healthcheck:
        #  test: ["CMD-SHELL", "pg_isready", "-U", "${KC_POSTGRES_USER}"]
        #  interval: 1s
        #  timeout: 5s
        #  retries: 10

  mmonit:
    image: jchonig/mmonit:4.3.4
    restart: unless-stopped
    container_name: m_monit
    hostname: m_monit
  #  log_driver: "syslog"
  #  log_opt:
  #    syslog-tag: monit
    volumes:
     - ./monit.conf.d:/opt/mmonit-4.3.4/conf
     - /var/run/docker.sock:/var/run/docker.sock
     - /srv:/hostsrv:ro
     - /data:/data:ro
     - /cache:/cache:ro
     - /root:/hostroot:ro
     - /archive:/archive:ro
     - /backups:/backups:ro
    cap_add:
     - SYS_PTRACE   
    security_opt:   
     - apparmor:unconfined
    expose:
     - 8080
    depends_on:
      - monit
  monit:
    user: "911"
    image: maltyxx/monit:latest
    restart: unless-stopped
    container_name: monit
    hostname: monit 
    volumes:
      - ./monit.conf.d/monitrc:/etc/monitrc
    expose:
      - 2812

d. My complete Caddy config:

{$DOMAIN_NAME} {
        root * {$DOCUMENT_ROOT}
        encode zstd gzip
        file_server 

        header /* { 

                # Require HTTPS for subdomains, too
                Strict-Transport-Security "max-age=31536000; includeSubDomains; preload"

                # Disable MIME type sniffing
                X-Content-Type-Options nosniff

                # Don't allow embedding in other sites
                X-Frame-Options sameorigin

                # Only send the origin to other sites
                Referrer-Policy strict-origin-when-cross-origin

                # As strict as possible without breaking the site
                # Content-Security-Policy "default-src https:; font-src https: data:; img-src https: data: 'self' about:; script-src 'unsafe-inline' https: data:; style-src 'unsafe-inline' https:; connect-src https: data: 'self'" 

                # Disable powerful features we don't need
                Permissions-Policy "geolocation=(), camera=(), microphone=() interest-cohort=()"
        }

        respond /.git/* "Access denied" 403 {
                close
        }

        header /.well-known/matrix/server {
                Content-Type application/json
        }
        respond /.well-known/matrix/server 200 {
                body {"m.server":"matrix.localhost:443"}
                close
        }
        header /.well-known/matrix/client {
                Content-Type application/json
        }
        respond /.well-known/matrix/client 200 {
                body {"m.homeserver":{"base_url":"https://matrix.localhost"}}
                close
        }
}

matrix.{$DOMAIN_NAME} {
        # @blocked not client_ip {$ALLOWED_IP} # add more, space separated
        # respond @blocked "Access Denied {client_ip}" 403
        @site {
                path *
                not path .git
        }
        reverse_proxy @site synapse:8008 {
                transport http {
                        read_buffer 300KiB
                        write_buffer 300KiB
                        dial_timeout 30s
                        read_timeout 60s
                        write_timeout 60s
                }   
                flush_interval 10s
                request_buffers 16MB
                response_buffers 16MB
                stream_timeout 120s
                stream_close_delay 30s
        }
}

kc.{$DOMAIN_NAME} { 
        @site {
                path *
                not path .git
        }
        reverse_proxy @site keycloak:8448 {
                transport http {
                        read_buffer 300KiB
                        write_buffer 300KiB
                        dial_timeout 30s
                        read_timeout 60s
                        write_timeout 60s
                        tls_server_name kc.{$DOMAIN_NAME}
                }
                flush_interval 10s
                request_buffers 16MB
                response_buffers 16MB
                stream_timeout 120s
                stream_close_delay 30s
        }
}

turn.{$DOMAIN_NAME} {
        @site {
                path *
                not path .git
        }
        reverse_proxy @site coturn:5349 {
                transport http {
                        read_buffer 300KiB
                        write_buffer 300KiB
                        dial_timeout 30s
                        read_timeout 60s
                        write_timeout 60s
                        tls_server_name turn.{$DOMAIN_NAME}
                }
                flush_interval 10s
                request_buffers 16MB
                response_buffers 16MB
                stream_timeout 120s
                stream_close_delay 30s
        }
}

monitor.{$DOMAIN_NAME} {
        @site {
                path *
                not path .git
        }
        reverse_proxy @site mmonit:8080 {
                transport http {
                        read_buffer 300KiB
                        write_buffer 300KiB
                        dial_timeout 30s
                        read_timeout 60s
                        write_timeout 60s
                }
                flush_interval 10s
                request_buffers 16MB
                response_buffers 16MB
                stream_timeout 120s
                stream_close_delay 30s
        }
}

5. Links to relevant resources:

timelordx · March 21, 2025, 5:38pm

This might be a longer response than necessary, so if you just want to see a possible solution, scroll to the end of my post

I don’t believe Caddy would mess with proxied Content-Type headers.

For example:

Caddyfile:

{
	http_port 8080
}

:8080 {
	reverse_proxy https://www.w3schools.com {
		header_up Host www.w3schools.com
	}
}

Run Caddy:

$ caddy run --config Caddyfile

Check headers:

## JavaScript
$ curl -sI https://www.w3schools.com/lib/common-deps.js | grep -iE '^content-type'
content-type: application/javascript

$ curl -sI http://localhost:8080/lib/common-deps.js | grep -iE '^content-type'
Content-Type: application/javascript

## CSS
$ curl -sI https://www.w3schools.com/lib/w3schools32.css | grep -iE '^content-type'
content-type: text/css

$ curl -sI http://localhost:8080/lib/w3schools32.css | grep -iE '^content-type'
Content-Type: text/css

As you can see, the proxied data retains the original Content-Type.

Instead of relying on what Monit folks say, I’d check it myself. Instead of running:

curl -I {domain}/lib/js/application.js

try talking directly to Monit:

curl -I {monit}:8080/lib/js/application.js

and see what Monit actually returns.

If needed, you can enforce the Content-Type header in Caddy. In fact, to mess it up on purpose, I’d have to explicitly override it:

{
	http_port 8080
}

:8080 {
	reverse_proxy https://www.w3schools.com {
		header_up Host www.w3schools.com
		header_down -Content-Type
	}
	
	@css path *.css
	header @css Content-Type "text/css3"

	@js path *.js
	header @js Content-Type "application/javascript5"
}

Check the enforced headers:

$ curl -sI http://localhost:8080/lib/common-deps.js | grep -iE '^content-type'
Content-Type: application/javascript5

$ curl -sI http://localhost:8080/lib/w3schools32.css | grep -iE '^content-type'
Content-Type: text/css3

Xanazf · March 24, 2025, 11:06am

you were right, M/Monit’s localhost returns text/plain within the docker container, i’ll go ask them, though imma keep this thread open for now cuz maybe there’s something in my Caddy config that’s wrong

Mohammed90 · March 24, 2025, 11:35am

The Docker image is likely missing the mailcap package.