Responses not compressed with `encode gzip`

Help
1

1. Caddy version:

2.6.4

2. How I installed, and run Caddy:

xcaddy

a. System environment:

Linux/Ubuntu

b. Command:

caddy run

c. Service/unit/compose file:

N/A

d. My complete Caddy config:

foo.example.com {
    encode zstd gzip
    header {
        Content-Security-Policy "default-src 'self' 'unsafe-inline' 'unsafe-eval' data: ;"
        Referrer-Policy same-origin
    }
    log {
        output file /var/log/caddy/foo.log
    }
    reverse_proxy unix//run/foo/web.sock
    tls {
        dns cloudflare {env.CLOUDFLARE_API_KEY}
    }
}

3. The problem I’m having:

The responses are not compressed. I can confirm by not seeing Content-Encoding response header and both ‘size’ and ‘transfer size’ in firefox devtools show close values.

4. Error messages and/or full log output:

curl --compressed --dump-header - -o /dev/null https://foo.example.com/some-big.html
alt-svc: h3=":443"; ma=2592000
cache-control: no-store, no-transform
content-security-policy: default-src 'self' 'unsafe-inline' 'unsafe-eval' data: ;
content-security-policy: frame-src 'self'
content-type: text/html; charset=UTF-8
date: date of response
referrer-policy: same-origin
server: Caddy
set-cookie: Some cookie blah blah
x-frame-options: SAMEORIGIN

5. What I already tried:

I tried to reproduce with

  1. caddy’s file-server directive. Responses are compressed as expected
  2. Reverse proxy to python -m http.server. Also works . Responses are compressed.

One thing I noticed is that my real application does not send ‘Content-Length’ header. Does that cause caddy to not compress response? It does send the content-type header though which is compressible (text/html)

6. Links to relevant resources:

Side Note: Documentation does not have ‘brotli’ as compression option. But I see in source code. Not sure docs need to be updated or brotli is not ready yet.

2

I’m not sure about why it’s not encoding, actually. I’ve made a query about it. In the meantime:

There is a standard brotli sidecar module: https://caddyserver.com/docs/modules/http.precompressed.br, which should allow opportunistic serving pre-compressed assets adjacent to the requested file. However, I understand that the brotli encoder module is third-party: https://caddyserver.com/docs/modules/http.encoders.br, so you’d need to compile Caddy with GitHub - ueffel/caddy-brotli: Brotli compression encoder for Caddy included if you need this on-the-fly.

1 Like
3

Thanks! Understood now. Got misled by just looking at the directory names in source.

4

Update for you:

  • match is a response matcher. Only matching responses are encoded. The default looks like this:
match {
    header Content-Type text/*
    header Content-Type application/json*
    header Content-Type application/javascript*
    header Content-Type application/xhtml+xml*
    header Content-Type application/atom+xml*
    header Content-Type application/rss+xml*
    header Content-Type image/svg+xml*
}

—https://caddyserver.com/docs/caddyfile/directives/encode

Your Content-Type does match text/* so that’s all good there, but:

  • minimum_length the minimum number of bytes a response should have to be encoded (default: 512).

—https://caddyserver.com/docs/caddyfile/directives/encode

The encoder won’t operate unless we have at least 512 bytes to bother encoding, by default. Here’s where Caddy attempts to determine the content length:

It first checks if the buffered response we have is greater than the length; if it isn’t, it checks whether the supplied Content-Length meets the required length. I understand this would be useful in cases where the response is chunked but the first chunk is smaller than the minimum, in which event Caddy would trust the length reported by the upstream server instead.

So, I’d be leaning towards the issue here being that the response is too small to bother encoding.

1 Like
5

The responses are definitely much bigger than 512 bytes. Was ~ 65Kb. From the code, it looks like the length of first write call is only considered. I will try to put some logging around and check what is happening.

2 Likes
6

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.