About respond encoding

wordlesswind · November 12, 2021, 4:05pm

1. Caddy version (`caddy version`):

v2.4.6 h1:HGkGICFGvyrodcqOOclHKfvJC0qTU7vny/7FhYp9hNw=

2. How I run Caddy:

a. System environment:

Debian GNU/Linux 11 (bullseye), Docker Engine - Community 20.10.10

b. Command:

docker-compose up -d

c. Service/unit/compose file:

Dockerfile

FROM caddy:builder-alpine AS builder

RUN xcaddy build \
    --with github.com/caddy-dns/cloudflare

FROM caddy:alpine

RUN apk upgrade --no-cache && apk add tzdata --no-cache \
    && cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime \
    && echo "Asia/Shanghai" > /etc/timezone && apk del tzdata

COPY --from=builder /usr/bin/caddy /usr/bin/caddy

docker-compose.yml

version: "3.9"

services:
    proxy:
        build: ./proxy/dir
        restart: unless-stopped
        expose:
            - "2019"
        ports:
            - "80:80"
            - "443:443"
            - "443:443/udp"

d. My complete Caddyfile or JSON config:

{
        acme_ca https://acme-v02.api.letsencrypt.org/directory
        key_type p256
        preferred_chains smallest
        servers :443 {
                protocol {
                        strict_sni_host
                }
        }
}

example.com {
        tls {
                dns cloudflare <key>
        }
        encode zstd gzip
        respond "网站正在建设当中……"
        header {
                Content-Type "text/plain; charset=utf-8"
        }
}

# Refer to the Caddy docs for more information:
# https://caddyserver.com/docs/caddyfile

3. The problem I’m having:

Since the Simplified Chinese environment of Windows works on GBK encoding, if the web page does not declare the encoding, the browser will use GBK encoding by default.

I don’t know how Caddy uses respond, but I use UTF-8 encoding to save Caddyfile. So Caddy should respond through UTF-8.
This caused me to display garbled characters when I visited the Chinese webpage.
I can only add the header manually to declare the encoding used. Could Caddy add an automatic declaration mechanism?

4. Error messages and/or full log output:

None

5. What I already tried:

header {
        Content-Type "text/plain; charset=utf-8"
}

6. Links to relevant resources:

None

francislavoie · November 12, 2021, 10:37pm

wordlesswind:

    proxy:
        build: ./proxy/dir
        restart: unless-stopped
        expose:
            - "2019"
        ports:
            - "80:80"
            - "443:443"
            - "443:443/udp"

Please don’t forget to use a volume for the /data directory, otherwise you risk losing your certs and keys, and potentially hit rate limits if Caddy gets stuck in a restart loop. See the docs on Docker about this.

I also don’t see a mount for /etc/caddy/Caddyfile here, and I don’t see the Caddyfile being copied in your Dockerfile. How are you making sure Caddy runs from the right config?

Hmm. I can’t replicate the issue, it works for me. I saved the following in a file, UTF-8 encoded (used VS Code to create and save the file)

{
    admin off
    debug
}

:8881 {
        encode zstd gzip
        respond "网站正在建设当中……"
        header {
                Content-Type "text/plain; charset=utf-8"
        }
}

Then I ran Caddy with this Caddyfile, and used both curl and Firefox to confirm that it looks okay:

Do note that Caddy will not compress/encode responses under a certain size, due to being inefficient/wasteful. By default this minimum is 512 bytes.

I tried copy-pasting that string repeatedly to make it bigger than the minimum, to make sure compression also works, and it does (Firefox shows it received a compressed response).

respond "网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……"

wordlesswind · November 13, 2021, 5:15am

Sorry, I removed volume roughly, it exists in the actual docker-compose.yml.

In order to better reflect the problem, I used Caddy for Windows to run Caddy on my local device.

Caddyfile:

:2019 {
        respond "网站正在建设当中……"
}

As I said before, the Windows system of the browser in the Simplified Chinese environment works in GBK encoding.
Google Chrome

The console is displayed normally, because the console of Windows 10 and above is working in UTF-8 encoding.

The situation of Mozilla Firefox is rather dumbfounding. Since it only accepts text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8, Caddy responds 404. Even if I add the header Content-Type "text/html; charset=utf-8", it doesn’t work. It seems that Caddy treats respond as plain by default, so it directly returns 404.

:2019 {
        respond "网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……网站正在建设当中……"
}

francislavoie · November 13, 2021, 5:43am

That’s actually not Caddy responding there… Notice there’s no Server: Caddy in the response. I don’t know what server you’re connecting to instead, but… It’s not Caddy.

Other than that, I’m sorry, I don’t think I’ll be much more help because I’ve never even heard of GBK encoding. Encodings aren’t something I ever need to care about as an English speaker, I guess.

wordlesswind · November 13, 2021, 7:08am

That is the problem what I mean. The response design does not take into account the experience of non-ASCII users.
This makes me have to add headers, although it is easy. But this is still a kind of design flaw.

And Mozilla Firefox did visit Caddy.

francislavoie · November 13, 2021, 7:22am

You’re sending requests to the admin API, not to your configured site.

system · December 12, 2021, 4:06pm

This topic was automatically closed after 30 days. New replies are no longer allowed.