H3-29 Access denied

Help
1

1. Caddy version (caddy version):

v2.4.3 h1:Y1FaV2N4WO3rBqxSYA8UZsZTQdN+PwcoOcAiZTM8C0I=

2. How I run Caddy:

a. System environment:

Debian GNU/Linux 10 (buster), Docker

b. Command:

docker-compose up -d

c. Service/unit/compose file:

version: "3.9"

services:
    proxy:
        build: ./proxy/dir
        volumes:
            - ./proxy/data/:/data/
            - ./proxy/config/:/config/
            - ./proxy/caddy/Caddyfile:/etc/caddy/Caddyfile
            - ./app/html/:/www/
        restart: unless-stopped
        expose:
            - "2019"
        ports:
            - "80:80"
            - "443:443"
            - "443:443/udp"

FROM caddy:builder-alpine AS builder

RUN xcaddy build \
    --with github.com/caddy-dns/cloudflare \
    --with github.com/porech/caddy-maxmind-geolocation

FROM caddy:alpine

RUN apk upgrade --no-cache && apk add tzdata --no-cache \
    && cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime \
    && echo "Asia/Shanghai" > /etc/timezone && apk del tzdata

COPY --from=builder /usr/bin/caddy /usr/bin/caddy

d. My complete Caddyfile or JSON config:

{
        acme_ca https://acme-v02.api.letsencrypt.org/directory
        key_type p256
        preferred_chains smallest
        servers :443 {
                protocol {
                        experimental_http3
                        strict_sni_host
                }
        }
}

:80 {
        redir https://steamstatus.cn{uri}
        header {
                X-Content-Type-Options nosniff
                X-Frame-Options DENY
                Referrer-Policy strict-origin-when-cross-origin
                Permissions-Policy "interest-cohort=()"
                -Server
        }
}

:443 {
        redir https://steamstatus.cn{uri}
        header {
                X-Content-Type-Options nosniff
                X-Frame-Options DENY
                Referrer-Policy strict-origin-when-cross-origin
                Permissions-Policy "interest-cohort=()"
                -Server
        }
}

steamstatus.cn {
        tls {
                dns cloudflare <key>
        }
        root * /www/steamstatus.cn/
        encode zstd gzip
        file_server
        header {
                Strict-Transport-Security "max-age=63072000; includeSubDomains; preload"
                X-Content-Type-Options nosniff
                X-Frame-Options DENY
                Referrer-Policy strict-origin-when-cross-origin
                Permissions-Policy "interest-cohort=()"
                -Server
        }

        @cmn-Hans {
                path_regexp cmn-Hans ^/cmn-Hans/(.*)$
        }
        redir @cmn-Hans /{http.regexp.cmn-Hans.1}

        @avif path *.avif
        header @avif Content-Type "image/avif"
}

3. The problem I’m having:

If I use h3-29 to access the website, I will be denied access (403).

4. Error messages and/or full log output:

{"level":"info","ts":1628647741.1526628,"logger":"http.log.access.log0","msg":"handled request","request":{"remote_addr":"<myip>:1038","proto":"HTTP/2.0","method":"GET","host":"steamstatus.cn","uri":"/","headers":{"Sec-Ch-Ua":["\"Chromium\";v=\"92\", \" Not A;Brand\";v=\"99\", \"Microsoft Edge\";v=\"92\""],"Dnt":["1"],"Upgrade-Insecure-Requests":["1"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.131 Safari/537.36 Edg/92.0.902.67"],"Sec-Fetch-Site":["none"],"Sec-Fetch-Dest":["document"],"Accept-Encoding":["gzip, deflate, br"],"Cache-Control":["max-age=0"],"Sec-Ch-Ua-Mobile":["?0"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"],"Sec-Fetch-Mode":["navigate"],"Sec-Fetch-User":["?1"],"Accept-Language":["zh-CN,zh-TW;q=0.9,zh;q=0.8,en;q=0.7,en-US;q=0.6,en-GB;q=0.5"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","proto_mutual":true,"server_name":"steamstatus.cn"}},"common_log":"61.146.161.48 - - [11/Aug/2021:10:09:01 +0800] \"GET / HTTP/2.0\" 200 3826","duration":0.007227059,"size":3826,"status":200,"resp_headers":{"Referrer-Policy":["strict-origin-when-cross-origin"],"Strict-Transport-Security":["max-age=63072000; includeSubDomains; preload"],"X-Content-Type-Options":["nosniff"],"Content-Type":["text/html; charset=utf-8"],"Vary":["Accept-Encoding"],"X-Frame-Options":["DENY"],"Content-Encoding":["gzip"],"Permissions-Policy":["interest-cohort=()"],"Alt-Svc":["h3=\":443\"; ma=2592000,h3-34=\":443\"; ma=2592000,h3-32=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000"],"Etag":["\"qxgpho7ku\""],"Last-Modified":["Sat, 07 Aug 2021 09:10:36 GMT"]}}
{"level":"error","ts":1628647742.0002728,"logger":"http.log.access.log0","msg":"handled request","request":{"remote_addr":"172.18.0.1:54987","proto":"HTTP/3","method":"GET","host":"steamstatus.cn","uri":"/favicon-32x32.png","headers":{"Accept":["image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8"],"Sec-Fetch-Site":["same-origin"],"Sec-Fetch-Dest":["image"],"Accept-Encoding":["gzip, deflate, br"],"Pragma":["no-cache"],"Cache-Control":["no-cache"],"Sec-Ch-Ua":["\"Chromium\";v=\"92\", \" Not A;Brand\";v=\"99\", \"Microsoft Edge\";v=\"92\""],"Sec-Ch-Ua-Mobile":["?0"],"Accept-Language":["zh-CN,zh-TW;q=0.9,zh;q=0.8,en;q=0.7,en-US;q=0.6,en-GB;q=0.5"],"Dnt":["1"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.131 Safari/537.36 Edg/92.0.902.67"],"Sec-Fetch-Mode":["no-cors"],"Referer":["https://steamstatus.cn/"]},"tls":{"resumed":false,"version":0,"cipher_suite":0,"proto":"","proto_mutual":true,"server_name":""}},"common_log":"172.18.0.1 - - [11/Aug/2021:10:09:02 +0800] \"GET /favicon-32x32.png HTTP/3\" 403 0","duration":0.0000231,"size":0,"status":403,"resp_headers":{"Server":["Caddy"],"Alt-Svc":["h3=\":443\"; ma=2592000,h3-34=\":443\"; ma=2592000,h3-32=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000"]}}
{"level":"error","ts":1628647742.3640606,"logger":"http.log.access.log0","msg":"handled request","request":{"remote_addr":"172.18.0.1:54987","proto":"HTTP/3","method":"GET","host":"steamstatus.cn","uri":"/favicon-16x16.png","headers":{"Sec-Fetch-Site":["same-origin"],"Sec-Fetch-Mode":["no-cors"],"Sec-Fetch-Dest":["image"],"Referer":["https://steamstatus.cn/"],"Sec-Ch-Ua-Mobile":["?0"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.131 Safari/537.36 Edg/92.0.902.67"],"Accept":["image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8"],"Accept-Encoding":["gzip, deflate, br"],"Accept-Language":["zh-CN,zh-TW;q=0.9,zh;q=0.8,en;q=0.7,en-US;q=0.6,en-GB;q=0.5"],"Sec-Ch-Ua":["\"Chromium\";v=\"92\", \" Not A;Brand\";v=\"99\", \"Microsoft Edge\";v=\"92\""],"Dnt":["1"]},"tls":{"resumed":false,"version":0,"cipher_suite":0,"proto":"","proto_mutual":true,"server_name":""}},"common_log":"172.18.0.1 - - [11/Aug/2021:10:09:02 +0800] \"GET /favicon-16x16.png HTTP/3\" 403 0","duration":0.00001395,"size":0,"status":403,"resp_headers":{"Server":["Caddy"],"Alt-Svc":["h3=\":443\"; ma=2592000,h3-34=\":443\"; ma=2592000,h3-32=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000"]}}
{"level":"error","ts":1628647748.410192,"logger":"http.log.access.log0","msg":"handled request","request":{"remote_addr":"172.18.0.1:54987","proto":"HTTP/3","method":"GET","host":"steamstatus.cn","uri":"/","headers":{"Cache-Control":["max-age=0"],"Dnt":["1"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"],"Sec-Fetch-Dest":["document"],"If-Modified-Since":["Sat, 07 Aug 2021 09:10:36 GMT"],"Sec-Ch-Ua":["\"Chromium\";v=\"92\", \" Not A;Brand\";v=\"99\", \"Microsoft Edge\";v=\"92\""],"Sec-Ch-Ua-Mobile":["?0"],"Sec-Fetch-Site":["none"],"Sec-Fetch-Mode":["navigate"],"Sec-Fetch-User":["?1"],"Upgrade-Insecure-Requests":["1"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.131 Safari/537.36 Edg/92.0.902.67"],"Accept-Encoding":["gzip, deflate, br"],"Accept-Language":["zh-CN,zh-TW;q=0.9,zh;q=0.8,en;q=0.7,en-US;q=0.6,en-GB;q=0.5"],"If-None-Match":["\"qxgpho7ku\""]},"tls":{"resumed":false,"version":0,"cipher_suite":0,"proto":"","proto_mutual":true,"server_name":""}},"common_log":"172.18.0.1 - - [11/Aug/2021:10:09:08 +0800] \"GET / HTTP/3\" 403 0","duration":0.000013939,"size":0,"status":403,"resp_headers":{"Server":["Caddy"],"Alt-Svc":["h3=\":443\"; ma=2592000,h3-34=\":443\"; ma=2592000,h3-32=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000"]}}

5. What I already tried:

I try to connect to the website via IPv6 or IPv4 single stack.
I don’t know what is causing this problem. Is it because GCP’s VNC network is special?

6. Links to relevant resources:

2

Hmm, strange.

Try adding debug to your global options, to see if anything more is written out to the logs when it happens.

Unfortunately, I’m not aware of any mechanism in the HTTP/3 stack that would cause 403s. I haven’t dug into the stack though, so it’s possible that something does, in quic-go (the underlying implementation that Caddy uses)

3

I added debug to the global options, and it did output a message about quic-go:

proxy_1  | {"level":"debug","ts":1628671053.5479918,"logger":"http","msg":"starting server loop","address":"[::]:443","http3":true,"tls":true}
proxy_1  | 2021/08/11 16:37:33 failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 2048 kiB, got: 416 kiB). See https://github.com/lucas-clemente/quic-go/wiki/UDP-Receive-Buffer-Size for details.

I ran sysctl -w net.core.rmem_max=2500000 command according to its requirements.

The situation has not changed, but I remembered that the MTU of the GCP VNC network is 1460. Is this the cause of the problem?
I changed the MTU to 1500, but it still has no effect.

4

@marten-seemann do you have any suggestions? Thanks!

5

The UDP receive buffer size is definitely unrelated.
There’s nothing in quic-go that should cause a 403 to be generated.

6

I tested it today and found that all of my devices have such problems, so the problem should appear on Docker.
I don’t know why it is proxying UDP traffic. In all devices, the incoming IP address displayed is 172.18.0.1.

I am currently researching this issue.

7

I found that it has nothing to do with this: I am connected via IPv6, but there is no iPv6 address in the bridge network, it needs to be converted.
I gave up and disabled HTTP/3 temporarily.

1 Like
8

This topic was automatically closed after 30 days. New replies are no longer allowed.