How can I handle empty host variables?

1. Caddy version (caddy version):

v2.4.6 h1:HGkGICFGvyrodcqOOclHKfvJC0qTU7vny/7FhYp9hNw=

2. How I run Caddy:

Systemd

a. System environment:

Debian , Caddy compiled no .deb Package Version

d. My complete Caddyfile or JSON config:

{
    debug
    on_demand_tls {
        ask      http://{$API}/check/html
        interval 2m
        burst    5
    }

    storage s3 {
            host {$AWS_HOST}
            bucket {$CONFIG}
            access_id {$AWS_ACCESS_KEY_ID}
            secret_key {$AWS_SECRET_ACCESS_KEY}
            insecure false
            prefix "ssl"
    }

    order s3proxy first

    servers {
        protocol {
                experimental_http3
        }
    }

}

https:// {

    tls {
        on_demand
    }

    encode zstd gzip

    @wraxx header_regexp sub Host (.*)\.hi.{$PROVIDER}
    handle @wraxx {
        s3proxy {
            bucket {$SITES}
            region {$AWS_REGION}
            endpoint {$AWS_HOST}
            index  index.html
            root   /{re.sub.1}
            errors /{re.sub.1}/index.html
        }

        @cdn not path / */ *.html *.htm
        redir @cdn https://cdn.{$PROVIDER}/{re.sub.1}{uri} permanent
    }

    handle /formmailer {
        reverse_proxy {$API} {
            header_up Secure {host}
            header_up X-Forwarded-Host {host}
            header_up Authentication {$API_AUTH}
        }
    }

    handle {
        s3proxy {
            bucket {$SITES}
            region {$AWS_REGION}
            endpoint {$AWS_HOST}
            index  index.html
            root   /{host}
            errors /{host}/index.html
        }

        @cdn not path / */ *.html *.htm
        redir @cdn https://cdn.{$PROVIDER}/{host}{uri} permanent
    }

}

3. The problem I’m having:

Every other Requests I will encounter empty host headers from the same Browser Session (Refresh CTRL+R, Tested with Firefox and Chrome. Works with curl) and the log file will show {http.request.host} insted of the hostname. How can I handle this empty host variables?

4. Error messages and/or full log output:

Feb 01 11:03:50 wraxx-html-0001 bash[9696]: {"level":"debug","ts":1643709830.4053314,"logger":"http.handlers.s3proxy","msg":"get from S3","bucket":"wraxx","key":"/dash.simonjenny.dev/index.html"}
Feb 01 11:00:47 wraxx-html-0001 bash[9696]: {"level":"debug","ts":1643709647.560506,"logger":"http.handlers.s3proxy","msg":"get from S3","bucket":"wraxx","key":"/{http.request.host}/index.html"}

5. What I already tried:

I tried deactivate the s3proxy. Problem still existed.

Huh? Why would the request have an empty Host header? That sounds invalid.

Please take a look at your browser inspector’s Network tab to see what headers are being sent.

This is the request the Firefox sends when I get this error:

GET / HTTP/2
Host: dash.simonjenny.dev
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:96.0) Gecko/20100101 Firefox/96.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8
Accept-Language: de,en-US;q=0.7,en;q=0.3
Accept-Encoding: gzip, deflate, br
DNT: 1
Connection: keep-alive
Upgrade-Insecure-Requests: 1
Sec-Fetch-Dest: document
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: none
Sec-Fetch-User: ?1
If-Modified-Since: Tue, 01 Feb 2022 08:31:29 GMT
If-None-Match: "eebdf3a3c7f860e8d25116f91d3879ca"
Cache-Control: max-age=0
TE: trailers

it does not differ from a valid request at all.

Can you turn on the log directive on your site to emit access logs? Does host show as empty in the access logs as well?

This is the Log Entry for a faulty request:

{"level":"error","ts":1643728342.2497942,"logger":"http.log.access.log0","msg":"handled request","request":{"remote_addr":"84.73.131.131:62959","proto":"HTTP/2.0","method":"GET","host":"dash.simonjenny.dev","uri":"/","headers":{"Dnt":["1"],"Upgrade-Insecure-Requests":["1"],"Sec-Fetch-Mode":["navigate"],"Sec-Fetch-Site":["none"],"Sec-Fetch-User":["?1"],"If-None-Match":["\"eebdf3a3c7f860e8d25116f91d3879ca\""],"Te":["trailers"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8"],"Sec-Fetch-Dest":["document"],"If-Modified-Since":["Tue, 01 Feb 2022 08:31:29 GMT"],"Accept-Language":["de,en-US;q=0.7,en;q=0.3"],"Accept-Encoding":["gzip, deflate, br"],"Cache-Control":["max-age=0"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:96.0) Gecko/20100101 Firefox/96.0"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","proto_mutual":true,"server_name":"dash.simonjenny.dev"}},"common_log":"84.73.131.131 - - [01/Feb/2022:16:12:22 +0100] \"GET / HTTP/2.0\" 403 0","user_id":"","duration":0.021201968,"size":0,"status":403,"resp_headers":{"Server":["Caddy"],"Alt-Svc":["h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000"]}}

The request subsequently fails as a 403 because the s3proxy cannot resolve the key /{http.request.host}/index.html

This is a valid request:


{"level":"info","ts":1643728340.3514144,"logger":"http.log.access.log0","msg":"handled request","request":{"remote_addr":"84.73.131.131:62959","proto":"HTTP/2.0","method":"GET","host":"dash.simonjenny.dev","uri":"/","headers":{"Accept-Encoding":["gzip, deflate, br"],"Dnt":["1"],"Sec-Fetch-Mode":["navigate"],"Sec-Fetch-User":["?1"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:96.0) Gecko/20100101 Firefox/96.0"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8"],"Accept-Language":["de,en-US;q=0.7,en;q=0.3"],"Upgrade-Insecure-Requests":["1"],"Sec-Fetch-Dest":["document"],"Sec-Fetch-Site":["none"],"Cache-Control":["max-age=0"],"Te":["trailers"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","proto_mutual":true,"server_name":"dash.simonjenny.dev"}},"common_log":"84.73.131.131 - - [01/Feb/2022:16:12:20 +0100] \"GET / HTTP/2.0\" 200 526","user_id":"","duration":0.012387202,"size":526,"status":200,"resp_headers":{"Content-Encoding":["gzip"],"Vary":["Accept-Encoding"],"Server":["Caddy"],"Alt-Svc":["h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000"],"Content-Type":["text/html; charset=utf-8"],"Etag":["\"eebdf3a3c7f860e8d25116f91d3879ca\""],"Last-Modified":["Tue, 01 Feb 2022 08:31:29 GMT"]}}

Well, the host looks fine there.

So I have to guess this is a bug with the s3proxy plugin, not correctly replacing placeholders for that config value all the time.

:man_shrugging:

1 Like

Bummer… okay I will try to create a GH Issue.
Thanks for looking into this!

1 Like

This topic was automatically closed after 30 days. New replies are no longer allowed.