Disabling JSON Returns for File Server Browse

Help
1

1. The problem I’m having:

I am currently hosting a file directory on a domain, I have extracted the default template and slightly modified it to change out the text from the default “Running on Caddy” to “Hosted by TypicalNerds”.

  • I run my site through cloudflare to provide ddos protection and have caching enabled on non-media.
  • I’ve noticed that the site periodically returns a JSON file, likely when a bot scrapes the site instead of a user opening it, Cloudflare is then caching this response and serving it to normal users and it’s causing issues as users quite understandably, cannot make sense of a wall of json text being thrown at them.
  • My current goal is to try and disable the JSON returns and always serve html, it’s current unwanted behaviour seems to reflect the default behaviour according to the documentation I read on the caddy site as I believe that will resolve the issue.
  • As a troubleshooting step, I did a big no no and asked chatgpt to help me remove the header tag and it seemed to work for a couple days but this solution in the end didn’t work, as the behaviour has returned.

2. Error messages and/or full log output:

JSON above is an example of broken data located at:

/Archives/prerolls.video/CaptPharos/

3. Caddy version:

v2.10.2

4. How I installed and ran Caddy:

I don’t remember exactly how I installed it, but I know it’s installed as a linux package and running as a systemd service. (Friend helped me set it up)

a. System environment:

Debian 12
Caddy Installed and setup as System Service

b. Command:

starting caddy: systemctl start caddy

changing config: caddy reload --config /etc/caddy/Caddyfile

restarting caddy: systemctl restart caddy

d. My complete Caddy config:

# Pre-Roll Community Site
prerolls.typicalnerds.uk {
        tls /certificates/prerolls/cert.pem /certificates/prerolls/privkey.pem
        root * /mnt/prerolls
        @json header Accept application/json
        request_header @json -Accept
        handle {
                file_server {
                        root /mnt/prerolls
                        browse /prerollserver/browsertemplate.html
                }
        }
#       Uncomment to put site in maintenance mode.
#       respond "🚧 Site Under Maintenance 🚧 Please check back soon." 503
}

5. Links to relevant resources:

2

woah really? Cloudflare isn’t accounting for the Accept header? That seems bad.

3

Try matching like this

@json header Accept *json*
4

Cloudflare caching is broken by design.

Caddy sends Vary: Accept to say that the Accept key from the request is important for caching. Cloudflare ignores this.

Cloudflare does not consider vary values in caching decisions.

3 Likes
5

The day after I made the original post I updated my cache config in CF when I noticed I can set it so that if “application/json” is detected in the “accept” header that seems like it’s bypassing the cache, I’d still like to know how to disable it within caddy as well just in case.
I tried to reply yesterday but it for some reason I had a 2 hour cooldown.

6

Gave this a shot, still no change in behaviour.
I’m probably gonna have to look for some third party program to handle this site, even though I just need to host a file browser.

7

Maybe you can add Cache-Control: no-store for requests that end in / (directory listings)? Do listings need to be cached at all?

@directory path */
header @directory Cache-Control no-store
2 Likes