How to filter out requests with status 200 from log

saldorin · June 16, 2025, 9:51am

1. The problem I’m having:

Hi there!
I’m using Caddy mainly as a reverse proxy for my services running in containers.
I’m splitting my logging for each subdomain to have finer grained fail2ban filtering available.
Works great so far, but Silverbullet is sending A LOT of requests (Log grows to several MB in a couple of minutes), even while idling - logrotate is fine and all, but i also want to limit ssd-wear.

My wish is to filter out Status 200 out of the logs, but so far i haven’t had any luck with any method or hint i found.

My current attempt looks like this:

# general Logging
{
    log {
        output file /var/log/caddy/access.log {
            roll_size 10MB
            roll_keep 10
        }
        format json
    }
}
# logging for subdomain/service
silverbullet.saldorin.duckdns.org {
        reverse_proxy localhost:3000 {
                header_up X-Real-IP {remote_host}
        }

        @not200 not expression `{err.status_code} != 200`

        log_skip @not200
        log {
                output file /var/log/caddy/silverbullet.access.log
        }
}

Unfortunately, this still logs status 200 requests…
Hopefully, someone can point me in the right direction?

2. Error messages and/or full log output:

No related output in the general log. Example log of Silverbullet-request:

{"level":"info","ts":1750066419.574309,"logger":"http.log.access.log1","msg":"handled request","request":{"remote_ip":"192.168.1.1","remote_port":"54714","client_ip":"192.168.1.1","proto":"HTTP/3.0","method":"GET","host":"silverbullet.saldorin.duckdns.org","uri":"/service_worker.js","headers":{"Service-Worker":["script"],"Accept-Encoding":["gzip, deflate, br, zstd"],"Dnt":["1"],"Sec-Fetch-Site":["same-origin"],"Cache-Control":["max-age=0"],"Sec-Fetch-Mode":["same-origin"],"Priority":["u=4"],"Accept":["*/*"],"Alt-Used":["silverbullet.saldorin.duckdns.org"],"Sec-Fetch-Dest":["serviceworker"],"Accept-Language":["en-US,en;q=0.5"],"User-Agent":["Mozilla/5.0 (X11; Linux x86_64; rv:139.0) Gecko/20100101 Firefox/139.0"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h3","server_name":"silverbullet.saldorin.duckdns.org"}},"bytes_read":0,"user_id":"","duration":0.003773312,"size":18175,"status":200,"resp_headers":{"Via":["1.1 Caddy"],"Cache-Control":["no-cache"],"Vary":["Accept-Encoding"],"Content-Type":["application/javascript"],"Content-Encoding":["br"],"Content-Length":["18175"],"Date":["Mon, 16 Jun 2025 09:33:39 GMT"]}}

3. Caddy version:

v2.10.0 h1:fonubSaQKF1YANl8TXqGcn4IbIRUDdfAkpcsfI/vX5U=

4. How I installed and ran Caddy:

a. System environment:

Debian 12 x86 VM running atop Proxmox, managed by systemd, as described in Install — Caddy Documentation

b. Command:

systemctl start caddy.service

c. Service/unit/compose file:

# caddy.service
#
# For using Caddy with a config file.
#
# Make sure the ExecStart and ExecReload commands are correct
# for your installation.
#
# See https://caddyserver.com/docs/install for instructions.
#
# WARNING: This service does not use the --resume flag, so if you
# use the API to make changes, they will be overwritten by the
# Caddyfile next time the service is restarted. If you intend to
# use Caddy's API to configure it, add the --resume flag to the
# `caddy run` command or use the caddy-api.service file instead.

[Unit]
Description=Caddy
Documentation=https://caddyserver.com/docs/
After=network.target network-online.target
Requires=network-online.target

[Service]
Type=notify
User=caddy
Group=caddy
ExecStart=/usr/bin/caddy run --environ --config /etc/caddy/Caddyfile
ExecReload=/usr/bin/caddy reload --config /etc/caddy/Caddyfile --force
TimeoutStopSec=5s
LimitNOFILE=1048576
PrivateTmp=true
ProtectSystem=full
AmbientCapabilities=CAP_NET_ADMIN CAP_NET_BIND_SERVICE

[Install]
WantedBy=multi-user.target

d. My complete Caddy config:

# The Caddyfile is an easy way to configure your Caddy web server.
#
# Unless the file starts with a global options block, the first
# uncommented line is always the address of your site.
#
# To use your own domain name (with automatic HTTPS), first make
# sure your domain's A/AAAA DNS records are properly pointed to
# this machine's public IP, then replace ":80" below with your
# domain name.


# Create log that is parceable for fail2ban
{
    log {
        output file /var/log/caddy/access.log {
            roll_size 10MB
            roll_keep 10
        }
        format json
    }
}

# Setup DNS API
*.saldorin.duckdns.org {
        tls {
                dns duckdns {
                        token xxx
                }
        }
}

# Setup base dir
 :80, :443 {
        respond /health "Im healthy" 200

        log
}

# Refer to the Caddy docs for more information:
# https://caddyserver.com/docs/caddyfile


silverbullet.saldorin.duckdns.org{
        reverse_proxy localhost:3000 {
                header_up X-Real-IP {remote_host}
        }

        @not200 not expression `{err.status_code} != 200`

        log_skip @not200
        log {
                output file /var/log/caddy/silverbullet.access.log
        }
}

[other services...]

5. Links to relevant resources:

Some posts/suggestions that i tried to implement/used for further research:

https://caddy.community/t/exclude-certain-requests-from-logs-based-on-request-path/15607
https://caddy.community/t/how-to-disable-logs-with-a-matcher/16882 (max 4 links allowed )

Relevant issue (i think?):

github.com/caddyserver/caddy

[log_skip] Suppress logging for HTTP errors raised via `error` directive

opened 07:55PM - 14 Oct 24 UTC

bminer

Opening issue per @francislavoie here: https://caddy.community/t/suppress-loggin…g-for-http-errors-raised-via-error-directive/25959 ## Problem Caddy always emits a log to `http.log.error` when the `error` directive is used. I would like to ignore specific errors like `error 404` and `error 403`, for example. It appears that `log_skip` and `log_name` only work for access logs (i.e. `http.log.access`), not for error logs (i.e. `http.log.error`). To be clear, I still want DEBUG level logs for `http.log.error` (because things like bad Go templates emit DEBUG level errors), but I don't want 404 errors. ## Proposed Solution Ideally, I'd like to add something to my `error_handler` like: ``` @noLog expression `{err.status_code} in [301, 302, 307, 308, 403, 404]` log_skip @noLog ``` ...but as described above, this does not work at the moment. Another option might be to suppress logging when using the `error` directive, for example: ``` error 404 { log_skip } ``` ## Example Log Entry ``` { "level": "debug", "ts": "2024-10-11T18:15:00.093Z", "logger": "http.log.error", "msg": "", "request": { "remote_ip": "REDACTED", "remote_port": "REDACTED", "client_ip": "REDACTED", "proto": "HTTP/2.0", "method": "POST", "host": "example.blakeminer.com", "uri": "/404", "headers": { "User-Agent": [ "curl/7.81.0" ], "Accept": [ "*/*" ], "Content-Type": [ "application/json" ], "Content-Length": [ "2" ] }, "tls": { "resumed": false, "version": 772, "cipher_suite": 4865, "proto": "h2", "server_name": "example.blakeminer.com" } }, "duration": 9.6901e-5, "status": 404, "err_id": "REDACTED", "err_trace": "caddyhttp.StaticError.ServeHTTP (staticerror.go:109)" } ``` ## Other Info Caddy v2.8.4 h1:q3pe0wpBj1OcHFZ3n/1nl4V4bxBrYoSoab7rL9BMYNk= Installed on Ubuntu using apt package manager: `APT-Sources: https://dl.cloudsmith.io/public/caddy/stable/deb/debian any-version/main amd64 Packages` Ubuntu 22.04.5 LTS x86_64 Launched with systemd 249 ## Example Caddyfile ``` { # grace period for clients on server reload grace_period 60s log default { output file /var/log/caddy/default.log { roll_size 10MiB roll_keep 5 roll_keep_for 30d } format json { time_format iso8601 } exclude http.log.access http.log.error } log error { output file /var/log/caddy/errors.log { roll_size 10MiB roll_keep 2 roll_keep_for 30d } format json { time_format iso8601 } level DEBUG include http.log.error } } (hsts) { @enabled not vars noHSTS true # Default age is 180 days header @enabled Strict-Transport-Security "max-age=15552000" } (main_site) { import hsts encode zstd gzip handle_errors { # Just respond with 404.html or error.html root * /var/www/default import no_cache try_files /{host}-{http.error.status_code}.html /{http.error.status_code}.html /error.html templates file_server # For redirects, just redirect @redir { expression `{err.status_code} in [301, 302, 307, 308]` expression `{http.response.header.Location} != ""` } respond @redir "" # For JSON or non-GET requests, respond with JSON error message @json not { not header_regexp Content-Type \/json$ method GET } route @json { header Content-Type application/json respond `\{"message":"{http.error.status_text}","status":{http.error.status_code}\}` } } } example.blakeminer.com { import main_site log_skip error 404 } ```

timelordx · June 16, 2025, 8:39pm

silverbullet.saldorin.duckdns.org {
    reverse_proxy localhost:3000 {
        header_up X-Real-IP {remote_host}

        @200 {
            status 200
        }
        handle_response @200 {
            log_skip
        }
    }

    log {
        output file /var/log/caddy/silverbullet.access.log
    }
}

saldorin · June 17, 2025, 8:33am

Ah, i had tried that handle_response (with a slightly different matcher), but not inside the reverse_proxy statement i think? Although that is indeed what my second linked thread suggested…oh well…

With your suggestion, it doesn’t quite work, as silverbullet doesn’t work anymore (curiously, i get 308 with curl that is not logged and 404 with a browser, that IS logged..), but when including copy_response in the handle_response section, it finally excludes 200 from the logs
Thank you!

Now that the log isn’t as cluttered, i see that there are quite some 304s as well - it seems like the matcher works as an OR-statement by default?
From a quick test, it seems like

 @200or304 {
            status 200
            status 304
        }

catches both 200 and 304 responses - i would have assumed that these matchers behave more like AND-statements
Or is there something i’m missing with this?

timelordx · June 17, 2025, 9:29am

Same matchers are OR’ed, different matchers are AND’ed.

@or1 {
    status 200 304
}

@or2 {
    status 200
    status 304
}

@or3 {
    header key1 value1
    header key1 value2
}

@and1 {
    header key1 value1
    header key2 value2
}

@and2 {
    method POST
    path /somepath
}

etc.

saldorin · June 17, 2025, 9:44am

Ah got it - kind of makes sense