How to disable logs with a matcher?

[navi (GJ)]

1. Output of caddy version:

2.5.2-alpine docker image.

2. How I run Caddy:

I built a custom caddy image with the rate limiter plugin.

FROM caddy:2.5.2-builder-alpine AS builder

RUN xcaddy build \
--with github.com/mholt/caddy-ratelimit

FROM caddy:2.5.2-alpine

COPY --from=builder /usr/bin/caddy /usr/bin/caddy

a. System environment:

2.5.2-alpine docker image. Docker running on Ubuntiu 22.04.

b. Command:

docker-compose up

c. Service/unit/compose file:

services:
  caddy:
    container_name: caddy_rl
    image: caddy_rl
    restart: unless-stopped
    ports:
        - "80:80"
        # - "443:443"
    volumes:            
    - ./caddy/Caddyfile:/etc/caddy/Caddyfile
    # - $PWD/site:/srv
    - /home/.../caddie_data/:/data
    - /home/.../caddy_config:/config
    - /home/.../uploads/:/var/www/uploads/img/
    - /home/.../caddy_logs/:/var/log/

d. My complete Caddy config:

{
  admin off

  order rate_limit before basicauth

  log {
    output file /var/log/access.log {
    roll_size 40MiB
    roll_uncompressed
    roll_local_time
    }
  }  
}

(common) {
  header /* {
    -Server
  }
}

http://mtkk.localhost {
  @static_asset {
    path_regexp static \.(webp|svg|css|js|jpg|png|gif|ico|woff|woff2)$
  }

  @hashed_asset {
    path_regexp static \.(css|js)$
  }

  log

  header {
    # # disable FLoC tracking
    # Permissions-Policy interest-cohort=()

    # # enable HSTS
    # Strict-Transport-Security max-age=31536000;

    # disable clients from sniffing the media type
    X-Content-Type-Options nosniff

    # clickjacking protection
    X-Frame-Options DENY

    # keep referrer data off of HTTP connections
    Referrer-Policy no-referrer-when-downgrade
  }

  rate_limit {
    distributed
    zone static_example {
      key    static
      events 100
      window 1m
    }
	}

  root * /var/www/uploads/img/
  file_server @static_asset
  reverse_proxy adonis_app:3333
  encode zstd gzip

	header ?Cache-Control max-age=3600
	header @hashed_asset Cache-Control max-age=31536000

  import common
}

3. The problem I’m having:

It’s not a problem. I am new to Caddie and would like to ask:

• Is the api admin enabled by default?
• How to disable logs with a matcher, or what other pattern is commonly used? Basically I would like to disable access logs for static files.
• The logs are not working unless I move the log settings inside the site section. Why?

Also, please feel free to critique the whole config file.

4. Error messages and/or full log output:

5. What I already tried:

log @static_asset {}
log @static_asset off
These are not valid.

6. Links to relevant resources:

[francislavoie]

@navi

Is the api admin enabled by default?

Yes. It’s necessary to perform config reloads, because those are done via the API. The caddy reload command actually just does an HTTP request to the API to push the new config.

I strongly recommend keeping it on. Otherwise, you have to restart the whole container to change the config, which causes downtime. You can do zero-downtime config reloads with docker-compose exec -w /etc/caddy caddy caddy reload; sets the working directory to /etc/caddy so that the Caddyfile is “right there”, then executes a command in the caddy container, with the command being caddy reload.

@navi

How to disable logs with a matcher, or what other pattern is commonly used? Basically I would like to disable access logs for static files.

It’s currently not possible, but it is in the works.

caddyhttp: Implement `skip_log` handler

@navi

The logs are not working unless I move the log settings inside the site section. Why?

The log directive enables access logs. By default, access logs are not enabled (because it can be quite noisy).

The log global option (at the top) configures logs globally (all logs, including access logs, which are just one kind of logs that Caddy emits).

Typically, people want to write their access logs to one place, and the runtime logs (the rest) somewhere else.

@navi

- /home/.../caddie_data/:/data

caddy, not caddie

@navi

  @static_asset {
    path_regexp static \.(webp|svg|css|js|jpg|png|gif|ico|woff|woff2)$
  }

  @hashed_asset {
    path_regexp static \.(css|js)$
  }

Named matchers can be one-liners if you’re only using one matcher:

@static_asset path_regexp static \.(webp|svg|css|js|jpg|png|gif|ico|woff|woff2)$

@hashed_asset path_regexp static \.(css|js)$

Make sure to run caddy fmt --overwrite /etc/caddy/Caddyfile to clean up your config. Since it’s in Docker, you might need to run it like this:

docker-compose exec caddy caddy fmt --overwrite /etc/caddy/Caddyfile

You’ll notice the whitespace will be much more consistent, using tabs everywhere.

@navi

(common) {
  header /* {
    -Server
  }
}

You don’t need this. Removing the Server header is pointless. It is in no way harmful. It doesn’t reveal any information that couldn’t otherwise be figured out (e.g. by analyzing TLS handshake patterns etc).

@navi

  file_server @static_asset
  reverse_proxy adonis_app:3333

Keep in mind that Caddyfile directives are sorted according to a predetermined order. Since reverse_proxy orders higher than file_server, it will always run first, before file_server does. Which means Caddy will never serve static files for you, with this config.

Instead, you should use can use either route to override the order, or you can use handle blocks to make parts mutually exclusive. I tend to recommend handle more often.

http://mtkk.localhost {
	encode gzip

	@static path_regexp static \.(webp|svg|css|js|jpg|png|gif|ico|woff|woff2)$
	handle @static {
		root * /var/www/uploads/img
		file_server
	}

	handle {
		reverse_proxy adonis_app:3333
	}
}

Also, be careful with the header directive and matchers; see this article which explains:

Composing in the Caddyfile

Web server config files are mostly about expressing HTTP handling logic. Often, various handlers or middlewares need to be “composed” to form a cohesive HTTP handler that returns the desired responses. Structurally, config files are basically limited to two dimensions: rows and columns (or, lines and tokens). Syntactically, a config file’s second dimension can expand into more dimensions or scopes when blocks are opened (which are usually represented by curly braces { } and/or indentation, simi…

[navi (GJ)]

Thanks for the detailed answer! It’s great help. I will have to read it a couple times

So I have added a log global option and also a log directive. I would assume the log directive picked up the global option configs, but apparently that’s not the case?

[matt (Matt Holt)]

@navi

How to disable logs with a matcher, or what other pattern is commonly used? Basically I would like to disable access logs for static files.

Can I ask why you want to do this? What is it you are actually trying to achieve?

[navi (GJ)]

Hi Matt, logging static files adds no value to us and makes the logs a lot bigger. Also, logging is viewed as expensive when it comes to performance, so it’s an optimization.

[matt (Matt Holt)]

Have you thought about only storing logs you value then?

Also, how much of a hit is Caddy’s logging causing exactly? We use a high efficiency, zero-allocation logger. I’d be interested to see evidence of performance penalties caused by it.

Since this feature would be a little gnarly, I just want to make sure the cost is justified.

[navi (GJ)]

Have you thought about only storing logs you value then?

Yes, that is exactly what I want to do. Is there a way to do it without matchers? I am a complete newbie to Caddy. Hacked this config together yesterday based on the docs.

To be honest I have no benchmark to prove anything about logging. All I know is I have seen it in most nginx prod configs so I do it too.

[navi (GJ)]

The file server is not serving files, getting 404 on everything. And I don’t see logs at all.

Inside the Caddy container the files are there:

/var/www/uploads/img # ls 
blog   tanar
/var/www/uploads/img # ls tanar
kukiu-muki-120x120.webp  kukiu-muki-240x240.webp  makk-ancsi-240x240.webp  makk-ancsi-60x60.webp
kukiu-muki-180x180.webp  makk-ancsi-160x160.webp  makk-ancsi-320x320.webp  makk-ancsi-80x80.webp

I had to add order for log otherwise log couldn’t be added to handle. The docs don’t mention the default order of logs, or why it should be ordered inside a handle block.

Current Caddyfile:

{
	# admin off

	order rate_limit before basicauth
	order log before basicauth

	# log {
	# 	output file /var/log/access.log {
	# 		roll_size 40MiB
	# 		roll_uncompressed
	# 		roll_local_time
	# 	}
	# }
}

(common) {
	header /* {
		-Server
	}
}

http://mtkk.localhost {
	@static_asset {
		path_regexp static \.(webp|svg|css|js|jpg|png|gif|ico|woff|woff2)$
	}

	@hashed_asset {
		path_regexp static \.(css|js)$
	}

	header {
		# # disable FLoC tracking
		# Permissions-Policy interest-cohort=()

		# # enable HSTS
		# Strict-Transport-Security max-age=31536000;

		# disable clients from sniffing the media type
		#X-Content-Type-Options nosniff

		# clickjacking protection
		X-Frame-Options DENY

		# keep referrer data off of HTTP connections
		Referrer-Policy no-referrer-when-downgrade
	}

	handle @static_asset {
		root * /var/www/uploads/img
		file_server browse
	}

	handle {
		reverse_proxy adonis_app:3333
		log {
			output stdout
		}
	}

	encode zstd gzip

	header ?Cache-Control max-age=3600
	header @hashed_asset Cache-Control max-age=31536000

	import common

	log {
		output stdout
	}

	rate_limit {
		zone static {
			key static
			events 60
			window 1s
		}
	}
}

Edit: logs inside handle blocks seem to do nothing, so I guess I have to log everything or nothing.
Edit 2: looks like the proxy handler takes precedence so the file server never runs. Why?

[francislavoie]

@navi

I had to add order for log otherwise log couldn’t be added to handle. The docs don’t mention the default order of logs, or why it should be ordered inside a handle block.

It’s not correct to do that. The log directive is not a handler directive, so it cannot apply “sometimes”, it’s always “all the time”. Run caddy adapt --pretty --config Caddyfile to see the adapted JSON config, and you’ll see what I mean – the logging config is not part of the routes, it’s elsewhere, part of a top-level logging object in the JSON.

There’s a few other directives that work like this, such as tls, bind, and handle_errors, which don’t configure handler routes, but other “meta” things.

@navi

looks like the proxy handler takes precedence so the file server never runs. Why?

Directives are sorted according to this predetermined order:

Caddyfile Directives - Caddy Documentation

Using route or handle blocks with matchers is how you tell Caddy which handlers to use for a given request. I’ve explained that in my earlier comment.

The reason for the default order is to give a “good enough” default order that should work for most users. But if you’re using directives of a similar “class” together (like directives that write responses, such as file_server, reverse_proxy and respond) then we had to just make a decision as to which one we order where; that works for some users, and doesn’t for others. Not much we can do to make it work for everyone out of the box. But we provide a bunch of escape hatches to override that (like handle, route and the order global option). I strongly suggest using handle where possible, using order only when using plugins, and only then using route if you can’t figure out how to make it work otherwise.

[navi (GJ)]

Thanks again for your answers.

I stripped down the config to figure out why isn’t the file server working.
I am trying to reach /img/tanar/filename.webp. I tried matching /img, /img/, /img/*. None of them serve the file or the static file browser.

http://caddy.localhost {
	root /img/* /var/www/uploads/img
	file_server browse

	log {
		output stdout
	}
}

Result: 404 for everything, empty file browser on /.

If I change the matcher to *, I can reach the file browser under / and also the subfolders and images inside.

http://caddy.localhost {
	root * /var/www/uploads/img
	file_server browse

	log {
		output stdout
	}
}

It’s hard to understand

[francislavoie]

The way file_server works, is it takes the request path, and appends it to the root you defined. So if your root is /var/www/uploads/img and your request is /img/tanar/filename.webp, then it will look at:

/var/www/uploads/img/img/tanar/filename.webp

Notice there’s a doubled /img in there. That’s probably your issue.

You’ll need to either change your root (not ideal if you don’t want to serve things that are a folder up) or strip the /img part of the request path before handling the request.

One way to strip it is with the handle_path directive, which is a shortcut for a handle + a uri strip_prefix.

handle_path /img* {
	root * /var/www/uploads/img
	file_server
}

[navi (GJ)]

That did it, thanks!