Logging tweaks for better useability

Help
1

1. The problem I’m having:

I am looking to improve the readability of my log, as well as create some new logs. Specifically. my log right now is massive and very hard to read.

  1. How do I rotate my logfile or limit it to a certain size?
  2. How can I reduce what is shown in my log? Or, can I just create an additional log that has just what I want?
  3. Specifically I want another log that only shows IPs that are being blocked.

3. Caddy version:

v 2.8.4

4. How I installed and ran Caddy:

a. System environment:

Windows 10, downloaded the Caddy exe

b. Command:

caddy run

d. My complete Caddy config:

myserv.sysnet.me {
	@blocked not remote_ip 103.1.22.211 165.44.81.12
	respond @blocked "Server Offline" 403

	log {
		output stderr
		output file caddylog.txt
		level info
	}
}
2

See the docs log (Caddyfile directive) — Caddy Documentation

That’s vague. What do you mean?

You can use GitHub - caddyserver/transform-encoder: Log encoder module for custom log formats to manipulate the log output.

This doesn’t make sense, you can only have one output at a time. Only the last output in your config in one log will be taken.

If you want two separate logs, you can do that, but you need to make them two different named logs in the same site. Like log a { and log b {

2 Likes
3 
myserv.sysnet.me {
	@blocked not remote_ip 103.1.22.211 165.44.81.12
	respond @blocked "Server Offline" 403

	output file caddylog.txt {
		roll_keep_for 240h
	}
}

I made this change which hopefully will keep the log down to 10 days duration. Is this correct, or do I still need a log { } section?

I’m having a hard time with the transform encoder. Is there a more simple way, or even a value I can search on to see when an IP is blocked? Even if I don’t manipulate it as-is – it was just hard to read since it’s been such a massive file.

And by reducing what is shown, I think I may need to take down the log level? Basically trying to have it just report less in the file, but the 10 day limit might be good.

4

You still need to encapsulate this in a log directive.

I think that format transform "{common_log}" is about as simple as it’s going to get, unfortunately.

If you’ve got a tool like jq you can filter for it with the default access logs.

Caddy’s default access logs look a little something like this:

2024/08/30 00:05:42.988	info	http.log.access.log0	handled request	{"request": {"remote_ip": "::1", "remote_port": "52657", "client_ip": "::1", "proto": "HTTP/2.0", "method": "GET", "host": "localhost", "uri": "/", "headers": {"User-Agent": ["curl/8.9.1"], "Accept": ["*/*"]}, "tls": {"resumed": false, "version": 772, "cipher_suite": 4865, "proto": "h2", "server_name": "localhost"}}, "bytes_read": 0, "user_id": "", "duration": 0.000005, "size": 0, "status": 0, "resp_headers": {"Server": ["Caddy"], "Alt-Svc": ["h3=\":443\"; ma=2592000"]}}
2024/08/30 00:06:01.403	error	http.log.access.log0	handled request	{"request": {"remote_ip": "::1", "remote_port": "52675", "client_ip": "::1", "proto": "HTTP/2.0", "method": "GET", "host": "localhost", "uri": "/", "headers": {"User-Agent": ["curl/8.9.1"], "Accept": ["*/*"]}, "tls": {"resumed": false, "version": 772, "cipher_suite": 4865, "proto": "h2", "server_name": "localhost"}}, "bytes_read": 0, "user_id": "", "duration": 0.000010875, "size": 14, "status": 403, "resp_headers": {"Server": ["Caddy"], "Alt-Svc": ["h3=\":443\"; ma=2592000"], "Content-Type": ["text/plain; charset=utf-8"]}}

jq can rip the JSON from these lines with a filter:

jq -R '.[index("{"): 1+rindex("}")] | fromjson' caddy.log

From there you could select for 403 responses:

jq 'select(.status==403)'

And print just the client IP:

jq '.request.client_ip'

All together in one:

jq -R '.[index("{"): 1+rindex("}")] | fromjson | select(.status==403) | .request.client_ip' caddy.log

Which prints ::1 when run on the above two lines.

1 Like
5

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.