Module to customize your JSON logs

I’ve created a log encoder module for Caddy.

You can find it at github.com/leodido/caddy-jsonselect-encoder.

It lets the user choose what info to emit in the JSON logs (it wraps the default json log encoder).

How? Specifying a selector, like so:

log {
  format jsonselect "{level} {ts} {request>host} {duration}"
}

Also, it lets the user change the resulting JSON structure (for example by changing the keys).

I was inspired by the format-encoder (thank you!) which is somewhat very similar to this.

Anyway, there are some differences in the internals of the plugin:

  • the parsing of the selector happens at provisioning time, not at encoding time
  • you can specify to which key (even nested ones) you want to set the value your extracting from the standard JSON entries

For example, you can select what to emit and define the keys to match the Stackdriver log entry format.

log {
  format jsonselect "{severity:level} {timestamp:ts} {httpRequest>requestMethod:request>method} {httpRequest>protocol:request>proto} {httpRequest>status:status} {httpRequest>responseSize:size} {httpRequest>userAgent:request>headers>User-Agent>[0]}" {
    time_format "rfc3339_nano"
    level_format "upper"
  }
}

Which outputs:

{"severity":"INFO","timestamp":"2021-07-19T14:48:56.262966Z","httpRequest":{"protocol":"HTTP/2.0","requestMethod":"GET","responseSize":17604,"status":200,"userAgent":"Mozilla/5.0 ..."}}

I’ve built it for the company (Gitpod) I’m currently working but I believe it can be useful to other Caddy lovers :slight_smile:

Feel free to ask me questions, review it, give me advice on how to further improve it.

Thank you!

2 Likes

This looks very similar to GitHub - caddyserver/format-encoder: Log encoder module for custom log formats

Indeed, as said I was inspired by the amazing format-encoder :blush:

I just needed to move the parsing of the selectors to the provisioning phase, and a mechanism to reshape/remap all the resulting JSON keys.

I was not sure whether sending a PR towards format-encoder with such changes would have been accepted, thus I went down the path of writing my similar plugin.

In case you want, I can definitely send a PR towards format-encoder! I’d love to!

Bests,
Leo

1 Like

My bad, I totally missed that line in your post

:+1:

I think you could consider it. Looping in @Mohammed90 who put some work into it

1 Like

Please feel free to shoot the PR! It doesn’t seem like a breaking change, and I like the optimizations of where it’s heading. :slight_smile:

2 Likes

This topic was automatically closed after 30 days. New replies are no longer allowed.