MrAlex
(Alex)
May 4, 2020, 4:21pm
1
1. My Caddy version (caddy version
):
2.0rc3
2. How I run Caddy:
a. System environment:
Ubuntu 20.04 LTS, installed via apt
, enabled by default with systemd
.
b. My complete Caddyfile or JSON config:
FQDN {
# Set this path to your site's directory.
root * /usr/share/caddy
# Enable the static file server.
file_server
log {
format json
output net unix//var/cache/caddy.socket
}
}
3. The problem I’m having:
I’d like to be able to feed the output from the access log in a custom JSON format via syslog to the matomo live log importer , with the expected json
output of:
'{"ip": "$remote_addr",'
'"host": "$host",'
'"path": "$request_uri",'
'"status": "$status",'
'"referrer": "$http_referer",'
'"user_agent": "$http_user_agent",'
'"length": $bytes_sent,'
'"generation_time_milli": $request_time,'
'"date": "$time_iso8601"}';
4. What I already tried:
Output is set to JSON
and sent via net unix socket
to rsyslog; /etc/rsyslog.d/10-matomo.conf
:
$AddUnixListenSocket /var/cache/caddy.socket
$template matomo,"%msg%\n"
if $syslogfacility-text == 'local0' then ^/usr/local/matomo/matomo.sh;matomo
/usr/local/matomo/matomo.sh
:
#!/bin/sh
echo "${@}" | /usr/local/matomo/import_logs.py \
--url=https://localhost/matomo/ --token-auth=<SECRET> \
--enable-http-errors --enable-http-redirects --enable-static --enable-bots \
--idsite=1 --recorders=4 --log-format-name=nginx_json -
Of course, there is no data imported as the JSON format is not as expected. Does anyone know if this is possible? I tried getting my head around mmjsonparse
to parse the incoming JSON and output the JSON as desired, but couldn’t figure out how to.
Cheers!
I think you can use the single_field
format and set the field to your custom json string using request placeholders (or their equivalent Caddyfile shorthands )
MrAlex
(Alex)
May 4, 2020, 6:07pm
3
I’ve tried format single_field {host}{hostport}{method}{path}
just to see what it outputs, but it doesn’t seem to be any different to format json
{
"level": "info",
"ts": 1588615390.2087238,
"logger": "http.log.access.log0",
"msg": "handled request",
"request": {
"method": "GET",
"uri": "/",
"proto": "HTTP/2.0",
"remote_addr": "x.x.x.x:51780",
"host": "x.x.x",
"headers": {
"Accept-Encoding": ["gzip, deflate, br"],
"If-None-Match": ["\"q9dj779fm\""],
"Te": ["trailers"],
"User-Agent": ["Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:68.0) Gecko/20100101 Firefox/68.0"],
"Accept": ["text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"],
"Accept-Language": ["en-US,en-US;q=0.7,en;q=0.3"],
"Dnt": ["1"],
"Upgrade-Insecure-Requests": ["1"],
"If-Modified-Since": ["Sun, 26 Apr 2020 02:35:31 GMT"],
"Cache-Control": ["max-age=0"]
},
"tls": {
"resumed": false,
"version": 772,
"ciphersuite": 4865,
"proto": "h2",
"proto_mutual": true,
"server_name": "x.x.x"
}
},
"common_log": "x.x.x.x - - [04/May/2020:20:03:10 +0200] \"GET / HTTP/2.0\" 304 0",
"latency": 0.004685767,
"size": 0,
"status": 304,
"resp_headers": {
"Server": ["Caddy"],
"Etag": ["\"q9dj779fm\""]
}
}
matt
(Matt Holt)
May 4, 2020, 11:23pm
4
The single_field
encoder just uses an existing field from the log entry as the value for the log entry.
Caddy’s log encoders are modular – if one doesn’t suit your needs, it can be swapped out for another one.
You might find this helpful: Making Caddy logs more readable - #5 by JeanLucLacroix
MrAlex
(Alex)
May 14, 2020, 1:00pm
5
Thanks, that was a great help. Also congratulations on the release!
system
(system)
Closed
June 13, 2020, 1:00pm
6
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.