Is it possible to have more than 1 log output file for same site?

Help
1

I was reading the log directive at log (Caddyfile directive) — Caddy Documentation and it only mentions the one output file. But I was wondering if Caddy v2 supports more than one output file + format pairing for same site ?

For instance with Nginx I can setup multiple access logs for same site using different log formats i.e. set site to log to default combined format + log to a separate log file using json format.

In Nginx it would be

access_log /path/to/access.log combined buffer=256k flush=5m;
access_log /path/to/access_log.json main_json buffer=256k flush=5m;

then JSON output version would be something like

cat access_log.json | tail -1 | jq -r .
{
  "http_cf_ray": "58d8c8f393bbe049-DFW",
  "gzip_ratio": "4.50",
  "pipe": ".",
  "server_protocol": "HTTP/1.1",
  "request_method": "GET",
  "body_bytes_sent": "23142",
  "status": "200",
  "args": "/threads/ovh-new-infrastructure-line-severs-intel-xeon-e-22xx-xeon-silver-amd-epyc-7371.18392/&",
  "request_uri": "/threads/ovh-new-infrastructure-line-severs-intel-xeon-e-22xx-xeon-silver-amd-epyc-7371.18392/",
  "request": "GET /threads/ovh-new-infrastructure-line-severs-intel-xeon-e-22xx-xeon-silver-amd-epyc-7371.18392/ HTTP/1.1",
  "time_iso8601": "2020-05-03T08:59:16+00:00",
  "time_local": "03/May/2020:08:59:16 +0000",
  "remote_port": "",
  "msec": "1588496356.551",
  "connection": "5908",
  "connection_requests": "1",
  "pid": "18672",
  "request_id": "29e7f4290c9ecf06e780552b25883aba",
  "request_length": "875",
  "remote_addr": "66.249.73.203",
  "remote_user": "",
  "bytes_sent": "23992",
  "http_referer": "",
  "http_user_agent": "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)",
  "http_x_forwarded_for": "66.249.73.203",
  "http_host": "domain.com",
  "server_name": "domain.com",
  "request_time": "0.100",
  "upstream": "127.0.0.1:9000",
  "upstream_connect_time": "0.000",
  "upstream_header_time": "0.100",
  "upstream_response_time": "0.100",
  "upstream_response_length": "104181",
  "upstream_cache_status": "BYPASS",
  "ssl_protocol": "TLSv1.3",
  "ssl_cipher": "TLS_AES_256_GCM_SHA384",
  "scheme": "https"
}
2

I believe that in JSON you can define as many arbitrary logs as you like, but the Caddyfile is a bit less capable.

1 Like
3

Could you clarify with an example of how that would look for per hostname logging ? So domain1.com and domain2.com each having 2 output file equivalent logs for single_field and json formats ? Or is logging via JSON method for the entire server and not per hostname ?

4

Logging is configured at the top level within the Caddy v2 JSON config structure:

That means that logs are configured each totally individually from any and all log generators. You narrow it down to what results you want to see. For example:

All defined logs accept all log entries by default, but you can filter by level and module/logger names. A logger’s name is the same as the module’s name, but a module may append to logger names for more specificity. For example, you can filter logs emitted only by HTTP handlers using the name “http.handlers”, because all HTTP handler module names have that prefix.

—JSON Config Structure - Caddy Documentation

1 Like
5

Hello,

As Matthew says, we can’t have multiple log directive per block (rather logical I guess) but we can easily define every log for each block :

domain1.com {

  root * /var/www/domain1

  log {
    output file access_dom1.log
    format single_field common_log
  }
}

domain2.com {

  root * /var/www/domain2

  log {
    output file access_dom2.log
    format json
  }
}

Inspired from log (Caddyfile directive) — Caddy Documentation

1 Like
6

The documentation needs a specific example to go with the text for better clarification :slight_smile:

So in my example Caddyfile

{
    http_port   81
    https_port  4444
    #experimental_http3
}

http://caddy2.domain.com:81 {

    header x-powered-by "caddy centminmod"
    header X-Content-Type-Options nosniff
    header X-XSS-Protection "1; mode=block"
    encode gzip
    
    # Set this path to your site's directory.
    root * /usr/share/caddy
    
    # Enable the static file server.
    file_server
    
    # Or serve a PHP site through php-fpm:
    # php_fastcgi localhost:9000
    log {
        output file /var/log/caddy/access_http.log {
            roll_size 100mb
            roll_keep 10
            roll_keep_for 720h
        }
        format single_field common_log
        level INFO
    }
}

https://caddy2.domain.com:4444 {

    tls internal
    header x-powered-by "caddy centminmod"
    header X-Content-Type-Options nosniff
    header X-XSS-Protection "1; mode=block"
    encode gzip
    
    # Set this path to your site's directory.
    root * /usr/share/caddy
    
    # Enable the static file server.
    file_server
    
    # Or serve a PHP site through php-fpm:
    # php_fastcgi localhost:9000
    log {
        output file /var/log/caddy/access_https.log {
            roll_size 100mb
            roll_keep 10
            roll_keep_for 720h
        }
        format single_field common_log
        level INFO
    }
}

JSON output would be

curl -s localhost:2019/config/ | jq -r '.logging'
{
  "logs": {
    "default": {
      "exclude": [
        "http.log.access.log0",
        "http.log.access.log1"
      ]
    },
    "log0": {
      "encoder": {
        "field": "common_log",
        "format": "single_field"
      },
      "include": [
        "http.log.access.log0"
      ],
      "level": "INFO",
      "writer": {
        "filename": "/var/log/caddy/access_http.log",
        "output": "file",
        "roll_keep": 10,
        "roll_keep_days": 30,
        "roll_size_mb": 96
      }
    },
    "log1": {
      "encoder": {
        "field": "common_log",
        "format": "single_field"
      },
      "include": [
        "http.log.access.log1"
      ],
      "level": "INFO",
      "writer": {
        "filename": "/var/log/caddy/access_https.log",
        "output": "file",
        "roll_keep": 10,
        "roll_keep_days": 30,
        "roll_size_mb": 96
      }
    }
  }
}

so to add json log for https port 4444 site would be like below ?

    "log1json": {
      "encoder": {
        "field": "common_log",
        "format": "json"
      },
      "include": [
        "http.log.access.log1"
      ],
      "level": "INFO",
      "writer": {
        "filename": "/var/log/caddy/access_https.json",
        "output": "file",
        "roll_keep": 10,
        "roll_keep_days": 30,
        "roll_size_mb": 96
      }
    }
7

"field" is not a valid key adjacent to "format": "json", it only applies to "format": "single_field".

JSON Config Structure - Caddy Documentation
JSON Config Structure - Caddy Documentation

Other than that, yeah, that looks like it makes sense to me. Give it a shot, see if anything explodes.

8

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.