Combined log in Caddy2 + error log question

Help
#1

Firstly - sorry for flooding the forums with the questions, let’s hope that’s one of the last ones :slight_smile:

Is there a simple way to set log format to combined log in Caddy2? single_field is fine for Common Log (log (Caddyfile directive) — Caddy Documentation), but Combined log adds referrer/user agent in addition to it.

Also, I see the following in documentation:
log Enables access/request logging

That’s an access log. Is there something similar for error log per-vhost? For example - where stderr from PHP would be logged, aborted requests (due to timeouts), denied access entries etc. would be logged.

Thank you!

Where to put error log in Caddyfile?
Fcgiwrap - errors & variables
#2

Caddy 2 uses structured logging, which is how we recommend doing logging. It’s more efficient (zero-allocation, in fact) and provides more options for processing in a pipeline. Most users will not need to use more than the already-default log – maybe with minor customizations – in order to be able to do the processing they want. For example, if you need to split certain logs out to different files, your system can read each incoming log message for a certain field, and then switch which file it gets written to.

With Caddy 2, you’ll need to discard assumptions from old web servers and think about things in a new – usually simpler, more refined and compartmentalized – way.

#3

@matt any suggestions for the error log per-vhost?

Regarding the log format - it’s totally fine for humans, but I guess log analyzers (stats: AWstats, webalizer, analog), firewalls (like CSF), control panels aren’t ready for Caddy format, unless custom hooks are written. This would slow our integration down, as that’d be a lot of work, but it’s doable… Just wanted to be sure there is no easy way to set combined log format in caddyfile.

#4

Yes, like I said, just direct each log message to your preferred file based on the hostname of the request in the log message.

Well, this isn’t “Caddy format” – this is just JSON (but can be any structured log format), and it’s been around for a decade or more… unfortunately, Common Log Format and Combined Log Format aren’t very useful these days except in legacy systems, as you’ve noted. (For instance, CLF doesn’t even include the request’s hostname?? How is that even useful??)

There are numerous tools available which can ingest streaming log messages and direct them to files or where ever else.

#5

Hm… I’m not sure which configuration option is used for that? (to filter stderr’s by domain name and direct them to files defined). We provide an option for end-customers to check their error-logs on Nginx/Apache/OpenLiteSpeed/LiteSpeed, because it’s hard for them to debug error cases otherwise. Or did you have some own-coded log parser in your mind?

#6

Choose any log processing tool/utility and use that – Caddy’s not a log aggregator, it’s a server. So choose any tool that you like best and go wild.

I haven’t written one. But I guess somebody could write one as a Caddy app if they wanted to.

#7

The same argument could be used for:

{
	...
	log {
		output file /var/log/caddy/domains/domain.com.log {
			roll_disabled
		}
	}
}

But there must have been a reason why you’ve made this available :slight_smile: Anyway, it’s clear that it’s unavailable for error logging and implementation isn’t planned. Thanks for letting me know this, I’ll just check/code some workarounds then.

1 Like
#8

Not really – dumping logs to files is pretty minimally expected even by most streaming log tools. But logic that splices and dices logs into different outputs based on the values of certain fields is much more complex and really quite a different problem domain.

#9

Are you saying stderr is logged to access log? Because your documentation clearly states (as mentioned in my very first post):
log Enables access/request logging

#10

I don’t understand the question. Stderr is an output, not an input.

In any case, the quoted documentation is correct.

#11

Okay, let’s start the discussion again :slight_smile:

I have /home/admin/public_html/index.php, which has a syntax error. FastCGI outputs it to stderr. Are you saying this stderr output from FastCGI interface gets logged to the caddy access/request log defined in vhost configuration?

#12

Sorry, I still don’t understand. Are you asking why FastCGI is emitting logs to stderr instead of to Caddy?? I don’t understand what FastCGI’s error logs have to do with Caddy…

logged to the caddy access/request log

The access log is just that: a log of access (i.e. requests)…

#13

Okay. Nginx example:
2017/07/16 10:35:16 [error] 1365#1365: *5 FastCGI sent in stderr: “PHP message: PHP Parse error: syntax error, unexpected ‘$autoloader’ (T_VARIABLE) in /var/www/path/to/project/index.php on line 14” while
reading response header from upstream, client: 192.168.33.1, server: example.dev, request: “GET / HTTP/1.1”,
upstream: “fastcgi://unix:/run/php/php7.0-fpm.sock:”, host: “example.dev”

Would Caddy log this at all?

#14

Ah, I see, you mean from the fastcgi’s stderr relative to a request, not the php-fpm stderr output.

Yes, Caddy gets those errors, and writes them to its error log. But an error log is different from an access or request log.

#15

That’s why I was asking in my initial post if there is any possibility to specify error_log for vhost, because I was unable to find it in documentation :slight_smile:

I’m sure you log way more things to the error log, than just stderr from the fastcgi interface (denied access entries, timeouts etc.).

So, can we specify it per-vhost, or the only solution is to code something that reads main error log and splits it per-vhost? (I wouldn’t like to do this as it’s error-prone, would require parsing/locking etc…)

#16

@matt please also let me know if it’s worth opening github issue for it, if it’s not natively supported now. Thanks.

#17

Like I said, aggregating and processing logs is outside the scope of the HTTP app of Caddy – all it does is emit logs. If you want to process them more specifically later, you can do that because they are structured logs, not the old-school rather useless “common log format”. There are a ton of tools that can process logs, and someday maybe a Caddy app will even be written just for log processing.

#18

@matt I think you didn’t get the point yet :slight_smile:

Apache, Nginx, OpenLiteSpeed, Litespeed all offer error_log functionality per vhost.

I guess Caddy doesn’t (from your answers and confusion). So the question is: are you planning to support it, like all of the webservers mentioned above do?

I’m really not sure why it’s so confusing…

{
	...
	log {
		output file /var/log/caddy/domains/domain.com.log
	}
	error_log {
		output file /var/log/caddy/domains/domain.com.error.log
	}
}

Any plans to support it? If no - it’s clear and we can close the case :slight_smile:

2 Likes
#19

I would also like to be able to have two files, one for access/standard log, and the other one for errors. It was possible and convenient with Caddy 1, so it would be great for me if it would be possible with Caddy 2…

1 Like
Alternative to "errors" directive in v2
#20

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.