Log file buffer / async?

Help
1

Hello,

I don’t find any information on the documentation. Is the log feature (to log each request) do a synchronous write (each request = one write on the file) or an asynchronous write?

If asynchronous, how I can control the buffer size or the delay before writing to hdd?

Thanks

2

Caddy uses Go’s log package, which is a synchronous logger.

3

In our production environment, law force us to keep at least 1 year of access log. even with log rotation. log could became a bottleneck in high-traffic website.

Any plan to get it asynchronous ?

4

No problem, just use:

log important-access.log {
	rotate_size 100 # can be any value you want
	rotate_age  365
	rotate_keep 0
}

Setting rotate_keep to 0 should cause an indefinite number of log files to be kept, and then they will be deleted after 1 year with rotate_age.

AFAIK we have no plans to implement async logging, the standard lib doesn’t either: proposal: log: Asynchronous logging · Issue #15065 · golang/go · GitHub (we’d probably have to switch logging libraries to do it)

5

Async logging seems like a really good use-case for a plugin. Is log information available to plugins @matt?

6

Anything has access to the process log, but other than that, logs aren’t currently shared.

7

Thanks matt.

Is it possible to rotate by date or to use date variable on the file name ? I mean, 1 file = 1 day ?

8

@zero-x-baadf00d No, because that doesn’t do anything to help conserve disk space, which is the point of log rotation. But by default, each entry in the log has the timestamp, so you can still get date information from that inline with the data itself.

9

To save space on the caddy server, we ship all the logs to a central logging facility (in our case, ElasticSearch). Having a year of logs dotted about on various servers sounds like a mess to interrogate.

10

Would it be helpful if we added something like rotate_off or rotate off to disable log rotation?

11

The root of the question is not about log ration though is it?

I think the point is that if the logging is synchronous, then on a high-volume site the logging could become a bottleneck.

I’m not saying that’s particularly valid or not, and sequential writes to a file can be stupid-fast, but that’s how I read the OP’s question.

12

You’re right, maybe I misinterpreted your earlier comment.

Async logging is on my long-term TODO list (as is improved logging in general) for what it’s worth.

13

In our high performance/traffic situation as gateway-proxy, caddy log make top tmpString. I think reduce gc/malloc and async flush maybe better for this high performance/traffic :slight_smile:

14

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.