Analyzing Caddy's log files real time with GoAccess?

Just to clarify and be as unambiguous as possible:

GoAccess can not ingest your Caddy process log.

The Caddy process log is specified by the -log flag when you run Caddy.

You need to ensure you are not outputting the process log to the same file you are ingesting with GoAccess.

You need to ensure that the file you ingest with GoAccess contains only access logs, which are specified with the log directive in your Caddyfile.

In the Caddy.sh launch script I have

caddy (...) -log /share/Public/caddy/caddy.log

and in the goaccess.conf I have

log-file /share/Public/caddy/caddy.log

As far as I read here[1], it seems that log-file serves to Specify the path to the input log file.

Am I missing something even more obvious than I thought? (it’s well possible) :slight_smile:

[1] GoAccess - Manual Page

-log /share/Public/caddy/caddy.log is the incorrect type of log. It is a process log, not an access log.

GoAccess can not analyse your process log.

Instead of using -log /share/Public/caddy/caddy.log, use the log directive in your Caddyfile:

https://caddyserver.com/v1/docs/log

For example:

example.com {
  log /path/to/access.log
}

With a line in your goaccess.conf:

log-file /path/to/access.log
2 Likes

Ahhhhhh! :sweat_smile:

This is an excellent point!

Should I have a different *.log file for each site (and therefore have multiple log-file /path/to/access.log) or can I make each site pointing to the same *.log file?

Thanks a lot!

Both options will work.

Either way, GoAccess will analyse all the log lines that you’re putting in all the specified log files.

1 Like

Thank you Matthew.

I have now configured the file, but I cannot access to the index.html file from Caddy as I do not know how can I point to the famous index.html.

I do not know if is counts, but currently Caddy is set to use the following -root /home/Qhttpd

The caddy.conf (for accessing GoAccess) is the following:

sub.domain.com {
       import gzipconf
       import addheader
       log /share/Public/GoAccessLogs/goaccess-access.log
       proxy / http://192.168.10.37:7890 {
                keepalive 32
                transparent
                websocket
        }
}

The goaccess.conf is the following (in this example I want to see stats on the access to GoAccess done via Caddy):

port 7890
daemonize true
log-format COMMON
log-file /share/Public/GoAccessLogs/goaccess-access.log
output index.html
real-time-html true

Under /home/Qhttpd/Web/goaccess/ I can see index.html (I only created the goaccess folder and then goaccess created the index.html file alone).

However I have problems to access the file index.html (it has permission 666), whatever I type:

https://sub.domain.com
https://sub.domain.com/goaccess
https://sub.domain.com/goaccess/index.html

Once again I am surely missing something really simple…

Thanks!

Ok, I solved it: it seems that if you use the HTML layout you have to use an existing webserver. In my case I have one on port 82, so just I configured Caddy to that port and it works:

https://sub.domain.com/goaccess/index.html
http://192.168.10.37:82/goaccess/index.html

I am just not sure what is then the purpose of port 7890…

It is not a Caddy problem, but probably someone here knows why the index.html file is static and not live updating. Thanks!

The index.html file is generated once at GoAccess startup and stays the same after that point. For live updating, the site connects you to a websocket which supplies the new information. If the websocket isn’t enabled or can’t connect, you’ll only ever get first-run data.

Thanks Matthew,

How can I then connect it, or check if it is?

Your browser’s developer console - network tab - should give you some insight as to the status of the websocket connection.

I checked and I receive the following error on the Google Chrome Dev Tool:

WebSocket connection to 'wss://sub.domain.com:7890/' failed: Establishing a tunnel via proxy server failed.
setWebSocket @ (index):28
initialize @ (index):28
window.onload @ (index):28

The problem could be originated by the fact that Caddy is set to look at http://192.168.10.37:82/goaccess/index.html (which is the port where my Apache Server is running, i.e. the sole location that I managed to make it accessible for reading the index.html file).

How can I solve it? Thanks!

Well, you need:

  • An externally-accessible reverse proxy to the GoAccess websocket
  • To have GoAccess point your browser at that externally-accessible URI

Caddy can handle the former quite easily (looks like you’re already doing this in your latest Caddyfile). The latter you’ll need to look into GoAccess’ own configuration options.

In practice you are saying to point the reverse proxy to port 7890 and to just look at the goaccess config file to find a directory which is accessible by the websocket to put the index.html, right?

Not 100% sure I understand what you mean by that, sorry.

The index file and websocket are handled very separately. One is a file served from the disk, the other is an upstream server you can reverse proxy to.

You need to configure GoAccess to announce the correct URI through which an external client can reach the websocket. Currently it’s announcing port 7890, which isn’t working.

From what I got here[1], it seems that the problem is that Caddy is requesting a wss and not a ws. Indeed I see that if I try to use the LAN IP to connect to goaccess, I have the “green dot” on it (meaning that the index.http is successful at establishing the websocks connection on port 7890).

Is there then any specific settings to use on Caddy?

Thanks!

[1] [Help] Realtime HTML not working · Issue #1577 · allinurl/goaccess · GitHub

This is probably a result of GoAccess being ambiguous on scheme and your browser is deciding to use WSS when it’s connected to the site over HTTPS. Connecting over HTTP will likewise result in a WS connection, which works over LAN.

Port 7890 is also probably not firewalled inside your internal network, but is it available externally? If you want, you can configure Caddy to listen on this port externally and proxy it to GoAccess internally, that’s one way about it.

There’s no magic setting to make this one just work. You need to decide where you want your index file to be publicly accessible, where you want the websocket to be publicly accessible, and you need to configure GoAccess to point a client (the browser) at the correct URI for the websocket. It’s entirely up to you as to how to organise things and whether, and where, you want Caddy to sit in between the client and the application.

Thank you for the detailed answer Matthew.

Does this mean that no one here is using GoAccees through Caddy over HTTPS?

As the main purpose of putting something behind a reverse proxy is to do not open ports externally, I will not open port 7890, but I would probably just use HTTP.

However, would this current limitation be solved (with version 1.x or with the upcoming version 2.x of Caddy)?

Thanks

By the way, is there a way to automatically redirect HTTPS requests to HTTP (not viceversa)?

No :wink: