Per-host Prometheus metrics

I just upgraded to Caddy 2 and so far I am quite happy. However, the only thing that I’m really missing compared to v1 is the ability to have dedicated metrics per virtual host. Previously, there was the third-party prometheus plugin. Now, Caddy has its own metrics directive, but to my understanding, it only exposes “global” statistics. I’d like to see, for instance, total number of requests for each of my configured virtual hosts / domains / endpoints without having to parse the access logs. Is there a way to achieve this?

1 Like

/cc @hairyhenderson

Hi @n1try,

One issue with the v1 metrics plugin’s per-host metrics was that when the hostname is determined based on the Host header, it’s an unlimited cardinality value. This was the default behaviour, and I have direct experience of this causing massive monitoring problems :wink:. Prometheus metrics are best with low cardinality, so I decided for the initial metrics support to not support this. The user-defined host label was fine, though.

There is currently a server label, which is set to the name of the server (srv0 by default from a Caddyfile). If you have multiple servers configured you can differentiate that way (i.e. if you have multiple blocks listening to different ports).

I’m open to an issue being logged in GitHub - caddyserver/caddy: Fast, multi-platform web server with automatic HTTPS requesting a host label so we can discuss this further. Right now the metrics support is not very configurable, so this would also require some thought on how we configure it.

An important point too is whether we want to allow using the Host header to set the label (which has unlimited cardinality), or whether we want to use something user-defined only. This is how the v1 plugin worked.

2 Likes

Thanks a lot for your clarifications! I agree that determining the label according to the Host header is problematic. However, something like a “tagging” concept might be introduced in a way that the metrics directives allows for a host (or more generic label / tag`) to be set. I’ll open an issue on GH for further discussion.

1 Like

For anyone finding this thread online, there’s a Github issue for this here: Support per-host Prometheus metrics · Issue #3784 · caddyserver/caddy · GitHub

Maybe I’m misunderstanding, but is this line implying we can do this with user-defined labels? If so, how do we define them manually?

1 Like

Sorry, I see that I was unclear. What I meant was that in the v1 plugin we were able to user-define labels for this. We still need to add support for this in the v2 core metrics support.

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.