How to monitor "requests per seconds"

1. The problem I’m having:

How can I monitor the requests per seconds my caddy server have?
In the /metrics I can not see this value.

3. Caddy version:

I use caddy-docker-proxy: v2.8.10

4. How I installed and ran Caddy:

In a docker compose file

a. System environment:

Debian Bookworm

Is really nobody monitoring the requests per second? :grinning:

On Monitoring Caddy with Prometheus metrics — Caddy Documentation, you can find:

For example, to see the per-second request rate, as averaged over 5 minutes:

rate(caddy_http_requests_total{handler="file_server"}[5m])

This metric (TYPE counter) has the following HELP:

# HELP caddy_http_requests_total Counter of HTTP(S) requests made.

In my Caddy instance, it has the label handler="subroute" instead of handler="file_server".

Hope this helps.

2 Likes

Thank you!

I don’t have this field at all: caddy_http_requests_total

This is what I only have. Why is this missing? How did you activate this in your caddy config?

curl localhost:2019/metrics | grep http
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  6719    0  6719    0     0  1400k      0 --:--:-- --:--:-- --:--:-- 1640k
# HELP caddy_admin_http_requests_total Counter of requests made to the Admin API's HTTP endpoints.
# TYPE caddy_admin_http_requests_total counter
caddy_admin_http_requests_total{code="200",handler="load",method="POST",path="/load"} 40
# HELP promhttp_metric_handler_requests_in_flight Current number of scrapes being served.
# TYPE promhttp_metric_handler_requests_in_flight gauge
promhttp_metric_handler_requests_in_flight 1
# HELP promhttp_metric_handler_requests_total Total number of scrapes by HTTP status code.
# TYPE promhttp_metric_handler_requests_total counter
promhttp_metric_handler_requests_total{code="200"} 162539
promhttp_metric_handler_requests_total{code="500"} 0
promhttp_metric_handler_requests_total{code="503"} 0

You need to enable metrics for the http servers. See here:

1 Like

Hi Mohammed90,

I am really lost with the config. I would like to see a very basic example to start from.

The simplest config is nut working for me.

I want to make metrics and admin available outsite of my docker container.

This is what I tried so far with no luck.

:2019 {
	metrics /metrics
}

{
	admin :2019
}

=> curl localhost:2019/metrics
This gives me the above output

=> curl localhost:2019/config/
This is always emptry

How is your config looking?

That’s not right. The metrics enabled by default are the admin metrics. These are what you see, even though you used the metrics handler. You haven’t enabled metrics on the server configuration itself.

Configuring servers is done in the global options. The options section goes at the top of the Caddyfile, not at the bottom. You need to read this page.

A rough configuration may be like this:

{
    admin <ip-address>:2019
    servers {
        metrics
    }
}
example.com {
     # config
}

Do not make your admin endpoint globally accessible. That’s not secure!

1 Like

I tried it like that:

{
    admin 0.0.0.0:2019
    servers {
        metrics
    }
}

But it is still not working. The Server is behind a firewall an 2019 is only internal available.

Do I have to work with wildcard instead of 0.0.0.0 or something else to make it work?

Define “not working”. What’s your full config? What do you see when you call <ip-address>:2019/metrics ?

1 Like

Lets forget for now about the admin.

This is my current CaddyFile:

{
    servers {
        metrics
    }
}

Within my container I can call
curl localhost:2019/metrics

outside of my container I get this:
curl localhost:2019/metrics
curl: (56) Recv failure: Connection reset by peer

(Port 2019 is exposed)

Caddy’s default admin listen address is localhost:2019, which means it listens for requests from 127.0.0.1.

When using Docker, 127.0.0.1 means “this same container”. Connections from the host are like as if it was from a totally different machine.

You’d need to change Caddy to admin :2019 to listen on all interfaces, i.e. accept connections from outside the container.

1 Like
{
    admin :2019
    servers {
        metrics
    }
}

Do you mean like that? And this should work, even with CDP?

Ah, no. CDP manages the admin endpoint itself because it needs to control the config.

You can serve metrics separately on a different port like this:

{
	servers {
		metrics
	}
}

:2020 {
	metrics
}

<the rest of your sites go here>

With CDP you can either put this as labels on your Caddy service itself, or use a base Caddyfile (see CDP README)

1 Like

Thank you so much!
Finally it is working.

curl localhost:2019/metrics | grep http | grep total
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0# HELP caddy_admin_http_requests_total Counter of requests made to the Admin API's HTTP endpoints.
# TYPE caddy_admin_http_requests_total counter
caddy_admin_http_requests_total{code="200",handler="load",method="POST",path="/load"} 1
# HELP caddy_http_requests_total Counter of HTTP(S) requests made.
# TYPE caddy_http_requests_total counter
caddy_http_requests_total{handler="metrics",server="srv0"} 46
caddy_http_requests_total{handler="static_response",server="srv2"} 34
caddy_http_requests_total{handler="subroute",server="srv1"} 1475
caddy_http_requests_total{handler="subroute",server="srv2"} 2
100 86388    0 86388    0     0  19.1M      0 --:--:-- --:--:-- --:--:-- 20.5M
# HELP promhttp_metric_handler_requests_total Total number of scrapes by HTTP status code.
# TYPE promhttp_metric_handler_requests_total counter
promhttp_metric_handler_requests_total{code="200"} 46
promhttp_metric_handler_requests_total{code="500"} 0
promhttp_metric_handler_requests_total{code="503"} 0

I am also able to load this into Prometheus and visualize it in Grafana.

Which Grafana dashboard do you use?

1 Like