Getting logs while using the load balancer?

[Ed_McQuinn]

1. The problem I’m having:

I am trying to get detailed logs about my load balancing setup. We are running multiple versions of the same models on differing ports. I basically just want simple http request statistics for each route and port. I understand there may be a limitation with caddy based on the github issues, but I was wondering if anyone could point me in a direction to log this stuff to grafana? :

{
        acme_dns google_domains placeholder
        servers {
                metrics
        }
}

placeholder.placeholder.com {
        route /v2/models/Nous-Hermes-Llama2-13b/* {
                reverse_proxy {
                        to http://localhost:8010 http://localhost:8020
                        lb_policy round_robin

    }
        }
        route /v2/models/Yi-34B/* {
                reverse_proxy {
                        to http://localhost:8030
                        lb_policy round_robin
                }
        }
        route /v2/models/neural-chat-7b-v3-1/* {
                reverse_proxy {
                        to http://localhost:8040
                        lb_policy round_robin
                }
        }
        route /v2/models/WizardCoder-15B-V1.0/* {
                reverse_proxy {
                        to http://localhost:8050
                        lb_policy round_robin
                }
        }
        route /v2/models/zephyr-7b-beta/* {
                reverse_proxy {
                        to http://localhost:8060
                        lb_policy round_robin
                }
        }
}  

2. Error messages and/or full log output:

 I do not have any errors

3. Caddy version:

4. How I installed and ran Caddy:

a. System environment:

Ubuntu

[francislavoie]

You can use this admin endpoint to get health info and current number of active requests:

API - Caddy Documentation

Our opentelemetry/prometheus metrics are unfortunately limited in scope because we don’t have anyone on the core team which uses or understands that functionality.