Help adding /metrics proxying to working config

[AndyMcCall]

1. Output of caddy version:

2. How I run Caddy:

Caddy is run via docker-compose.

a. System environment:

CentOS, Docker, Docker-Compose

b. Command:

docker-compose up

c. Service/unit/compose file:

caddy:
    container_name: caddy
    image: 'index.docker.io/caddy:2.5.1-alpine@sha256:6e62b63d4d7a4826f9e93c904a0e5b886a8bea2234b6569e300924282a2e8e6c'
    cpus: 4
    mem_limit: '4g'
    environment:
      - 'XDG_DATA_HOME=/caddy-storage/data'
      - 'XDG_CONFIG_HOME=/caddy-storage/config'
      - 'SRC_FRONTEND_ADDRESSES=service-frontend-0:3080'
      - 'SRC_SITE_ADDRESS=192.168.1.10'
      - 'SRC_ACME_EMAIL=andy.mccall@myemail.com'
    volumes:
      - 'caddy:/caddy-storage'
      - '/opt/service/caddy/builtins/https.custom-cert.Caddyfile:/etc/caddy/Caddyfile'
      - '/opt/service/certs.d/192.168.1.10.cert:/service.pem'
      - '/opt/service/certs.d/192.168.1.10.key:/service.key'
    ports:
      - '0.0.0.0:80:80'
      - '0.0.0.0:443:443'
    networks:
      - service-network
    restart: always

d. My complete Caddy config:

:443
@http {
                protocol http
}
redir @http https://{host}{uri}
tls /service.pem /service.key
reverse_proxy {
                to 192.168.1.10
                trusted_proxies 0.0.0.0/0
}

3. The problem I’m having:

This config works find for serving the service out via https terminating the SSL, however, I need to also add some proxying for a path, that being the /metrics path. This should proxy to service2 that is served on port 9100. I’m new to Caddy, and the config that works was created by a vendor, but they won’t support me adding info to their config (I understand why), but it’s still something I need to do.

4. Error messages and/or full log output:

When I use my (poorly) created config file, the caddy service continually restarts in docker.

5. What I already tried:

I’ve tried adding the config in for the /metrics proxying like this:

:443
@http {
                protocol http
}
redir @http https://{host}{uri}
tls /service.pem /service.key
reverse_proxy {
                to 192.168.1.10
                trusted_proxies 0.0.0.0/0
                /metrics 192.168.1.10:9100
}

6. Links to relevant resources:

[IndeedNotJames (Emily Lange)]

@AndyMcCall

and the config that works was created by a vendor, but they won’t support me adding info to their config

Their provided Caddyfile is terrible.

@http {
	protocol http
}
redir @http https://{host}{uri}

This redirect makes no sense, as you won’t ever be able to match protocol http on a https:// (:443) site block/vhost.
Furthermore, that redirect already happens as part of Automatic HTTPS — Caddy Documentation

reverse_proxy {
	to 192.168.1.10
	trusted_proxies 0.0.0.0/0
}

trusted_proxies 0.0.0.0/0 is a very bad idea, as you allow literally any IPv4 to fake its IP.
See docs/caddyfile/directives/reverse_proxy#defaults for a bit more details on that.

It’s also a bit odd to write 192.168.1.10 instead of 192.168.1.10:80, but eh, whatever.

---

As for your docker-compose file, I really like that the image it pinned by its digest.
But please update it to v2.6.2 by using caddy:2.6.2-alpine instead, or if you want to pin it again, use

caddy:2.6.2-alpine@sha256:25a0097607868fb05a89a5ab9fea2f2ea4cecdc89d887d7dcee8c778a21b9e1f

And because v2.6.0 enabled http/3 by default, add
- '0.0.0.0:443:443/udp' to your ports:

Any reason why you have a bunch of env variables that you aren’t using, like SRC_FRONTEND_ADDRESSES?

---

Last but not least, a proper Caddyfile which also passes /metrics to 192.168.1.10:9100 would look something like:

:443 {
	tls /service.pem /service.key

	reverse_proxy 192.168.1.10:80 {
		trusted_proxies 0.0.0.0/0
	}
	reverse_proxy /metrics 192.168.1.10:9100
}

But consider dropping the trusted_proxies 0.0.0.0/0, as it’s fairly insecure, unless you know what you are doing.

I highly recommend going through the various tutorials at The Caddyfile — Caddy Documentation

[AndyMcCall]

Thank you for your help @IndeedNotJames - I’ve been able to get it working now. You’re a superstar!

I’ve upgraded the image from the supplier pinned one to the one suggested, removed the match that won’t be instigated and got the /metrics path to work.

I haven’t fully tested, but I think the trusted_proxies is in there because this is a multi-container docker compose solution that uses caddy to make inter-services calls.

The variables are there as the config file is a template that’s populated via Terraform variables. I forgot to swap them out when making my post.

I’ve passed the updates back to supplier too.

[AndyMcCall]

@IndeedNotJames is it possible to add a redirect into this too?

I tried this:

# this handles mynewdomain.com
:443 {
	tls /service.pem /service.key

	reverse_proxy 192.168.1.10:80 {
		trusted_proxies 0.0.0.0/0
	}
	reverse_proxy /metrics 192.168.1.10:9100 
}

myolddomain.com {
      redir https://mynewdomain.com
}

But this didn’t work, I think because it’s getting caught by the :443 block, but if I put the redir in the :443 block I’d create a loop.

[IndeedNotJames (Emily Lange)]

Can you elaborate on what you mean by

@AndyMcCall

But this didn’t work, […]

and maybe provide logs and the output of curl --verbose --location myolddomain.com?

@AndyMcCall

I think because it’s getting caught by the :443 block, but if I put the redir in the :443 block I’d create a loop.

No, it shouldn’t get caught by the :443 block.
If you absolutely want to put it in the :443 block, though, you would have to use named matchers.

[AndyMcCall]

Doh!

It was working. Caddy is sat behind an AWS load balancer. As I was restarting Caddy the load balancer was detecting the reload and serving a 503 error. If I gave it a few more seconds then I’d have realised it was working fine. The only reason why I was testing so quickly was because I didn’t want the load balancer to detect the outage and recycle the ASG.

It’s all working with the config I posted