How to configure Caddy v2 to use a different port based on subpath

[SemoTech]

1. The problem I’m having:

I am attempting to configure Caddy v2 in the following scenario and need assistance:

I am deploying a docker container using Portainer for a web tool (teslamate) that has two components, each accessible using a different port. The stack name will be “teslamate” and it will be accessed as https://teslamate.example.com which needs port 4000 as well as https://teslamate.example.com/grafana which needs port 3000
Caddy to handle SSL from Let’s Encrypt…

Not sure what the proper configuration should look like, I made a config but need some input…

2. Error messages and/or full log output:

When I access https://tm.example.com from the LAN nothing else is rendered and Caddy logs show this error:

ERR | ts=1718561624.8732326 logger=http.log.error msg=tls: first record does not look like a TLS handshake request={"remote_ip":"192.168.1.199","remote_port":"65252","client_ip":"192.168.1.199","proto":"HTTP/2.0","method":"GET","host":"tm.example.com","uri":"/","headers":{"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8"],"Sec-Fetch-Mode":["navigate"],"Cookie":["REDACTED"],"Authorization":["REDACTED"],"Accept-Language":["en-US,en;q=0.8"],"Sec-Fetch-User":["?1"],"Accept-Encoding":["gzip, deflate, br, zstd"],"Priority":["u=0, i"],"Cache-Control":["max-age=0"],"Sec-Ch-Ua":["\"Not/A)Brand\";v=\"8\", \"Chromium\";v=\"126\", \"Brave\";v=\"126\""],"Sec-Ch-Ua-Mobile":["?0"],"User-Agent":["Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36"],"Sec-Fetch-Site":["none"],"Sec-Fetch-Dest":["document"],"Sec-Ch-Ua-Platform":["\"macOS\""],"Upgrade-Insecure-Requests":["1"],"Sec-Gpc":["1"]},"tls":{"resumed":true,"version":772,"cipher_suite":4865,"proto":"h2","server_name":"tm.example.com"}} duration=0.000985858 status=502 err_id=bgskc9nkg err_trace=reverseproxy.statusError (reverseproxy.go:1269) 

3. Caddy version:

caddy v2.8.4

4. How I installed and ran Caddy:

a. System environment:

Ubuntu Server
Docker
Docker compose v2
Portainer

b. Command:

n/a

c. Service/unit/compose file:

n/a

d. My complete Caddy config:

tm.example.com {
  handle_path /grafana* {
        encode gzip
              basic_auth / {
                     gf-admin $23ihnqbeocbqebfihbadfvnbsrgbsfb
           }
    reverse_proxy {
     to https://teslamate:3000
     transport http {
     tls
     tls_insecure_skip_verify
     }
  }
  }

  handle {
        encode gzip
              basic_auth / {
                      tm-admin $vweonqvbqvuabworhuqfvjanefjn
              }
    reverse_proxy {
     to https://teslamate:4000
     transport http {
     tls
     tls_insecure_skip_verify
     }
  }
  }
}

5. Links to relevant resources:

Separate case on same issue opened on Teslamate forum as well:

Is it possible to use a single FQDN for both Teslamate and Grafana? ·...

[francislavoie]

Please run caddy fmt -w on your config to clean up the indentation. It’s really difficult to follow as-is.

@SemoTech

basic_auth / {

Keep in mind that this matches only / exactly, so it would only apply auth to requests to the root of your site and no other paths, which kinda defeats the purpose.

@SemoTech

    reverse_proxy {
     to https://teslamate:3000

Are you sure your app is serving HTTPS? It’s simpler and more performant to proxy over HTTP instead.

There’s no benefit to proxying over HTTPS when the upstream app is in the same network, because the part of the connection that needs to be encrypted is the part going over public networks, not the part within your private network. You’re also throwing all the actual security that HTTPS would provide by setting tls_insecure_skip_verify.

Regarding subpath routing, see this article:

The "subfolder problem", OR, "why can't I reverse proxy my app into a subfolder?"

Have you ever tried to reverse proxy an app into its own little subfolder of your domain name? Makes things neat, doesn’t it? Using example.com/foo/ for one app, example.com/bar/ for another. If you’re coming here from one of the selfhosted communities, you might be thinking along the lines of example.com/sonarr/, example.com/radarr/ etc. Chances are, you’ve tried some configuration along these lines: example.com { redir /sonarr /sonarr/ handle_path /sonarr/* { reverse_proxy localhost…

You’d be better off using a subdomain per app, unless each app has somekind of “base path” configuration that it uses to build paths in its HTML responses.

[SemoTech]

Thanks for the help @francislavoie apprecaited!

OK, I re-formatted the Caddyfile inside the Caddy container w/ “docker exec 9ffb213c2624 caddy fmt --override /etc/caddy/Caddyfile” and also changed the HTTPS references to HTTP while removing the transport and tls entries.

Latest Caddyfile is as follows:

tm.example.com {
        handle_path /grafana* {
                encode gzip
                basic_auth / {
                        gf-admin $23ihnqbeocbqebfihbadfvnbsrgbsfb
                }
                reverse_proxy {
                        to http://teslamate:3000
                }
        }

        handle {
                encode gzip
                basic_auth / {
                         tm-admin $vweonqvbqvuabworhuqfvjanefjn
                }
                reverse_proxy {
                        to http://teslamate:4000
                }
        }
}

While https://tm.example.com now works (thanks again), I get an “HTTP ERROR 502” and Caddy shows this in the log when attempting to access https://tm.example.com/grafana from the LAN:

ERR | ts=1718564414.3428218 logger=http.log.error msg=dial tcp 172.22.0.4:3000: connect: connection refused request={"remote_ip":"192.168.1.199","remote_port":"61136","client_ip":"192.168.1.199","proto":"HTTP/2.0","method":"GET","host":"tm.example.com","uri":"/grafana","headers":{"Sec-Gpc":["1"],"Priority":["u=0, i"],"Sec-Ch-Ua":["\"Not/A)Brand\";v=\"8\", \"Chromium\";v=\"126\", \"Brave\";v=\"126\""],"Sec-Fetch-Site":["none"],"Sec-Fetch-User":["?1"],"Upgrade-Insecure-Requests":["1"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8"],"Sec-Fetch-Mode":["navigate"],"Sec-Fetch-Dest":["document"],"Accept-Encoding":["gzip, deflate, br, zstd"],"User-Agent":["Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36"],"Sec-Ch-Ua-Mobile":["?0"],"Sec-Ch-Ua-Platform":["\"macOS\""],"Accept-Language":["en-US,en;q=0.8"],"Cookie":["REDACTED"],"Authorization":["REDACTED"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","server_name":"tm.example.com"}} duration=0.00089654 status=502 err_id=yqt2j3i9a err_trace=reverseproxy.statusError (reverseproxy.go:1269) 

[francislavoie]

@SemoTech

dial tcp 172.22.0.4:3000: connect: connection refused

Are you sure that container has something listening on port 3000? Caddy isn’t able to connect to anything at that port.

@SemoTech

basic_auth / {

Reminder again to remove the / here, it makes it only apply to exactly / and nothing else, which is not what you want probably.

[SemoTech]

Hi @francislavoie,

Is this updated Caddyfile correct to resolve the basic_auth path issue for /grafana while still using the other login for /?

tm.example.com {
        handle_path /grafana* {
                encode gzip
                basic_auth /grafana* {
                        gf-admin $23ihnqbeocbqebfihbadfvnbsrgbsfb
                }
                reverse_proxy {
                        to http://teslamate:3000
                }
        }

        handle {
                encode gzip
                basic_auth / {
                         tm-admin $vweonqvbqvuabworhuqfvjanefjn
                }
                reverse_proxy {
                        to http://teslamate:4000
                }
        }
}

And yes port 3000 is in-use and listened on by Grafana. Here is an excerpt of the Grafana log:

logger=provisioning.dashboard t=2024-06-16T18:13:03.803316105Z level=info msg="starting to provision dashboards"

logger=http.server t=2024-06-16T18:13:03.803886084Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=http subUrl=/grafana socket=

logger=grafanaStorageLogger t=2024-06-16T18:13:03.805094719Z level=info msg="Storage starting"

logger=provisioning.dashboard t=2024-06-16T18:13:03.844342334Z level=info msg="finished to provision dashboards"

in docker-compose here is the config for grafana and it uses both the teslamate network “tm-net:” and “caddy-net:”. It is also setup to be a subpath of tm.example.com as tm.example.com/grafana

  grafana:
    image: teslamate/grafana:latest
    restart: always
    environment:
      - DATABASE_USER=${TM_DB_USER}
      - DATABASE_PASS=${TM_DB_PASS}
      - DATABASE_NAME=${TM_DB_NAME}
      - DATABASE_HOST=database
      - GRAFANA_PASSWD=${GRAFANA_PW}
      - GF_SECURITY_ADMIN_USER=${GRAFANA_USER}
      - GF_SECURITY_ADMIN_PASSWORD=${GRAFANA_PW}
      - GF_AUTH_BASIC_ENABLED=true
      - GF_AUTH_ANONYMOUS_ENABLED=false
      - GF_SERVER_DOMAIN=${FQDN_TM}
      - GF_SERVER_ROOT_URL=%(protocol)s://%(domain)s/grafana
      - GF_SERVER_SERVE_FROM_SUB_PATH=true
      - GF_SMTP_ENABLED=true
      - GF_SMTP_FROM_NAME=${GF_SMTPNAME}
      - GF_SMTP_FROM_ADDRESS=${GF_SMTPSENDER}
      - GF_SMTP_HOST=${GF_SMTPHOST}                      
      - GF_SMTP_USER=${GF_SMTPUSER}                   
      - GF_SMTP_PASSWORD=${GF_SMTPPASS}
      - GF_SMTP_SKIP_VERIFY=false
    user: "root:root"
    networks:
      tm-net:
      caddy-net:     
    ports:
      - 3000:3000
    labels:
      - "com.centurylinklabs.watchtower.enable=true"
    volumes:
      - /root/docker/teslamate/teslamate-grafana-data:/var/lib/grafana

Lastly, here is the output of the command showing ports in-use in docker:
docker container ls --format “table {{.ID}}\t{{.Names}}\t{{.Ports}}” -a

CONTAINER ID   NAMES                               PORTS
32f8bx3e0f64   teslamate-grafana-1                 0.0.0.0:3000->3000/tcp, :::3000->3000/tcp
64b3747a09bd   teslamate-teslamate-1               0.0.0.0:4000->4000/tcp, :::4000->4000/tcp
bbc8df816c6c   teslamate-database-1                5432/tcp
06ed94df2401   teslamate-mosquitto-1               1883/tcp

Any other suggestions?
Thanks.

[francislavoie]

@SemoTech

Is this updated Caddyfile correct to resolve the path issue?

No, just remove the path matcher from basic_auth entirely, you’re already within a handle which matched the path so you don’t need it, it’s redundant.

@SemoTech

    ports:
      - 3000:3000

You only need this if you need something outside of Docker to access that container. If you’re proxying through Caddy, then you don’t need to publish the port.

@SemoTech

logger=http.server t=2024-06-16T18:13:03.803886084Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=http subUrl=/grafana socket=

Looks like Grafana is configured to serve the path /grafana, so you could possibly use handle instead of handle_path (the latter strips the path prefix before proxying).

But still, the problem is somekind of networking problem between Caddy and Grafana. Are you sure teslamate is the right name to use to connect? Try grafana (the Docker service name) instead.

Try reverse_proxy grafana:3000

[SemoTech]

Hi @francislavoie,

I see on the path matcher from basic_auth, and correct, Grafana is setup to serve the path /grafana. I switched from “handle_path” to “handle” and updated the Caddyfile.

I now managed to get a modal password prompt and only the “gf-admin” credentials worked, so thats good, but then I got a blank page and what looks like same error in Caddy logs about “dial tcp 172.22.0.4:3000: connect: connection refused”

Here is the latest updated Caddyfile:

tm.example.com {
        handle /grafana* {
                encode gzip
                basic_auth  {
                        gf-admin $23ihnqbeocbqebfihbadfvnbsrgbsfb
                }
                reverse_proxy {
                        to http://teslamate:3000
                }
        }

        handle {
                encode gzip
                basic_auth / {
                         tm-admin $vweonqvbqvuabworhuqfvjanefjn
                }
                reverse_proxy {
                        to http://teslamate:4000
                }
        }
}

Interestingly, here is a screenshot of the containers from the “teslamate” Portainer Stack. If you look closely the 172.22.0.4 IP that Caddy is hitting when using /grafana is for the “teslamate” container, not the “grafana” one, so of course port 3000 is not open there, it is open only on the “grafana” container on IP 172.22.0.5.

Now, are you saying that I can/should remove the “ports: - 3000:3000” entry from the grafana configuraion in docker-compose.yml entirely, or should I replace with “ports: - 3000:433” ? What about the “ports: - 4000:4000” entry from the “teslamate” part of the docker-compose configuration?

Thank you.

[francislavoie]

@SemoTech

Now, are you saying that I can/should remove the “ports: - 3000:3000” entry from the grafana configuraion in docker-compose.yml entirely, or should I replace with “ports: - 3000:433” ? What about the “ports: - 4000:4000” entry from the “teslamate” part of the docker-compose configuration?

You don’t need ports: at all for any container that you’re only going to access through Caddy. You only need it when you’re exposing a port to the “outside world” (outside of the Docker stack). Caddy will have ports: for 80/443 because those are the HTTP and HTTPS ports respectively and you’ll have connections from the “outside world” coming into Caddy, but you don’t need to publish ports for apps that you’ll only access through Caddy.

And yeah like I said, change teslamate in your reverse_proxy to grafana instead, to point Caddy to connect to the correct container.

[SemoTech]

I think we got it @francislavoie, thank you so much for all the help!

Your fix to replace the reverse_proxy name from “teslamate” to grafana" for the /garfana* handle worked!

Here is the newest version of Caddyfile:

tm.example.com {
        handle /grafana* {
                encode gzip
                basic_auth  {
                        gf-admin $23ihnqbeocbqebfihbadfvnbsrgbsfb
                }
                reverse_proxy {
                        to http://grafana:3000
                }
        }

        handle {
                encode gzip
                basic_auth / {
                         tm-admin $vweonqvbqvuabworhuqfvjanefjn
                }
                reverse_proxy {
                        to http://teslamate:4000
                }
        }
}

One OCD issue I see is if I access https://tm.example.com/grafana I get the same modal login as https://tm.example.com which says:

Sign in
https://tm.example.com
Username:
Password:

any way to get this instead if accessing /grafana:

Sign in
https://tm.example.com/grafana
Username:
Password:

Can that maybe be accomplished with a change in the Caddyfile / configuration?

Lastly, does the Caddy authentication have to be modal (popup) or is there a way for it to be a simple HTML page?

My thanks again!

[francislavoie]

@SemoTech

Can that maybe be accomplished with a change in the Caddyfile / configuration?

You should configure the realm. See the docs:

basic_auth (Caddyfile directive) - Caddy Documentation

Also, you still have basic_auth / on your 2nd handle. Remove the slash.

You can also simplify your reverse_proxy like this:

reverse_proxy grafana:3000

You don’t need the long-form with to, and you don’t need http://, it’s implied.

[SemoTech]

Thank you very much @francislavoie for all the guidance! Learning more and more about Caddy syntax and options with your guidance.

My Caddyfile is getting very optimized, the latest version is:

tm.example.com {
        handle /grafana* {
                encode gzip
                basic_auth {
                        gf-admin $23ihnqbeocbqebfihbadfvnbsrgbsfb
                }
                reverse_proxy grafana:3000
        }

        handle {
                encode gzip
                basic_auth {
                         tm-admin $vweonqvbqvuabworhuqfvjanefjn
                }
                reverse_proxy teslamate:4000
        }
}

Now I looked at the basic_auth link you sent for the realm, but I cannot see how to configure Caddy to show the /grafana path, in the modal login box, or how to not use modal but just an HTTPS login page. Could you be so kind as to give an example?

[francislavoie]

@SemoTech

Could you be so kind as to give an example?

basic_auth bcrypt "This is the login for Grafana" {
	...
}

[SemoTech]

Thank you kindly again for your help @francislavoie .

I made the changes in Caddyfile as follows:

tm.example.com {
        handle /grafana* {
                encode gzip
                basic_auth bcrypt "Grafana Login" {
                        gf-admin $23ihnqbeocbqebfihbadfvnbsrgbsfb
                }
                reverse_proxy grafana:3000
        }

        handle {
                encode gzip
                basic_auth bcrypt "Teslamate Login" {
                         tm-admin $vweonqvbqvuabworhuqfvjanefjn
                }
                reverse_proxy teslamate:4000
        }
}

Sadly the login modal popup verbiage remains the same for either tm.example.com or tm.example.com/grafana.

Chrome’s modal popup shows:

Sign in
https://tm.example.com
Username:
Password:

Safari’s modal popup shows:

Sign in to tm.example.com:443
Your login information will be sent securely
User Name:
Password:

Is there anything else I could try?
Thank you.

[francislavoie]

Nope, that’s the only controls that exist. Browsers are totally allowed to ignore the “realm”.

If you want a fancier login form, then consider using something like Authelia + Caddy’s forward_auth.

[SemoTech]

Understood, I will look into that @francislavoie

Again, my thanks for all the help!