Path handling issue using the map directive

1. Caddy version (caddy version):

v2.4.0-beta.1 built from the master including the Cloudflare module

2. How I run Caddy:

I use Caddyfile rather than JSON.

a. System environment:

FreeNAS 11.3 (a FreeBSD derivative)

b. Command:

service caddy start

c. Service/unit/compose file:

n/a

d. My complete Caddyfile or JSON config:

This is an extract from the relevant section of my Caddyfile that I’m looking at enhancing.

udance.com.au www.udance.com.au {

  encode gzip
  import tlsdns
  import logging udance
  import authproxy /phpmyadmin*

  reverse_proxy /tautulli* 10.1.1.26:8181
  reverse_proxy /transmission* 10.1.1.28:9091
  reverse_proxy 10.1.1.55
}

To avoid cluttering the above extract, I’ve not included the snippets. For completeness, I’ve included them below.

(tlsdns) {
  tls {
    dns cloudflare [REDACTED]
  }
}

(authproxy) {
  basicauth {args.0} {
    admin [REDACTED]
  }
}

(logging) {
  log {
    format json
    output file /var/log/caddy/{args.0}.log {
      roll_keep 7
    }
  }
}

3. The problem I’m having:

The Caddy block serves the domain and two sub-paths. I wanted some flexibility around taking individual services offline and thought that the map directive (which I’ve recently become a big fan of) could help me here.

Here’s my attempt at building this flexibility into the Caddy block with the focus, at this stage, just on the /tautulli sub-path. I noticed that there was a redirect to udance.com.au/tautulli/auth/login?redirect_uri=/tautulli if I entered udance.com.au/tautulli in a browser address bar, therefore, I had to use a regular expression for the path.

udance.com.au www.udance.com.au {

  encode gzip
  import tlsdns
  import logging udance
  import authproxy /phpmyadmin*

  map {path} {backend} {online} {

#   PATH            BACKEND        ONLINE
#---------------------------------------------------------------
    ~^/tautulli.*   10.1.1.26:8181   yes
#   /transmission   10.1.1.28:9091   yes
#   /               10.1.1.55:80     yes  
    default         unknown          yes
  }

# Error handling
  @unknown expression `{backend} == "unknown"`
  handle @unknown {
    respond "Denied {path}" 403
  }

#
  @offline expression `{online} == "no"`
  handle @offline {
    # Do whatever e.g. respond/redir 
  }

  reverse_proxy {path} {backend}
}

4. Error messages and/or full log output:

The solution works approximately 50% of the time. Reloading the page seems to randomly switch me between the Tautulli login screen or the browser responding with a 502 error.

There are invalid dial address entries in the Caddy log file with missing port address. Here’s a couple of entries:

{"level":"error","ts":1615865654.7320564,"logger":"http.log.error.log3","msg":"making dial info: upstream {http.request.uri.path}:invalid dial address /tautulli/home: address tautulli/home: missing port in address","request":{"remote_addr":"10.1.1.136:57213","proto":"HTTP/2.0","method":"GET","host":"udance.com.au","uri":"/tautulli/home","headers":{"Authorization":["Basic YWRtaW46OComQXZLTFJ0SCUyITJGaUFlZQ=="],"Upgrade-Insecure-Requests":["1"],"Cache-Control":["max-age=0"],"Sec-Ch-Ua":["\"Google Chrome\";v=\"89\", \"Chromium\";v=\"89\", \";Not A Brand\";v=\"99\""],"User-Agent":["Mozilla/5.0(Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.82 Safari/537.36"],"Accept-Language":["en-GB,en-US;q=0.9,en;q=0.8"],"Sec-Fetch-Site":["same-origin"],"Sec-Fetch-Mode":["navigate"],"Sec-Ch-Ua-Mobile":["?0"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"],"Sec-Fetch-User":["?1"],"Sec-Fetch-Dest":["document"],"Accept-Encoding":["gzip, deflate, br"],"Cookie":["tautulli_token_5837b2dff06e4f0e9f86551fd6ec5efc=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ1c2VyX2lkIjoxNTQzNTU3LCJ1c2VyIjoiYmFzaWxoIiwidXNlcl9ncm91cCI6ImFkbWluIiwiZXhwIjoxNjE4NDU2NzU2fQ._BgPiEFn3XchkSRdY6SsWfAmebv58it-waZ84Clf1r0; tk_or=%22%22; tk_lr=%22%22; wordpress_logged_in_01ad53d5bf4f84d52a16c29bd439eb90=basil%7C1645606391%7Cg0BlFaRlmpOtRQIufy2y3zkZMaJQ5kBwHgodcQZTCol%7C21fc4a76a8a48a1988b5664d18c3115b251ad355646ec0b0bbf00b4cd0c0aa34; wp-settings-1=libraryContent%3Dbrowse; wp-settings-time-1=1614070400;__cfduid=d9d479562ae0d9e9a028d749d6594ad101615662603"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","proto_mutual":true,"server_name":"udance.com.au"}},"duration":0.00032652,"status":502,"err_id":"uf8m3q16a","err_trace":"reverseproxy.statusError (reverseproxy.go:827)"}
{"level":"error","ts":1615865657.5854928,"logger":"http.log.error.log3","msg":"making dial info: upstream {http.request.uri.path}: invalid dial address /tautulli/home:address tautulli/home: missing port in address","request":{"remote_addr":"10.1.1.136:57213","proto":"HTTP/2.0","method":"GET","host":"udance.com.au","uri":"/tautulli/home","headers":{"Cache-Control":["max-age=0"],"Sec-Fetch-User":["?1"],"Accept-Language":["en-GB,en-US;q=0.9,en;q=0.8"],"Authorization":["Basic YWRtaW46OComQXZLTFJ0SCUyITJGaUFlZQ=="],"Sec-Ch-Ua":["\"Google Chrome\";v=\"89\", \"Chromium\";v=\"89\", \";Not A Brand\";v=\"99\""],"Upgrade-Insecure-Requests":["1"],"Accept-Encoding":["gzip, deflate, br"],"Sec-Ch-Ua-Mobile":["?0"],"Sec-Fetch-Site":["same-origin"],"Sec-Fetch-Mode":["navigate"],"Cookie":["tautulli_token_5837b2dff06e4f0e9f86551fd6ec5efc=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ1c2VyX2lkIjoxNTQzNTU3LCJ1c2VyIjoiYmFzaWxoIiwidXNlcl9ncm91cCI6ImFkbWluIiwiZXhwIjoxNjE4NDU2NzU2fQ._BgPiEFn3XchkSRdY6SsWfAmebv58it-waZ84Clf1r0; tk_or=%22%22; tk_lr=%22%22; wordpress_logged_in_01ad53d5bf4f84d52a16c29bd439eb90=basil%7C1645606391%7Cg0BlFaRlmpOtRQIufy2y3zkZMaJQ5kBwHgodcQZTCol%7C21fc4a76a8a48a1988b5664d18c3115b251ad355646ec0b0bbf00b4cd0c0aa34; wp-settings-1=libraryContent%3Dbrowse; wp-settings-time-1=1614070400; __cfduid=d9d479562ae0d9e9a028d749d6594ad101615662603"],"User-Agent":["Mozilla/5.0 (WindowsNT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.82 Safari/537.36"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"],"Sec-Fetch-Dest":["document"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","proto_mutual":true,"server_name":"udance.com.au"}},"duration":0.000250855,"status":502,"err_id":"x9x62bdu2","err_trace":"reverseproxy.statusError (reverseproxy.go:827)"}

Relevant output from caddy adapt --pretty

                                                {
                                                        "match": [
                                                                {
                                                                        "host": [
                                                                                "udance.com.au",
                                                                                "www.udance.com.au"
                                                                        ]
                                                                }
                                                        ],
                                                        "handle": [
                                                                {
                                                                        "handler": "subroute",
                                                                        "routes": [
                                                                                {
                                                                                        "handle": [
                                                                                                {
                                                                                                        "defaults": [
                                                                                                                "unknown",
                                                                                                                "yes"
                                                                                                        ],
                                                                                                        "destinations": [
                                                                                                                "{backend}",
                                                                                                                "{online}"
                                                                                                        ],
                                                                                                        "handler": "map",
                                                                                                        "mappings": [
                                                                                                                {
                                                                                                                        "input_regexp": "^/tautulli.*",
                                                                                                                        "outputs": [
                                                                                                                                "10.1.1.26:8181",
                                                                                                                                "yes"
                                                                                                                        ]
                                                                                                                }
                                                                                                        ],
                                                                                                        "source": "{http.request.uri.path}"
                                                                                                }
                                                                                        ]
                                                                                },
                                                                                {
                                                                                        "handle": [
                                                                                                {
                                                                                                        "handler": "authentication",
                                                                                                        "providers": {
                                                                                                                "http_basic": {
                                                                                                                        "accounts": [
                                                                                                                                {
                                                                                                                                        "password": [REDACTED],
                                                                                                                                        "username": "admin"
                                                                                                                                }
                                                                                                                        ],
                                                                                                                        "hash": {
                                                                                                                                "algorithm": "bcrypt"
                                                                                                                        },
                                                                                                                        "hash_cache": {}
                                                                                                                }
                                                                                                        }
                                                                                                }
                                                                                        ],
                                                                                        "match": [
                                                                                                {
                                                                                                        "path": [
                                                                                                                "/phpmyadmin*"
                                                                                                        ]
                                                                                                }
                                                                                        ]
                                                                                },
                                                                                {
                                                                                        "handle": [
                                                                                                {
                                                                                                        "encodings": {
                                                                                                                "gzip": {}
                                                                                                        },
                                                                                                        "handler": "encode"
                                                                                                }
                                                                                        ]
                                                                                },
                                                                                {
                                                                                        "group": "group9",
                                                                                        "handle": [
                                                                                                {
                                                                                                        "handler": "subroute",
                                                                                                        "routes": [
                                                                                                                {
                                                                                                                        "handle": [
                                                                                                                                {
                                                                                                                                        "body": "Denied {http.request.uri.path}",
                                                                                                                                        "handler": "static_response",
                                                                                                                                        "status_code": 403
                                                                                                                                }
                                                                                                                        ]
                                                                                                                }
                                                                                                        ]
                                                                                                }
                                                                                        ],
                                                                                        "match": [
                                                                                                {
                                                                                                        "expression": "{backend} == \"unknown\""
                                                                                                }
                                                                                        ]
                                                                                },
                                                                                {
                                                                                        "group": "group9",
                                                                                        "handle": [
                                                                                                {
                                                                                                        "handler": "subroute",
                                                                                                        "routes": [
                                                                                                                {
                                                                                                                        "handle": [
                                                                                                                                {
                                                                                                                                        "handler": "static_response",
                                                                                                                                        "headers": {
                                                                                                                                                "Location": [
                                                                                                                                                        "https://udance.statuspage.io"
                                                                                                                                                ]
                                                                                                                                        },
                                                                                                                                        "status_code": 302
                                                                                                                                }
                                                                                                                        ]
                                                                                                                }
                                                                                                        ]
                                                                                                }
                                                                                        ],
                                                                                        "match": [
                                                                                                {
                                                                                                        "expression": "{online} == \"no\""
                                                                                                }
                                                                                        ]
                                                                                },
                                                                                {
                                                                                        "handle": [
                                                                                                {
                                                                                                        "handler": "reverse_proxy",
                                                                                                        "upstreams": [
                                                                                                                {
                                                                                                                        "dial": "{http.request.uri.path}"
                                                                                                                },
                                                                                                                {
                                                                                                                        "dial": "{backend}"
                                                                                                                }
                                                                                                        ]
                                                                                                }
                                                                                        ]
                                                                                }
                                                                        ]
                                                                }
                                                        ],
                                                        "terminal": true
                                                },
  

5. What I already tried:

  1. Checked the Caddy log.
  2. Confirmed the regex expression used.

6. Links to relevant resources:

  1. Clarity on the map directive example

Ah, you can’t use placeholders as your matcher here. Caddy is parsing this as two separate backends, so it load balances between the two, meaning half the time it tries to use {path} as the upstream, and the other half, it uses {backend}. You can see this clearly in the JSON:

This is because matchers must either be something that starts with / (path matcher), with @ (named matcher) or exactly *; otherwise the Caddyfile adapter won’t take the first argument as a matcher, but instead pass it through to the argument parsing logic of that directive.

In your case, matching on {path} is a tautology, it’ll always be true because that’s actually the current path. Like if you tried to do a named matcher like @proxy path {path}, that would always be true. So I think you should just remove the {path} from that line and it should work just fine… if I understand what you’re trying to do.

2 Likes

That was tricky! I’m still mulling it over. One thing I’m beginning to appreciate, caddy adapt --pretty provides a level of detail that is hidden from view in the Caddyfile.

So, here is the final, working transformation of that Caddy block excerpt. It’s a lot more flexible than the original, but requires a greater depth of understanding to build.

udance.com.au www.udance.com.au {

  encode gzip
  import tlsdns
  import logging udance
  import authproxy /phpmyadmin*

  map {path} {backend} {online} {

#   PATH                BACKEND          ONLINE
#---------------------------------------------------------------
    ~^/tautulli.*       10.1.1.26:8181   yes
    ~^/transmission.*   10.1.1.28:9091   yes
    ~^/.*               10.1.1.55:80     yes
  }

  @offline expression `{online} == "no"`
  handle @offline {
    # Do whatever e.g. respond/redir 
  }

  reverse_proxy {backend}
}
2 Likes

This topic was automatically closed after 30 days. New replies are no longer allowed.