Reverse Proxy to dynamically Kubernetes Service

[reisman234 (Ralph)]

1. The problem I’m having:

I want to route traffic via a Kubernetes service to its belonging pod. Both service and pod are created dynamically, the pod’s name is a <uuid> and the name of the service is service-<uuid>.
Caddy is now in front of that and shout reverse_proxy on basis of the URI to the appropriate service.

E.g. /j/16057bff-86de-4995-9d69-cdaa19941259/api-call should reverse proxied to service-16057bff-86de-4995-9d69-cdaa19941259:8080/api-call

Until now, I figured out how to extract the UUID from the path and use it in a subsequent reverse_proxy directive, which works. But for now, I have no clue how to properly strip the prefix path with the uri directive.

2. Error messages and/or full log output:

No error

3. Caddy version:

v2.7.5

4. How I installed and ran Caddy:

a. System environment:

local minikube with Kubernetes 1.25

d. My complete Caddy config:

:8080 {

  @notFound `{err.status_code} == 404`

  route /j/ {
    respond "fail"
  }
  redir /j /j/

  route /j/* {
    #path_regexp job_id j/(.*)/
    uri strip_prefix /j/16057bff-86de-4995-9d69-cdaa19941259
    reverse_proxy http://service-16057bff-86de-4995-9d69-cdaa19941259:8080
  }

  @test {
    path_regexp test ^/dummy/(.*)
  }
  # this reverse_proxy works but I don't get the path sripped 
  reverse_proxy @test service-{re.test.1}.default:8080
    

  #@help {
  #    path_regexp test ^/prefix/(.*)
  #}
  #handle_path /prefix/(.*) {
  #  reverse_proxy @help service-{re.test.1}.workflow-api-demo:8080
  #}

  route /test/ {
    respond "Hello World"
  }
  redir /test /test/

}

[reisman234 (Ralph)]

I did some further testing with the Caddyfile and got the strip_prefix to work. This is currently the Caddyfile:

{
  debug
}
:8080 {

  @match_uuid {
    path_regexp job_id ^/j/([a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12})/
  }

  route @match_uuid {
    uri strip_prefix /j/{re.job_id.1}
    reverse_proxy service-{re.job_id.1}.default:8080
  }

  route /j/ {
    respond 404
  }
  redir /j /j/

  route /test/ {
    respond "Hello World"
  }
  redir /test /test/
}

But with this current solution, a new problem comes up. Every request with /j/* which does not contain a valid uuid the response is 200, but I don’t know why and how to redirect to /j/. I also added a default route like the following, but after applying that rule, every request did fail also with valid uuids.

route {
  respond 404
}

[francislavoie]

Use handle instead of route, which is mutually exclusive. So things that don’t match will fall through to the handle with no matcher.

:8080 {
	@match_uuid path_regexp job_id ^/j/([a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12})/
	handle @match_uuid {
		uri strip_prefix /j/{re.job_id.1}
		reverse_proxy service-{re.job_id.1}.default:8080
	}

	redir /test /test/
	handle /test/* {
		respond "Hello World"
	}

	handle {
		respond 404
	}
}

[reisman234 (Ralph)]

Thanks for your response, it works now as expected!

Could you perhaps explain why no logs appear in the route directive if no valid uuid is specified in the path? I find it a bit confusing if the path rule applies to an invalid uuid but is not executed completely and a 200 is returned as status_code. My expectation is that reverse_proxy should return an error as it cannot reach the uuid.

[francislavoie]

@reisman234

Could you perhaps explain why no logs appear in the route directive

What do you mean? What config did you use exactly? I didn’t enable the log directive in the config example I gave, so I can only guess what you tried unless you show it.

Either way, logging doesn’t having anything to do with routing (separate concepts). Please elaborate.

@reisman234

I find it a bit confusing if the path rule applies to an invalid uuid but is not executed completely and a 200 is returned as status_code. My expectation is that reverse_proxy should return an error as it cannot reach the uuid.

I’m also not following what you mean here. Show example requests with curl -v etc.