Only one connection present if only one backend provided in reverse proxy

Help
1

1. The problem I’m having:

I’m trying to build a web application with front-end and back-end separation using k8s, and I am planning to use Caddy as its reverse proxy.

The back-end Flask application has been deployed with backups using a k8s deployment and exposed by a k8s service.
I use wireshark to capture the traffic. From my observation, Caddy uses a long-lived connection to handle all requests to the back-end, regardless of who initiates the request. In this way, all requests will be routed via the long-lived connection to a single pod, and load balancing will not be performed. I think Caddy should initiate a connection to the back-end for each connection from the client, so that Kubernetes can perform load balancing.

I’m new in caddy, and I’m wondering if there is any configuration can meet my requirement or if there’s an issue with my current configuration

2. Error messages and/or full log output:

no error message here

3. Caddy version:

i use docker image caddy:alpine

/srv # caddy version
v2.6.4 h1:2hwYqiRwk1tf3VruhMpLcYTg+11fCdr8S3jhNAdnPy8=

4. How I installed and ran Caddy:

a. System environment:

ubuntu22.04 vps
using kind to deploy k8s
kind version 0.17.0

b. Command:

Dockerfile

FROM caddy:alpine

COPY ./Caddyfile /etc/caddy/Caddyfile
RUN echo "hello" > /srv/index.html

c. Service/unit/compose file:

caddy-deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: caddy
  template:
    metadata:
      labels:
        app: caddy
    spec:
      containers:
      - name: caddy
        image: docker.io/library/front:v1
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 80

flask-app-service.yaml

apiVersion: v1
kind: Service
metadata:
  name: flaskapp
spec:
  selector:
    app: flask-app
  ports:
  - name: flask-port
    protocol: TCP
    port: 3000
    targetPort: 3000

d. My complete Caddy config:

:80 {
    root * /srv
    try_files {path} {path}/ /index.html
    file_server

    handle_path /api/* {
        reverse_proxy flaskapp:3000
    }
}

e.What I already tried

I changed Caddyfile to this

    handle_path /api/* {
        reverse_proxy flaskapp:3000 {
            transport http {
                keepalive off
            }
        }
    }

In this case, each request becomes a new connection, even if the request header Connection: keep-alive is added.

What I need is for all long-lived connections to be treated as such, and for each connection to be distinguished so that load balancing can take effect.

2

Welcome @z33!

Go’s http.Transport pools connections to maximize throughput and decrease latency.

It sounds like Kubernetes is doing TCP load balancing, not HTTP load balancing. An HTTP load balancer (like Caddy’s reverse_proxy) would be able to LB individual HTTP requests no matter what connections they come in on. A TCP load balancer of course only cares about connections.

I don’t know anything about Kubernetes, but is there a way to configure HTTP LB instead of TCP LB?

If not, you could play with keep_alive:

(This kind of reminds me of the NTLM transport, which uses HTTP to authenticate the connection, violating the OSI layer boundaries, yuck – so I had to be careful to multiplex only the same-authed requests on their proper connection, not a lot of fun because it requires Keep-Alive to be enabled.)

3

Thank you for taking the time to answer my question.

I think you have got the point that what I need is TCP LB. It seems k8s can’t configure HTTP LB, so can Caddy perform like a TCP LB? I have read the document about keep-alive, but didn’t find something relevant. Setting keepalive to off will cause all connections to be broken down into short-lived connections.

Shamefully, I had never heard of HTTP LB before, I also have a simple question about how can it works. I can’t image how can Caddy handle all incoming request in only one outgoing connection. In TCP LB, proxy can associate incoming connections with outgoing connections, so every request can be handled correctly. When you handle all request in one outgoing connection, how can proxy know this response is corresponding to which request?

4

Caddy can be a TCP proxy, but you need to use GitHub - mholt/caddy-l4: Layer 4 (TCP/UDP) app for Caddy. It’s unfortunately JSON config only for now. You could terminate TLS with Caddy, then proxy the HTTP bytes transparently to k8s and let it do the rest. Maybe.

Another thing you could try re keepalives… You could try re-adding the Connection header as received by Caddy to the proxy request; the proxy drops any hop-by-hop headers by default and Connection is one of them, because it typically only makes sense for the first hop. Try this; I have no idea how it’ll behave but I’m curious to see if it helps. Add it within your reverse_proxy block (just above the transport):

header_up Connection {header.Connection}
5

thank you for your patience.

so you mean i need to set keepalive to false, then add connection header manually?
it’s a little difficult for me to understand how can this works, but I’ll give it a try

6

Yeah, that’s what I mean. I’m not sure if it’ll work at all, I’ve never tried it. Just a theory.

7

well, it seems work, but not quite…

here is my new Caddy file

:80 {
    handle_path /api/* {
        reverse_proxy flaskapp:3000 {
            header_up Connection {header.Connection}
            transport http {
                keepalive off
            }
        }
    }
    handle {
        root * /srv
        try_files {path} {path}/ /index.html
        file_server
    }
}

in this case, caddy would add user provided connection header, but it will also add a Connection: close due to the keepalive off

If I remove the transport block, the whole network will be messed up. And Caddy behaves randomly… However sometime it does work as expect, but when viewed in wireshark, all request traffic will be repeated twice and there will be a lot of tcp retransmissions, sometimes causing 502 errors.

8

Ah dang. Yeah, that’s a problem.

We’re using the Go stdlib HTTP client, and it has that built-in behaviour to add close if turning off keepalives. That’s a sensible behaviour usually, but your case is outside the happy path :frowning:

I don’t have a good answer for you unfortunately. I think you’ll have to look into using a smarter HTTP-based ingress for your k8s. It doesn’t need to be Caddy’s ingress project, it could be an nginx-based one or whatever. It just needs to be able to understand the difference between individual HTTP requests on the TCP connection and load balance those to whatever pod.

9

Okay, that’s too bad.
Actually, I can deploy my service without a reverse proxy – setting up CORS in my Flask app will be sufficient. However, I still think deploy without CORS would be more elegant.

Thanks again for your help. Have a nice day!

10

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.