Reverse Proxy with per-upstream configuration pt. 2

Help
1

Hi Caddy Community!

(I actually posted this originally over a month ago and then never followed up…my apologies.)

I’m currently using haproxy as a egress proxy to load balance between multiple external web services. I’m experimenting with Caddy, and have a general question I’d like to get some perspective on.

In my use case, I load balance between multiple upstream servers. The challenge is, each upstream server requires unique configuration to authenticate (some use JWT headers, others url parameters, etc).

After authenticating, the servers can be treated as identical. (They all serve the same content / respond to requests in a uniform way).

I get the sense this is a atypical use case for proxies in general. For haproxy I was able to accomplish this setup as follows:

 [haproxy] -- unix socket --> [haproxy] -- https --> [upstream svc],

This works, and I think I could accomplish the same with Caddy…but in my experience it’s a bit cumbersome to manage the configuration, tracing, logging, etc.

Ideally I’d like way to accomplish per-upstream server configuration for a reverse proxy, without a double-proxy setup.

Before I go much further experimenting, I thought it was worth asking a question just-in-case I’m missing a completely obvious solution (wouldn’t be the first time! :slight_smile: )

Thanks in advance!
-Mike

Follow up from Matt’s original questions:
I’m running a double proxy configuration for haproxy due to the fact that each upstream server requires slightly different mechanisms for authentication.

I can post a copy of one of the haproxy config’s in the forum or a github gist etc. if it’s helpful - but I’ll warn you they are pretty verbose. Maybe a good starting point would be to show an example Caddyfile which is close to what I’m trying to accomplish. It’s certainly far more readable than my haproxy config :slight_smile:

(Note: this setup does not actually work, but I think it’s close and makes the point of what I’m trying to accomplish)

{
    auto_https off
}

# Load Balancer
http://*:22340 {
     tracing {
        span rpc_request
    }

    rewrite / /v3/{$INFURA_PROJECT_ID}
    reverse_proxy http://localhost:22341 http://localhost:22342 http://localhost:22343
}

# Infura Provider
http://*:22341 {
     tracing {
        span rpc_request
    }

    rewrite / /v3/{$INFURA_PROJECT_ID}
    header X-Upstream "infura"

    reverse_proxy {
        header_up Host {upstream_hostport}
        header_up -X-Forwarded-*
        to https://mainnet.infura.io
    }
}

# Alchemy Provider
http://*:22342 {
     tracing {
        span rpc_request
    }

    rewrite / /v2/${ALCHEMY_KEY}
    header X-Upstream "alchemy"

    reverse_proxy {
        header_up Host {upstream_hostport}
        header_up -X-Forwarded-*
        to https://eth-mainnet.g.alchemy.com
    }
}

# Cloudflare Public
http://*:22343 {
     tracing {
        span rpc_request
    }

    rewrite / /
    header X-Upstream "cloudflare"

    reverse_proxy {
        header_up Host {upstream_hostport}
        header_up -X-Forwarded-*
        to https://cloudflare-eth.com
    }
}
2

A few updates to the Caddyfile. This is now working, but I’m trying to improve the setup.

Ideally I’d like to skip any sort of secondary connection between the load balancer and upstreams. If that’s not possible, maybe moving from a intermediate http request over tcp to unix socket for transport would be a improvement.

# Global Settings
{
    auto_https off
    # enable debug logging of upstream
    debug
}


# Load Balancer
http://*:22340 {
     tracing {
        span rpc_request
    }

    log {
        output stdout
        format json
        level debug
    }

    # Note: each of the upstream providers needs to be configured slightly different
    reverse_proxy http://localhost:22341 http://localhost:22342 http://localhost:22343
}

# Infura Provider
http://*:22341 {
     tracing {
        span rpc_request
    }

    log {
        output stdout
        format json
        level debug
    }

    rewrite / /v3/{$INFURA_PROJECT_ID}
    header X-Upstream "alchemy"

    reverse_proxy {
        header_up Host {upstream_hostport}
        header_up -X-Forwarded-*
        to https://mainnet.infura.io
    }
}

# Alchemy Provider
http://*:22342  {
    tracing {
        span rpc_request
    }

    log {
        output stdout
        format json
        level debug
    }

    rewrite / /v2/{$ALCHEMY_KEY}
    header X-Upstream "alchemy"

    reverse_proxy {
        header_up Host {upstream_hostport}
        header_up -X-Forwarded-*
        to https://eth-mainnet.g.alchemy.com
    }
}

# Cloudflare Public
http://*:22343 {
    tracing {
        span rpc_request
    }

    # rewrite / /
    header X-Upstream "cloudflare"

    reverse_proxy {
        header_up Host {upstream_hostport}
        header_up -X-Forwarded-*
        to https://cloudflare-eth.com
    }
}
1 Like
3

It might be doable with some kind of custom dynamic upstreams module. You would need to write some Go code for that.

As for listening on an Unix socket instead, that’s simple enough. I think the bind directive is all you’ll need:

4

This topic was automatically closed after 30 days. New replies are no longer allowed.