V2 load balancer example with Caddyfile

1. My Caddy version (caddy -version):

v2.0.0-beta.13

2. How I run Caddy:

command line: ./caddy2 start

a. System environment:

MacOS, command line, want to use docker once everything is running

b. Command:

./caddy2 start

d. My complete Caddyfile:

    127.0.0.1:2016 {
        reverse_proxy /  127.0.0.1:8050 127.0.0.1:8051 {
            #  lb_policy       round_robin
        }
    }

3. The problem I’m having:

I’m loving many things about Caddy. However, it’s very challenging to find docs for v2, links most often go to incompatible v1 documentation. AND when there are docs the examples are snippets that its not clear what to stick them into…a JSON API curl? a Caddyfile? The docs are challenging.

Like this, seems simple, but can’t for the life of me tweak it to work properly:
127.0.0.1:2016 {
reverse_proxy / 127.0.0.1:8050 127.0.0.1:8051 {
# lb_policy round_robin
}
}

What I would love: an example Caddyfile that sets up a simple load balancer.

6. Links to relevant resources:

https://caddyserver.com/docs/caddyfile/directives/reverse_proxy
https://caddyserver.com/docs/quick-starts/reverse-proxy

I think what’s tripping you up is the matcher. https://caddyserver.com/docs/caddyfile/concepts#matchers

Note that path matching is an exact match by default; you must append a * for a fast prefix match.

In your case, you can either use * for “all paths”, or just omit the matcher entirely, because your following arguments don’t look like paths (i.e. 127.0.0.1 doesn’t start with /).

I think this should just work:

127.0.0.1:2016 {
    reverse_proxy 127.0.0.1:8050 127.0.0.1:8051
}

To better understand why the docs are structured that way, it helps to understand how it works internally.

In Caddy v1, the only method of configuration was the Caddyfile. While it was very nice for simple configurations, it made many advanced use-cases tricky to implement and hard to automate.

Caddy v2 was a full rewrite, and the decision was to use JSON as the internal config structure. This gives a lot more flexibility and made it possible to simplify some features and make them fit more use-cases. In v2, the Caddyfile is essentially just a user-experience layer with a bunch of niceties to keep things easy to configure for people with basic to intermediate needs. It’s called a config adapter.


I’d like to know what exactly you found challenging in the documentation. Let us know specifically what you find hard to understand, and we can try to improve it.

Also, what links are pointing you to v1 documentation? If there’s any links inside the v2 docs leading to v1 docs, let us know where they are so we can fix them.

Thank you for the quick and detailed response!

I did attempt your solution in a Caddyfile. I made this recording just now which will should be very illuminating :slight_smile:

https://asciinema.org/a/296108

I will start to track links I mentioned from v2 to v1 docs. It’s mostly when I google for answers i end up on v1 pages, perhaps that’s my fault for not being smarter about my searching.

I want to say I have set a dozen Confluence development servers for my team requiring https and caddy has been amazing saving me sooo much time.

I really want to use it for everything, I’m just stumbling here and have tried a lot of things before reaching out. Thank you!

If you have a public key you can share I will even set up a VM with this very simple environment I’m demonstrating (for now just using the simple node http-server command for the two simple web servers).

Hmm, that is strange. Here’s a few things you could try:

  • Does it work with a single backend?

  • If you try with the caddy run command, it’ll run in the foreground, and we should see more logs.

  • Unfortunately, there’s no logging directive for Caddyfiles yet, but you can adapt your Caddyfile to JSON then add logging in the top-level JSON object:

    "logging": {
        "logs": {
            "default": {
                "level": "DEBUG",
                "writer": {
                    "filename": "debug.log",
                    "output": "file"
                }
            }
        }
    },

Thanks! I’m happy to figure out and work with JSON.

So I took the example exactly from the documentation on the home page, changing just the IP and ports which continues to exhibit the same behavior as my recorded demo :broken_heart: :

127.0.0.1:2016 # Your site's domain name

# Load balance between three backends with custom health checks
reverse_proxy 127.0.0.1:8050 127.0.0.1:8051 {
	lb_policy       random_choose 2
	health_path     /ok
	health_interval 10s
}

Then just to see if things were otherwise set up correctly I tested this which works as expected :white_check_mark: :

127.0.0.1:2016 # Your site's domain name

reverse_proxy 127.0.0.1:8050 {
}

So I’m thinking the very nice looking example from the home page: https://caddyserver.com/
with the heading:
HTTPS reverse proxy with custom load balancing and active health checks

is not actually working. Of course, I could be doing something silly, but as you can see I’m trying all the things I can think of.
I even switched out the simple test http server from node: “http-server -p 8050” to “python -m SimpleHTTPServer 8050” . Same result, working when tested directly.

Hmm, it’s looking like that might be a bug. These things can happen when in beta.

Were you able to enable logging via JSON config? It might reveal some issues.

1 Like

The Caddyfile of

127.0.0.1:2016 {
	reverse_proxy /  127.0.0.1:8050 127.0.0.1:8051
}

results in this JSON config:

{ "apps": { "http": { "servers": { "srv0": { "listen": [ ":2016" ], "routes": [ { "match": [ { "host": [ "127.0.0.1" ] } ], "handle": [ { "handler": "subroute", "routes": [ { "handle": [ { "handler": "reverse_proxy", "upstreams": [ { "dial": "127.0.0.1:8050" }, { "dial": "127.0.0.1:8051" } ] } ], "match": [ { "path": [ "/" ] } ] } ] } ], "terminal": true } ] } } } } }

So the resulting config has a host matcher of only 127.0.0.1. So when a request comes for localhost, Caddy doesn’t have a match for it, which results in empty response. The config that works is:

127.0.0.1:2016, localhost:2016 {
	reverse_proxy /  127.0.0.1:8050 127.0.0.1:8051
}

or

:2016 {
	reverse_proxy /  127.0.0.1:8050 127.0.0.1:8051
}

I’m mentally debating whether to label it as a bug. If it’s a bug, then we should look at any IP address in the range of 127.0.0.0/8 as a special case and imply matcher for localhost too.

3 Likes

Thanks for the good investigation!

It is not a bug, this is definitely intended. If the site address is defined to be a certain hostname in the Caddyfile, it will only answer to that hostname. To answer to other hostnames, either add them or omit the hostname (to answer to all hostnames), as @Mohammed90 suggested. :slight_smile:

1 Like