Hey guys!
My team and I try switching to Caddy from Nginx. We’re testing it for a few weeks already, and the server is really cool! I like the build tool, xcaddy, it makes custom builds super simple. And the config file, it’s pretty flexible and easy to read.
We mostly use Caddy as an edge web server works as a reverse proxy. It’s at least as good as Nginx in that role, but only when we have exactly one upstream per server. We tried multiple ways to set up multi-upstream servers, but still not sure why some of the ways don’t work properly, and what is the best way.
I’ll show what we did on a simplified example that can be run on any computer - a reverse proxy to a publically available website, https://www.garyvaynerchuk.com/
I’ll be testing things using 3 different uls, one per upstream:
- Upstream1, named ajaxGoogleapis: https://garyvaynerchuk.localhost/ajax-googleapis/ajax/libs/jquery/3.2.1/jquery.min.js?ver=3.2.1
- Upstream2, named awsCdn: https://garyvaynerchuk.localhost/aws-cdn/gv2016wp/wp-content/uploads/20191212171239/text-me.png
- Upstream3, named website: https://www.garyvaynerchuk.com/
Ways we tried
1. Using matches only
https://garyvaynerchuk.${TLD} {
${TLS_SETTING}
@ajaxGoogleapis path /ajax-googleapis/*
header @ajaxGoogleapis X-Caddy-Route "ajaxGoogleapis"
uri @ajaxGoogleapis strip_prefix "/ajax-googleapis"
reverse_proxy @ajaxGoogleapis {
header_up host ajax.googleapis.com
to https://ajax.googleapis.com
}
@awsCdn path /aws-cdn/*
header @awsCdn X-Caddy-Route "awsCdn"
uri @awsCdn strip_prefix "/aws-cdn"
reverse_proxy @awsCdn {
header_up host s3.amazonaws.com
to https://s3.amazonaws.com
}
@website path not /ajax-googleapis/* /aws-cdn/*
header @website X-Caddy-Route "website"
reverse_proxy @website {
header_up host www.garyvaynerchuk.com
to https://www.garyvaynerchuk.com
}
}
I have 3 matchers here, and I expect that the first two would catch specific prefixes only, and the 3rd one catches everything else. I also have a header
directive for every matcher that will help me see which matcher caught the request.
1.1 ajaxGoogleapis
- returns empty page (0 bytes)
- has
x-caddy-route: website
header
1.2 awsCdn
- returns empty page (0 bytes)
- has
x-caddy-route: website
header
1.3 website
- returns empty page (0 bytes)
- doesn’t have
x-caddy-route
header
2. Using matchers + route
https://garyvaynerchuk.${TLD} {
${TLS_SETTING}
@ajaxGoogleapis path /ajax-googleapis/*
route @ajaxGoogleapis {
header X-Caddy-Route "ajaxGoogleapis"
uri strip_prefix "/ajax-googleapis"
reverse_proxy {
header_up host ajax.googleapis.com
to https://ajax.googleapis.com
}
}
@awsCdn path /aws-cdn/*
route @awsCdn {
header X-Caddy-Route "awsCdn"
uri strip_prefix "/aws-cdn"
reverse_proxy {
header_up host s3.amazonaws.com
to https://s3.amazonaws.com
}
}
@website path not /wp-cdn/* /aws-cdn/* /ajax-googleapis/*
route @website {
header X-Caddy-Route "website"
reverse_proxy {
header_up host www.garyvaynerchuk.com
to https://www.garyvaynerchuk.com
}
}
}
It’s almost identical, but I’ve wrapped every header
and reverse_proxy
pair into route
instead of copying the named matcher every time.
2.1 ajaxGoogleapis
- works fine
2.2 awsCdn
- works fine
2.3 website
- returns empty page (0 bytes)
- doesn’t have
x-caddy-route
header
3. Matchers + route, “default” upstream unwrapped
https://garyvaynerchuk.${TLD} {
${TLS_SETTING}
@ajaxGoogleapis path /ajax-googleapis/*
route @ajaxGoogleapis {
header X-Caddy-Route "ajaxGoogleapis"
uri strip_prefix "/ajax-googleapis"
reverse_proxy {
header_up host ajax.googleapis.com
to https://ajax.googleapis.com
}
}
@awsCdn path /aws-cdn/*
route @awsCdn {
header X-Caddy-Route "awsCdn"
uri strip_prefix "/aws-cdn"
reverse_proxy {
header_up host s3.amazonaws.com
to https://s3.amazonaws.com
}
}
header X-Caddy-Route "website"
reverse_proxy {
header_up host www.garyvaynerchuk.com
to https://www.garyvaynerchuk.com
}
}
It’s almost identical to #2, the only difference - last reverse_proxy
and header
pair is unwrapped.
3.1 ajaxGoogleapis
- works fine
3.2 awsCdn
- works fine
3.3 website
- works fine
4. Matchers + handle, “default” upstream unwrapped
https://garyvaynerchuk.${TLD} {
${TLS_SETTING}
@ajaxGoogleapis path /ajax-googleapis/*
handle @ajaxGoogleapis {
header X-Caddy-Route "ajaxGoogleapis"
uri strip_prefix "/ajax-googleapis"
reverse_proxy {
header_up host ajax.googleapis.com
to https://ajax.googleapis.com
}
}
@awsCdn path /aws-cdn/*
handle @awsCdn {
header X-Caddy-Route "awsCdn"
uri strip_prefix "/aws-cdn"
reverse_proxy {
header_up host s3.amazonaws.com
to https://s3.amazonaws.com
}
}
header X-Caddy-Route "website"
reverse_proxy {
header_up host www.garyvaynerchuk.com
to https://www.garyvaynerchuk.com
}
}
Almost identical to #3, but uses handle
instead of route
. Behaves exactly the same, not sure what is the difference between the two directives.
4.1 ajaxGoogleapis
- works fine
4.2 awsCdn
- works fine
4.3 website
- works fine
Environment
1. Caddy version (caddy version
):
v2.2.1 h1:Q62GWHMtztnvyRU+KPOpw6fNfeCD3SkwH7SfT1Tgt2c=
Built with:
xcaddy build v2.2.1 && sudo setcap 'cap_net_bind_service=+ep' ./caddy
2. How I run Caddy:
a. System environment:
Pretty sure that it doesn’t matter, tested on multiple different machines. My primary test installation is Ubuntu 20.04.1, x86_64, non-docker
b. Command:
./caddy-vanilla/caddy run -config websites/conf-dist/Caddyfile_garyvaynerchuk