JnAlu
March 22, 2022, 7:02am
1
Assume that I want to A/B Test two sites. One site is hosted at HOST1, and the other, HOST2
If I do reverse_proxy HOST1 HOST2, where I ensure HOST1 and HOST2 are (almost) mutually exclusive in service - ie, they arn’t online/available at the same time, will user traffic be ever disrupted?
I say “almost” because, the service at HOST1 has the ability to know it will be decomissioned soon (I will send it a message before doing the switch) and will stop accepting new connections even if caddy attemps to forward it while continng to service existing requests
If this makes sense, and HOST1 HOST2 are HTTP servers, what status code should they send so that caddy not longer routes traffic to them?
Should I be solving this problem in a different way - like using a different LB algo or writing a custom reverse_proxy implmentation or similar?
JnAlu:
If this makes sense, and HOST1 HOST2 are HTTP servers, what status code should they send so that caddy not longer routes traffic to them?
Up to you. Use passive health checks and the unhealthy_status option. But HTTP status code 503 Service Unavailable is likely the right thing to use for that.
You haven’t said what your “current way” is, so I’m not sure what a “different way” could be.
Please fill out the help topic template, it’s important to get the full context for the question.
3 Likes
gcss
March 28, 2022, 5:07am
3
Curious if these can be solutions to the OP:
opened 08:50AM - 29 Mar 17 UTC
closed 12:43AM - 07 Mar 22 UTC
feature
### 1. What version of Caddy are you running (`caddy -version`)?
```
Caddy 0.9… .5
```
### 2. What are you trying to do?
Trying to proxy and load-balance to an upstream server, but, not by listing all IPs/hostnames, because the system I use (rancher) is providing multiple A records for service DNS requests.
Example dns reply using `dig`;
```
# dig hello-world.common-services.rancher.internal @169.254.169.250
; <<>> DiG 9.10.3-P4-Ubuntu <<>> hello-world.common-services.rancher.internal @169.254.169.250
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 64531
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;hello-world.common-services.rancher.internal. IN A
;; ANSWER SECTION:
hello-world.common-services.rancher.internal. 600 IN A 10.42.49.70
hello-world.common-services.rancher.internal. 600 IN A 10.42.131.117
hello-world.common-services.rancher.internal. 600 IN A 10.42.183.144
;; Query time: 0 msec
;; SERVER: 169.254.169.250#53(169.254.169.250)
;; WHEN: Wed Mar 29 08:37:12 UTC 2017
;; MSG SIZE rcvd: 103
```
Caddy resolves the host OK, but sends all requests to the IP address in the first A record.
### 3. What is your entire Caddyfile?
```text
https://api.hello-world-app.example.com {
log / stdout "{combined}"
tls some-email@hello-world-app.example.com
basicauth /signup hello-world-app mello-world-app
timeouts none
proxy / http://hello-world.common-services.rancher.internal:80 {
keepalive 0
transparent
websocket
}
}
:80 {
redir / https://{host}{uri} 301
}
```
### 4. How did you run Caddy (give the full command and describe the execution environment)?
Using Docker/Rancher, with the [dockerfile here](https://gist.github.com/furkanmustafa/e6cbec4961791e4dc3945eb2ea9f58e6).
### 6. What did you expect to see?
- Caddy does DNS request for entered proxy upstream hostname
- Considers all returned IP addresses as separate upstream host entry
- Does not cache this value for too long and updates it frequently;
- another way is; having an option to limit caching of this down to a few (~5?) seconds.
- or another way is allowing to send a signal (eg. HUP) to Caddy process to re-resolve proxy upstream hosts, so we can automate reloading upstream list without downtime.
### 7. What did you see instead (give full error messages and/or log)?
- All requests are forwarded to the first A record in the DNS response.
### 8. How can someone who is starting from scratch reproduce the bug as minimally as possible?
- Create a (sub)domain in any dns server / provider;
- Add two or more A entries, with the same (sub)domain but different IP addresses.
- Setup Caddy to proxy requests to that (sub)domain
- All requests will be proxied to the IP address in the first A record.
caddyserver:master ← caddyserver:proxy-refactor
opened 11:32PM - 08 Dec 21 UTC
Right now, proxy upstreams have to be hard-coded into the config (with the only … dyanamism coming from placeholders, which all act as a single upstream anyway). This change adds supports for truly dynamic upstreams, with the potential for every request to have different upstreams -- not only every request, but every _retry_ within a single request, too.
Instead of (or in addition to) specifying `upstreams` in your config, you could specify `dynamic_upstreams` and then define your upstream source module. Currently I'm implementing SRV and A/AAAA lookups as sources.
Fixes or related to:
- #1545 (currently the oldest open issue)
- #4174
- A currently-private sponsor's request. Hope to reveal this later!
When we're done, it will hopefully close or supercede #4350, #3801, #4446, and #4245. Closes #4341 too.
For dynamic SRV lookups:
```json
{
"handler": "reverse_proxy",
"dynamic_upstreams": {
"source": "srv",
"name": "_service._proto.name"
}
}
```
For dynamic A/AAAA DNS lookups:
```json
{
"handler": "reverse_proxy",
"dynamic_upstreams": {
"source": "a",
"name": "backends.example.com",
"port": "8080"
}
}
```
The default refresh interval for lookups is 1 minute, but can be configured with the `refresh` property. SRV lookups can also optionally be configured by their individual service, proto, and name components, or the entire domain as the name by itself.
Still need to integrate health checks, add docs and tests, and cache eviction.
May be of interest to @danlsgiga, @dazoot, @sorenisanerd, and @furkanmustafa.
CC: @jjiang-stripe and @cds2-stripe
Will cook up a POC so francislavoie can review.
Well, we don’t really know what they’re asking. We need clarification first. Hence why I asked that they fill out the help topic template, to avoid any confusion, and so we don’t waste our time making assumptions.
1 Like
gcss
March 28, 2022, 5:37am
5
Fair. I will create a new thread that has to do with LB
system
April 21, 2022, 7:02am
6
This topic was automatically closed after 30 days. New replies are no longer allowed.