1. Caddy version (caddy version
):
v2.1.1 h1:X9k1+ehZPYYrSqBvf/ocUgdLSRIuiNiMo7CvyGUQKeA=
2. How I run Caddy:
./caddy_linux_amd64 start
a. System environment:
CentOS 7 - VM with 5 network cards each with 1 ip address assigned (1.1.1.1, 2.2.2.2, 3.3.3.3, 4.4.4.4, and 5.5.5.5). The 6.6.6.6 ip address is an external ip address.
b. Command:
./caddy_linux_amd64 start
c. Service/unit/compose file:
n/a
d. My complete Caddyfile or JSON config:
1.1.1.1:80 {
respond /is_ok "Ok"
respond / "1.1.1.1"
respond /loadbalancer "1.1.1.1"
log {
output file /var/log/caddy/caddy-1.1.1.1.log
}
}
2.2.2.2:80 {
respond /is_ok "Ok"
respond / "2.2.2.2"
respond /loadbalancer "2.2.2.2"
log {
output file /var/log/caddy/caddy-2.2.2.2.log
}
}
3.3.3.3:80 {
respond /is_ok "Ok"
respond / "3.3.3.3"
respond /loadbalancer "3.3.3.3"
log {
output file /var/log/caddy/caddy-3.3.3.3.log
}
}
4.4.4.4:80 {
respond /is_ok "Ok"
respond / "4.4.4.4"
respond /loadbalancer "4.4.4.4"
log {
output file /var/log/caddy/caddy-4.4.4.4.log
}
}
www.example.com 5.5.5.5 {
bind 5.5.5.5
respond / "5.5.5.5"
reverse_proxy /loadbalancer 1.1.1.1:80 2.2.2.2:80 3.3.3.3:80 4.4.4.4:80 {
lb_policy round_robin
lb_try_duration 10s
lb_try_interval 1s
health_path /is_ok
health_status 200
health_body "Ok"
health_timeout 5s
health_interval 30s
}
log {
output file /var/log/caddy/caddy-master.log
}
}
3. The problem I’m having:
After startup and waiting 30 seconds for the reverse proxies to become ‘up’, when attempting to visit https://www.example.com/loadbalancer I continually receive 308 requests when I expect to receive the IP address. Visiting http://1.1.1.1/loadbalancer produces the correct response - and the same for the other three ip addresses (2.2.2.2, 3.3.3.3, and 4.4.4.4). Visiting http://1.1.1.1/is_ok also produces the correct response - and the same for the other three ip addresses (2.2.2.2, 3.3.3.3, and 4.4.4.4).
4. Error messages and/or full log output:
2020/09/04 21:49:56.294 INFO http.log.access handled request {"request": {"method": "GET", "uri": "/loadbalancer", "proto": "HTTP/1.1", "remote_addr": "2.2.2.2:60482", "host": "www.example.com", "headers": {"X-Forwarded-For": ["6.6.6.6"], "X-Forwarded-Proto": ["https"], "Accept-Encoding": ["gzip"], "User-Agent": ["insomnia/2020.3.3"], "Accept": ["*/*"]}}, "common_log": "2.2.2.2 - - [04/Sep/2020:17:49:56 -0400] \"GET /loadbalancer HTTP/1.1\" 308 0", "duration": 0.000054135, "size": 0, "status": 308, "resp_headers": {"Content-Type": [], "Server": ["Caddy"], "Location": ["https://www.example.com/loadbalancer"], "Connection": ["close"]}}
2020/09/04 21:49:56.323 INFO http.log.access handled request {"request": {"method": "GET", "uri": "/loadbalancer", "proto": "HTTP/1.1", "remote_addr": "3.3.3.3:52562", "host": "www.example.com", "headers": {"User-Agent": ["insomnia/2020.3.3"], "Accept": ["*/*"], "X-Forwarded-For": ["6.6.6.6"], "X-Forwarded-Proto": ["https"], "Accept-Encoding": ["gzip"]}}, "common_log": "3.3.3.3 - - [04/Sep/2020:17:49:56 -0400] \"GET /loadbalancer HTTP/1.1\" 308 0", "duration": 0.000114372, "size": 0, "status": 308, "resp_headers": {"Location": ["https://www.example.com/loadbalancer"], "Content-Type": [], "Server": ["Caddy"], "Connection": ["close"]}}
2020/09/04 21:49:56.360 INFO http.log.access handled request {"request": {"method": "GET", "uri": "/loadbalancer", "proto": "HTTP/1.1", "remote_addr": "4.4.4.4:45316", "host": "www.example.com", "headers": {"User-Agent": ["insomnia/2020.3.3"], "Accept": ["*/*"], "X-Forwarded-For": ["6.6.6.6"], "X-Forwarded-Proto": ["https"], "Accept-Encoding": ["gzip"]}}, "common_log": "4.4.4.4 - - [04/Sep/2020:17:49:56 -0400] \"GET /loadbalancer HTTP/1.1\" 308 0", "duration": 0.000053907, "size": 0, "status": 308, "resp_headers": {"Connection": ["close"], "Content-Type": [], "Server": ["Caddy"], "Location": ["https://www.example.com/loadbalancer"]}}
Log from Insomnia
* Preparing request to http://www.example.com/loadbalancer
* Current time is 2020-09-04T22:10:59.433Z
* Using libcurl/7.69.1-DEV OpenSSL/1.1.1d zlib/1.2.11 WinIDN libssh2/1.9.0_DEV nghttp2/1.40.0
* Using default HTTP version
* Disable timeout
* Enable automatic URL encoding
* Enable SSL validation
* Enable cookie sending with jar of 2 cookies
* Too old connection (1263 seconds), disconnect it
* Connection 33 seems to be dead!
* Closing connection 33
* TLSv1.3 (OUT), TLS alert, close notify (256):
* Trying 5.5.5.5:80...
* Connected to www.example.com (5.5.5.5) port 80 (#34)
> GET /loadbalancer HTTP/1.1
> Host: www.example.com
> User-Agent: insomnia/2020.3.3
> Accept: */*
* Mark bundle as not supporting multiuse
< HTTP/1.1 308 Permanent Redirect
< Connection: close
< Location: https://www.example.com/loadbalancer
< Server: Caddy
< Date: Fri, 04 Sep 2020 22:10:59 GMT
< Content-Length: 0
* Closing connection 34
* Issue another request to this URL: 'https://www.example.com/loadbalancer'
* Trying 5.5.5.5:443...
* Connected to www.example.com (5.5.5.5) port 443 (#35)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: C:\Users\Example\AppData\Local\Temp\insomnia_2020.3.3\2017-09-20.pem
* CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
* ALPN, server accepted to use h2
* Server certificate:
* subject: CN=www.example.com
* start date: Sep 4 14:13:50 2020 GMT
* expire date: Dec 3 14:13:50 2020 GMT
* subjectAltName: host "www.example.com" matched cert's "www.example.com"
* issuer: C=US; O=Let's Encrypt; CN=Let's Encrypt Authority X3
* SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x2dea3d138f0)
> GET /loadbalancer HTTP/2
> Host: www.example.com
> user-agent: insomnia/2020.3.3
> accept: */*
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* Connection state changed (MAX_CONCURRENT_STREAMS == 250)!
< HTTP/2 308
< date: Fri, 04 Sep 2020 22:10:59 GMT
< location: https://www.example.com/loadbalancer
< server: Caddy
< server: Caddy
< content-length: 0
* Received 0 B chunk
* Connection #35 to host www.example.com left intact
* Issue another request to this URL: 'https://www.example.com/loadbalancer'
* Found bundle for host www.example.com: 0x2dea6ac0960 [can multiplex]
* Re-using existing connection! (#35) with host www.example.com
* Connected to www.example.com (5.5.5.5) port 443 (#35)
* Using Stream ID: 3 (easy handle 0x2dea3d138f0)
> GET /loadbalancer HTTP/2
> Host: www.example.com
> user-agent: insomnia/2020.3.3
> accept: */*
< HTTP/2 308
< date: Fri, 04 Sep 2020 22:10:59 GMT
< location: https://www.example.com/loadbalancer
< server: Caddy
< server: Caddy
< content-length: 0
* Received 0 B chunk
* Connection #35 to host www.example.com left intact
* Issue another request to this URL: 'https://www.example.com/loadbalancer'
* Found bundle for host www.example.com: 0x2dea6ac0960 [can multiplex]
* Re-using existing connection! (#35) with host www.example.com
* Connected to www.example.com (5.5.5.5) port 443 (#35)
* Using Stream ID: 5 (easy handle 0x2dea3d138f0)
> GET /loadbalancer HTTP/2
> Host: www.example.com
> user-agent: insomnia/2020.3.3
> accept: */*
< HTTP/2 308
< date: Fri, 04 Sep 2020 22:10:59 GMT
< location: https://www.example.com/loadbalancer
< server: Caddy
< server: Caddy
< content-length: 0
* Received 0 B chunk
* Connection #35 to host www.example.com left intact
* Issue another request to this URL: 'https://www.example.com/loadbalancer'
* Found bundle for host www.example.com: 0x2dea6ac0960 [can multiplex]
* Re-using existing connection! (#35) with host www.example.com
* Connected to www.example.com (5.5.5.5) port 443 (#35)
* Using Stream ID: 7 (easy handle 0x2dea3d138f0)
> GET /loadbalancer HTTP/2
> Host: www.example.com
> user-agent: insomnia/2020.3.3
> accept: */*
< HTTP/2 308
< date: Fri, 04 Sep 2020 22:10:59 GMT
< location: https://www.example.com/loadbalancer
< server: Caddy
< server: Caddy
< content-length: 0
* Received 0 B chunk
* Connection #35 to host www.example.com left intact
* Issue another request to this URL: 'https://www.example.com/loadbalancer'
* Found bundle for host www.example.com: 0x2dea6ac0960 [can multiplex]
* Re-using existing connection! (#35) with host www.example.com
* Connected to www.example.com (5.5.5.5) port 443 (#35)
* Using Stream ID: 9 (easy handle 0x2dea3d138f0)
> GET /loadbalancer HTTP/2
> Host: www.example.com
> user-agent: insomnia/2020.3.3
> accept: */*
< HTTP/2 308
< date: Fri, 04 Sep 2020 22:10:59 GMT
< location: https://www.example.com/loadbalancer
< server: Caddy
< server: Caddy
< content-length: 0
5. What I already tried:
Spent better part of 2 days trying to work out a solution. Nothing has worked so far.
6. What I would expect:
When visiting http://5.5.5.5/loadbalancer I would receive a text of “1.1.1.1”, “2.2.2.2”, “3.3.3.3”, or “4.4.4.4” depending on the which backend proxy is chosen.