1. The problem I’m having:
I’m trying to configure Caddy to act as a reverse proxy in front of GitHub Enterprise Server which uses port 8443 for its management console. This port is blocked on my company’s network. When I access the URL for the reverse proxy I receive a 502 Bad Gateway error.
Note that if I run Caddy locally using caddy start and access it using http://localhost the reverse proxy appears to be working correctly and I am taken to the GitHub Enterprise Server management console. However, the URL in the web browser changes to the upstream URL and port; not sure if that is supposed to happen?
2. Error messages and/or full log output:
curl -vL https://manage.github.example.com/
* Trying [REDACTED IP ADDRESS]:443...
* Connected to manage.github.example.com (REDACTED) port 443 (#0)
* ALPN: offers h2
* ALPN: offers http/1.1
* CAfile: /etc/ssl/cert.pem
* CApath: none
* (304) (OUT), TLS handshake, Client hello (1):
* (304) (IN), TLS handshake, Server hello (2):
* (304) (IN), TLS handshake, Unknown (8):
* (304) (IN), TLS handshake, Certificate (11):
* (304) (IN), TLS handshake, CERT verify (15):
* (304) (IN), TLS handshake, Finished (20):
* (304) (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / AEAD-AES256-GCM-SHA384
* ALPN: server accepted h2
* Server certificate:
* subject: CN=manage.github.example.com
* start date: Mar 29 10:21:42 2023 GMT
* expire date: Jun 27 11:07:50 2023 GMT
* subjectAltName: host "manage.github.example.com" matched cert's "manage.github.example.com"
* issuer: C=US; O=Google Trust Services LLC; CN=GTS CA 1D4
* SSL certificate verify ok.
* Using HTTP2, server supports multiplexing
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* h2h3 [:method: GET]
* h2h3 [:path: /]
* h2h3 [:scheme: https]
* h2h3 [:authority: manage.github.example.com]
* h2h3 [user-agent: curl/7.86.0]
* h2h3 [accept: */*]
* Using Stream ID: 1 (easy handle 0x7fb21f011400)
> GET / HTTP/2
> Host: manage.github.example.com
> user-agent: curl/7.86.0
> accept: */*
>
< HTTP/2 502
< content-type: text/html; charset=UTF-8
< referrer-policy: no-referrer
< content-length: 332
< date: Wed, 29 Mar 2023 15:50:41 GMT
< alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
<
<html><head>
<meta http-equiv="content-type" content="text/html;charset=utf-8">
<title>502 Server Error</title>
</head>
<body text=#000000 bgcolor=#ffffff>
<h1>Error: Server Error</h1>
<h2>The server encountered a temporary error and could not complete your request.<p>Please try again in 30 seconds.</h2>
<h2></h2>
</body></html>
* Connection #0 to host manage.github.example.com left intact
Log entry:
{
"level": "debug",
"ts": 1680105520.7144086,
"logger": "http.handlers.reverse_proxy",
"msg": "upstream roundtrip",
"upstream": "github.example.com:8443",
"duration": 3.001128821,
"request": {
"remote_ip": "10.119.3.35",
"remote_port": "44493",
"proto": "HTTP/1.1",
"method": "GET",
"host": "github.example.com:8443",
"uri": "/",
"headers": {
"X-Forwarded-For": [
"10.119.3.35"
],
"User-Agent": [
"GoogleHC/1.0"
],
"X-Forwarded-Proto": [
"http"
],
"X-Forwarded-Host": [
"10.119.3.35"
]
}
},
"error": "dial tcp [GOOGLE LOAD BALANCER IP ADDRESS]:8443: i/o timeout"
}
{
"level": "error",
"ts": 1680105520.714512,
"logger": "http.log.error",
"msg": "dial tcp [GOOGLE LOAD BALANCER IP ADDRESS]:8443: i/o timeout",
"request": {
"remote_ip": "10.119.3.35",
"remote_port": "44493",
"proto": "HTTP/1.1",
"method": "GET",
"host": "10.119.3.35",
"uri": "/",
"headers": {
"User-Agent": [
"GoogleHC/1.0"
],
"Connection": [
"Keep-alive"
]
}
},
"duration": 3.001374337,
"status": 502,
"err_id": "3nxz5g64v",
"err_trace": "reverseproxy.statusError (reverseproxy.go:1272)"
}
3. Caddy version:
v2.6.4 h1:2hwYqiRwk1tf3VruhMpLcYTg+11fCdr8S3jhNAdnPy8=
4. How I installed and ran Caddy:
I am running Caddy as a Docker image on Google Kubernetes Engine. Caddy is running as a Kubernetes Deployment with a Kubernetes Ingress and NodePort Service bound to port 80 in front.
a. System environment:
Google Kubernetes Engine with automatic TLS turned off within Caddy as I’m using a Google-managed TLS certificate with the Google external HTTPS load balancers. I have verified that Caddy is running correctly by initially deploying a minimal “Hello World” Caddy configuration.
b. Command:
N/A
c. Service/unit/compose file:
Dockerfile:
FROM caddy
COPY proxy/Caddyfile /etc/caddy/Caddyfile
d. My complete Caddy config:
{
auto_https off
debug
}
:80 {
handle /health {
respond 200
}
reverse_proxy https://github.example.com:8443 {
header_up Host {upstream_hostport}
}
}