Ah okay, thanks for trying. I think I know what is going on now, I just need some time to work out the fix. That was very helpful, thank you again! Let me know if you discover anything else in the meantime.
This time the 502 error occurred before the authentication process. (which means that you will get a 502 error immediately while accessing the website)
You can try accessing http://test.hez2010.com
And WAC was directly hosted on https://test.hez2010.com:1080
A little more info… I found this in the HTTP/2 spec:
Alternatively, a server might use an error (Section 5.4) of type HTTP_1_1_REQUIRED to request the client use a protocol that supports renegotiation.
It seems strange to me that a server would advertise that it supports HTTP/2 if it just wants you to use HTTP/1.1.
Er… anyway… I bet if you remove both that NextProtos line I suggested above and the ConfigureTransport line, it will work for you too. Please let me know!
After commenting out the http2.ConfigureTransport() and NextProtos lines, I can open the Windows Admin Center.
However it stuck at the process “Bootstrapping the application” and there were many 400 errors in network requests. (I think it may relate to authentication)
@hez2010 For this one, you’ll need to look at the logs in WAC and see why it thinks the requests are bad, I think, because unfortunately the responses don’t contain any information…
I’ve been adding debug logs to the reverse proxy since this is a prime time to do that!
And actually, the upstream appears to be returning a 200 OK. Let me investigate further. Scratch that, I didn’t follow the log line correctly… and it is returning a 400 from upstream.
Edit: Okay, so, I don’t know why this is happening. We’ll need WAC to tell us why the requests are bad.
What config do you use on nginx or whatever other proxy you used before that works?
@hez2010 One more idea. By accessing directly, the only difference I can see between requests that succeed on the direct access and the ones that fail with the proxy are the absence of the Cookie request headers.
In my debug logs, the upstream replies with headers like "Set-Cookie": ["XSRF-TOKEN=66f5a97c-8ac9-47cd-8115-b7d3c49ec64e; path=/; secure"]} but I do not see these cookies being set, perhaps because the browser sees that the client is on an HTTP page.
This is just a hunch, and doesn’t explain why some requests fail and others succeed.
Still, you might try this on an HTTPS page and see if it works.
(Edit: I’m now almost certain that is at least one problem, if not the problem. If you go to the Application tab of the web inspector, then go to Storage → Cookies, you will see that no XSRF-TOKEN cookie is set on the proxied HTTP page – probably because it is a secure context cookie only – but it is set on the direct access HTTPS page.)
Dang. I hoped I had it. I cheated a bit and forced the cookie to set on HTTP pages by adding this to the reverse proxy config:
It successfully manipulated the response header so that the cookie was set and is being included on the proxied requests, BUT… still getting 400s back.
What is your working proxy config for whatever you’re using now?
NTLM appears to violate HTTP’s convention of “stateless” by using HTTP requests to authenticate underlying connections. So, once a connection is authenticated, all requests using that connection have to be aware that the underlying connection is authenticated and set headers accordingly: authentication takes multiple round-trips as well, so Go’s connection pooling doesn’t guarantee that requests to finish authentication happen on the same connection. (Keep-Alive is also required.)
NGINX does not guarantee this either, but it probably does connection pooling very differently in a way that happens to work for you. Based on the comments and what I am reading about nginx’s additional ntlm directive, it sounds like your current config is just working by sheer luck!
Docs for proxying NTLM are sparse, but the best help I have found is from nginx’s docs:
Allows proxying requests with NTLM Authentication. The upstream connection is bound to the client connection once the client sends a request with the “Authorization” header field value starting with “ Negotiate ” or “ NTLM ”. Further client requests will be proxied through the same upstream connection, keeping the authentication context.
In order for NTLM authentication to work, it is necessary to enable keepalive connections to upstream servers. The proxy_http_version directive should be set to “ 1.1 ” and the “Connection” header field should be cleared.
So, we basically need to implement this behavior into Caddy’s proxy, along with a config property to enable it.