Caddy NTLM reverse proxy constant 401 with Windows Admin Center

Help
1

1. The problem I’m having:

I’m trying to create a simple reverse proxy from a HTTP caddy server to the default HTTPS URL for the latest version of Windows Admin Center (Version 2311). I’m using the NTLM module to achieve this. When I navigate to the HTTP caddy page, I get the login as usual, however no matter how many times I input the correct username and password, I keep getting prompted to login over and over again with 401 errors. Using the same credentials on the normal HTTPS site works instantly of course.

2. Error messages and/or full log output:

DBG ts=1708748002.0027668 logger=http.handlers.reverse_proxy msg=selected upstream dial=192.168.1.20:9083 total_upstreams=1
DBG ts=1708748002.0076027 logger=http.handlers.reverse_proxy msg=upstream roundtrip upstream=192.168.1.20:9083 duration=0.004784956 request={"remote_ip":"100.0.0.1","remote_port":"56364","proto":"HTTP/1.1","method":"GET","host":"ubuntu-server:9082","uri":"/","headers":{"Pragma":["no-cache"],"Upgrade-Insecure-Requests":["1"],"Accept-Encoding":["gzip, deflate"],"Accept-Language":["en-US,en;q=0.9"],"Cache-Control":["no-cache"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36 Edg/121.0.0.0"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7"],"X-Forwarded-For":["100.0.0.1"],"X-Forwarded-Proto":["http"],"X-Forwarded-Host":["ubuntu-server:9082"]}} headers={"Content-Length":["0"],"Www-Authenticate":["Negotiate","NTLM"],"Date":["Sat, 24 Feb 2024 04:13:21 GMT"]} status=401
DBG ts=1708748016.0307705 logger=http.handlers.reverse_proxy msg=selected upstream dial=192.168.1.20:9083 total_upstreams=1
DBG ts=1708748016.0316873 logger=http.handlers.reverse_proxy msg=upstream roundtrip upstream=192.168.1.20:9083 duration=0.000864192 request={"remote_ip":"100.0.0.1","remote_port":"56364","proto":"HTTP/1.1","method":"GET","host":"ubuntu-server:9082","uri":"/","headers":{"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7"],"X-Forwarded-For":["100.0.0.1"],"Cache-Control":["max-age=0"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36 Edg/121.0.0.0"],"X-Forwarded-Proto":["http"],"Accept-Encoding":["gzip, deflate"],"Accept-Language":["en-US,en;q=0.9"],"Upgrade-Insecure-Requests":["1"],"X-Forwarded-Host":["ubuntu-server:9082"]}} headers={"Date":["Sat, 24 Feb 2024 04:13:36 GMT"],"Content-Length":["0"],"Www-Authenticate":["Negotiate","NTLM"]} status=401
DBG ts=1708748018.0792294 logger=http.handlers.reverse_proxy msg=selected upstream dial=192.168.1.20:9083 total_upstreams=1
DBG ts=1708748018.0844784 logger=http.handlers.reverse_proxy msg=upstream roundtrip upstream=192.168.1.20:9083 duration=0.005182853 request={"remote_ip":"100.0.0.1","remote_port":"56364","proto":"HTTP/1.1","method":"GET","host":"ubuntu-server:9082","uri":"/","headers":{"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36 Edg/121.0.0.0"],"Authorization":[],"Upgrade-Insecure-Requests":["1"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7"],"X-Forwarded-Host":["ubuntu-server:9082"],"X-Forwarded-For":["100.0.0.1"],"Cache-Control":["max-age=0"],"Accept-Encoding":["gzip, deflate"],"Accept-Language":["en-US,en;q=0.9"],"X-Forwarded-Proto":["http"]}} headers={"Www-Authenticate":["Negotiate <redacted>"],"Date":["Sat, 24 Feb 2024 04:13:38 GMT"],"Content-Length":["0"]} status=401
DBG ts=1708748018.08581 logger=http.handlers.reverse_proxy msg=selected upstream dial=192.168.1.20:9083 total_upstreams=1
DBG ts=1708748018.0876324 logger=http.handlers.reverse_proxy msg=upstream roundtrip upstream=192.168.1.20:9083 duration=0.001771184 request={"remote_ip":"100.0.0.1","remote_port":"56364","proto":"HTTP/1.1","method":"GET","host":"ubuntu-server:9082","uri":"/","headers":{"Upgrade-Insecure-Requests":["1"],"X-Forwarded-Host":["ubuntu-server:9082"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36 Edg/121.0.0.0"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7"],"X-Forwarded-For":["100.0.0.1"],"Cache-Control":["max-age=0"],"Authorization":[],"X-Forwarded-Proto":["http"],"Accept-Encoding":["gzip, deflate"],"Accept-Language":["en-US,en;q=0.9"]}} headers={"Content-Length":["0"],"Www-Authenticate":["Negotiate","NTLM"],"Date":["Sat, 24 Feb 2024 04:13:38 GMT"]} status=401

3. Caddy version:

Version 2.7.6

4. How I installed and ran Caddy:

Using Docker with instructions to build NTLM module

a. System environment:

Official docker container using builder

b. Command:

None, just what the normal docker build command runs.

c. Service/unit/compose file:

N/A

d. My complete Caddy config:

{
    debug
}

http://:80 {

    reverse_proxy {
        to  https://192.168.1.20:9083

        transport http_ntlm {
            tls_insecure_skip_verify
        }
    } 

    encode zstd gzip
}

5. Links to relevant resources:

2

Hmm, I’m not sure. I wrote the NTLM module a couple years ago mostly for fun, and someone else had an instance I could use to test it against. I haven’t really touched it since. So I know it works, at least under some circumstances, and haven’t had a problem report since then.

Isn’t Microsoft deprecating NTLM soon?

I’d be willing to work on this but probably would need a sponsorship to fund it, since I have a lot on my plate right now. Or anyone else could take a stab at it.

3

Completely understand and thank you for your reply!

Just wanted to give an update as I think I got it working for the most part! The issue was a weird mix of cached cookies from prior sessions and with cookies needing to be massaged similar to your older post regarding secure cookies. That helped a lot thanks!

Let me walk through what I did. First I changed my Caddyfile to the JSON format since I wasn’t able to figure out the proper regex syntax to get the same substring functionality working with cookie replacements.

Here is my config:

{
    "apps": {
        "http": {
            "servers": {
                "windows-admin-center": {
                    "listen": [
                        ":80"
                    ],
                    "routes": [
                        {
                            "handle": [
                                {
                                    "handler": "reverse_proxy",
                                    "transport": {
                                        "protocol": "http_ntlm",
                                        "tls": {
                                            "insecure_skip_verify": true
                                        }
                                    },
                                    "headers": {
                                        "response": {
                                            "replace": {
                                                "Set-Cookie": [
                                                    {
                                                        "search": "secure;",
                                                        "replace": ""
                                                    },
                                                    {
                                                        "search": "SameSite=None",
                                                        "replace": "SameSite=Lax"
                                                    }
                                                ]
                                            },
                                            "set": {
                                                "Content-Security-Policy": [
                                                    "frame-ancestors *"
                                                ]
                                            }
                                        }
                                    },
                                    "upstreams": [
                                        {
                                            "dial": "192.168.1.20:9083"
                                        }
                                    ]
                                }
                            ]
                        }
                    ]
                }
            }
        }
    }
}

What I had to do was to remove the secure; bit from Set-Cookie like you had suggested in your post first.

Then I realized that the browser would still complain about SameSite so I changed the cookie to Lax there.

After that the page would still not load claiming the cookie was secure so it can’t change it and what not, and I read that browsers commonly like to use cached cookies. I went ahead and cleared out all the cookies and then it started to load as expected!

I had a few questions I wanted to ask as well as I’ve been working through this.

  1. You’ll notice that I have another cookie set to relax the rules for Content-Security-Policy. I did that so that I could enable iframe access to the proxied page, since my goal was to have Windows Admin Center show up in my Home Assistant Dashboard. It shows up now with that cookie set, but WAC first shows a website saying that a Potential Security Issue Detected. Luckily it does let me continue and it loads as expected, but I don’t suppose there’s anything further I can do in the proxy layer to avoid that message showing up?

  2. I haven’t looked into how to do this yet, but it would be awesome if I could store the credentials for NTLM within the reverse proxy layer so that I would not need to input them every time the cookies expire. Would you have any guidance on how to do that?

Thanks a lot for all your help!

1 Like
4

There’s an example right at the bottom of the page here header (Caddyfile directive) — Caddy Documentation

header >Set-Cookie SameSite=None SameSite=Lax
header >Set-Cookie secure ""
5

Thank you for the response!

I’m not sure why what you suggested doesn’t work in the same way as the JSON configuration. In the JSON configuration, it is replacing all instances of the substring in the headers as expected.

In the format that you provided for the Caddyfile, it seems to just overwrite the header with the first parameter. I tried it quickly just now using header >Set-Cookie secure "" and even tried in the reverse_proxy directive using header_down Set-Cookie secure "", and all it seems to do is set the cookie to secure not actually replace it. Any ideas why they behave so differently?

6

You can adapt the Caddyfile to JSON with caddy adapt -p to see the difference.

The > marker sets defer so it causes the header to be manipulated on the way out, i.e. after another header was set.

I just noticed the issue, we check for "" (empty string) for the replacement, and if empty string we don’t use it as a replacement and make it set instead. So we don’t have a way to replace with “nothing” right now. But you could replace it with " " (single space) which should still produce a valid Set-Cookie value. You might need to replace secure; actually? :thinking:

I’ll try to fix that, it should be possible to replace with empty string.

7

Okay I have a fix for that empty string case:

8

Thank you for the quick response!

Seems like the empty string was the problem! I did add the a single space " " and it seems to work now as expected.

Looks like we did find a bug so I’m glad that this was caught and will get fixed when your change is merged into the release!

1 Like
9

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.