Basicauth causes site to return 301 or 308

Help
1

1. Caddy version (caddy version): 2.2.1

2. How I run Caddy:

As a Windows service set up by nssm, with the single parameter “run”. The Caddyfile is in the same directory alongside. The user has all necessary permissions

a. System environment:

Windows server 2019

b. Command:

As stated above - a Windows service with the command “caddy run”

c. Service/unit/compose file:

?

d. My complete Caddyfile or JSON config:

####################################################################################
# Top Matter
####################################################################################

{
	email pwh@cassland.org
}

####################################################################################
# Sites
####################################################################################

thrall.cassland.org {
	root * ..\cassland.org\html

	basicauth /CherwellSingers/* {
		cherwell JDJhJDE0JHBoRjNLQUFJTjNoWTMwSC80eVJBZXVFNHg0cnBRZGowNEZBSTJITFh6eURPLlZxbHVDdzA2
	}
	basicauth /Nic/* {
		NBH JDJhJDE0JHo5WGN5eGZ6SkNiRlEwYmtaZjlMMWVMeE1rVWIwZjNCNGhDdXJvRGRjVWRxQ1I5V3ozcWpH
	}
	basicauth /scores2/* {
		pauls JDJhJDE0JGZaLjhRbHVhUWpZZTVMZjhINHNKTGVpTFcyVEpMNDRZMWxTWXl2aEVJejYxREJVOXIzTVlD
	}

    @browsedirs {
    	path /Album*
	    path /images*
    	path /sounds*
    	path /Nic*
    	path /public*scores2
    	path /scores*
    	path /Varicam*
    	path /TascamMod*
    }
    file_server @browsedirs browse
	file_server
	encode gzip
	
	uri /images/QCavi.jpg replace .jpg .php
	uri /images/FireDog.jpg replace .jpg .php
	
	php_fastcgi localhost:9128 #(php74nts)

	log {
		output file .\Logs\CLaccess.log
		format single_field common_log
	}

    handle_errors {
        @404 {
            expression {http.error.status_code} == '404'
        }
        rewrite @404 /404.html
#       rewrite @404 /New404.html
   }
	
#	handle_errors {
#		rewrite * /{http.error.status_code}
#		reverse_proxy https://http.cat
#	}

}

####################################################################################

3. The problem I’m having:

Normal usage without authentication (i.e. with the basicauth directives commented out) works fine, whether serving files, browsing directories, or calling PHP.

However, if I include the basicauth directives and try to access the directories concerned, I get a blank page without a login request. For the request which is browsed the response is 301, and for the request with an index.html the response is 308.

4. Error messages and/or full log output:

2a02:8010:f01e:1:c153:d394:3126:9e3b - - [11/Nov/2020:20:08:31 +0000] "GET /CherwellSingers HTTP/2.0" 308 53
2a02:8010:f01e:1:c153:d394:3126:9e3b - - [11/Nov/2020:20:08:31 +0000] "GET /CherwellSingers/ HTTP/2.0" 0 0
[...]
2a02:8010:f01e:1:c153:d394:3126:9e3b - - [11/Nov/2020:20:42:07 +0000] "GET /Nic HTTP/2.0" 301 40
2a02:8010:f01e:1:c153:d394:3126:9e3b - - [11/Nov/2020:20:42:07 +0000] "GET /Nic/ HTTP/2.0" 0 0

5. What I already tried:

I have remade the basicauth directives and re-encoded the password hashes using Caddy as recommended.
I moved the basicauth directives from following the fileserver directives to preceding them.
Beyond this, I am at a loss what there is to try; this is a copy of a long-standing Caddy v1 site with the necessary Caddyfile changes, so I know the actual website files are good.

6. Links to relevant resources:

URLs as in the caddyfile

Thanks for any hints…
Paul

2

Hmm, well Caddy is serving a redirect from /Nic to /Nic/, this is coming from file_server's path canonicalization logic (it will redirect if the requested path is actually a directory, so that the path has a trailing / for directories – important for browse to work correctly)

What if you change the basicauth matcher to /Nic*, does it work any better?

You can add debug to your global options (beside the email option) to get more information logged to stdout. Make sure to configure nssm to write stdout and stderr to file!

3

Changing the matchers to remove the / didn’t help.

Running manually (rather than changing the service) and redirecting the outputs netted the following.
Nothing on stdout;
Just this on stderr (after the routing setup):

{"level":"info","ts":1605178214.3277133,"logger":"http.log.access.log0","msg":"handled request","request":{"remote_addr":"[2a02:8010:f01e:1:ec53:7e73:a978:26f6]:8789","proto":"HTTP/2.0","method":"GET","host":"thrall.cassland.org","uri":"/Nic","headers":{"Sec-Fetch-Site":["none"],"Accept-Encoding":["gzip, deflate, br"],"Accept-Language":["en,en-GB;q=0.9"],"Upgrade-Insecure-Requests":["1"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36 OPR/72.0.3815.320"],"Sec-Fetch-Mode":["navigate"],"Sec-Fetch-User":["?1"],"Sec-Fetch-Dest":["document"],"Cookie":["_ga=GA1.2.224991355.1601226547"],"Dnt":["1"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","proto_mutual":true,"server_name":"thrall.cassland.org"}},"common_log":"2a02:8010:f01e:1:ec53:7e73:a978:26f6 - - [12/Nov/2020:10:50:14 +0000] \"GET /Nic HTTP/2.0\" 0 0","duration":0,"size":0,"status":0,"resp_headers":{"Server":["Caddy"],"Www-Authenticate":["Basic realm=\"restricted\""]}}
{"level":"info","ts":1605178239.3592691,"logger":"http.log.access.log0","msg":"handled request","request":{"remote_addr":"[2a02:8010:f01e:1:ec53:7e73:a978:26f6]:8789","proto":"HTTP/2.0","method":"GET","host":"thrall.cassland.org","uri":"/CherwellSingers/","headers":{"Cookie":["_ga=GA1.2.224991355.1601226547"],"Dnt":["1"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36 OPR/72.0.3815.320"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"],"Sec-Fetch-Mode":["navigate"],"Accept-Language":["en,en-GB;q=0.9"],"Upgrade-Insecure-Requests":["1"],"Sec-Fetch-Site":["none"],"Sec-Fetch-User":["?1"],"Sec-Fetch-Dest":["document"],"Accept-Encoding":["gzip, deflate, br"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","proto_mutual":true,"server_name":"thrall.cassland.org"}},"common_log":"2a02:8010:f01e:1:ec53:7e73:a978:26f6 - - [12/Nov/2020:10:50:39 +0000] \"GET /CherwellSingers/ HTTP/2.0\" 0 0","duration":0.0061185,"size":0,"status":0,"resp_headers":{"Server":["Caddy"],"Www-Authenticate":["Basic realm=\"restricted\""]}}

I must admit, I’d expected something more obvious from the debug…

Paul

4

With further searching I found:
https://github.com/caddyserver/caddy/issues/3239

Applying the workaround in there seems to have “fixed” it, for both cases (browse and index.html).

The case is not quite the same as in that old report, because I’m not even getting the prompt for authorisation in the first place, but maybe this information will help find the proper solution.

Paul

5

Yeah, that’s strange. It seems like you’re getting a response with status 0 which doesn’t really make sense.

Could you try making a request with curl -v to see the full response you get?

If you don’t have curl installed on windows, you can use https://scoop.sh/ to install it.

6

This is without the workaround, of course:


C:\Users\paul.CASS>curl -v https://thrall.cassland.org/scores2
*   Trying 2a02:8010:f01e::73...
* TCP_NODELAY set
* Connected to thrall.cassland.org (2a02:8010:f01e::73) port 443 (#0)
* schannel: SSL/TLS connection with thrall.cassland.org port 443 (step 1/3)
* schannel: checking server certificate revocation
* schannel: sending initial handshake data: sending 190 bytes...
* schannel: sent initial handshake data: sent 190 bytes
* schannel: SSL/TLS connection with thrall.cassland.org port 443 (step 2/3)
* schannel: failed to receive handshake, need more data
* schannel: SSL/TLS connection with thrall.cassland.org port 443 (step 2/3)
* schannel: encrypted data got 3103
* schannel: encrypted data buffer: offset 3103 length 4096
* schannel: sending next handshake data: sending 93 bytes...
* schannel: SSL/TLS connection with thrall.cassland.org port 443 (step 2/3)
* schannel: encrypted data got 186
* schannel: encrypted data buffer: offset 186 length 4096
* schannel: SSL/TLS handshake complete
* schannel: SSL/TLS connection with thrall.cassland.org port 443 (step 3/3)
* schannel: stored credential handle in session cache
> GET /scores2 HTTP/1.1
> Host: thrall.cassland.org
> User-Agent: curl/7.55.1
> Accept: */*
>
* schannel: client wants to read 102400 bytes
* schannel: encdata_buffer resized 103424
* schannel: encrypted data buffer: offset 0 length 103424
* schannel: encrypted data got 119
* schannel: encrypted data buffer: offset 119 length 103424
* schannel: decrypted data length: 90
* schannel: decrypted data added: 90
* schannel: decrypted data cached: offset 90 length 102400
* schannel: encrypted data buffer: offset 0 length 103424
* schannel: decrypted data buffer: offset 90 length 102400
* schannel: schannel_recv cleanup
* schannel: decrypted data returned 90
* schannel: decrypted data buffer: offset 0 length 102400
< HTTP/1.1 200 OK
< Server: Caddy
< Date: Thu, 12 Nov 2020 16:20:23 GMT
< Content-Length: 0
<
* Connection #0 to host thrall.cassland.org left intact

C:\Users\paul.CASS>

Paul

7

Now this is getting a little confusing - In Edge and Firefox (which have not been logged in to the site) I am now getting the login popup for the /CherwellSingers path (which has an index file), but still not for the other paths (which are browsed). In my usual Opera browser I am still logged in following the test of the workaround, so I can’t easily test again for a while longer.

I will continue trying more things.

Later - it gets worse. If I click the x to close the login box without filling it in, the site is displayed! But not completely (internal links in the page are not shown) - and going to subpages brings up the login again, and again cancelling it gets me in…

Paul

8

OK, I’m getting much closer.

First, this is not a Windows issue - I have built a Linux server and (with much trouble, caused by SELinux) got Caddy running on it.

I then built up my Caddyfile bit by bit. And the addition which stopped basicauth working was my handle_errors directive.

If I include:

	handle_errors {
		@404 {
			expression {http.error.status_code} == "404"
		}
		rewrite @404 /404.html
		file_server
	}

then I get the blank page with no authorisation popup, as I’ve reported above, and the access log shows that the response was “0”.

If I include

	handle_errors {
		rewrite * /{http.error.status_code}
		reverse_proxy https://http.cat
	}

then the log contains a stream of errors:

192.168.1.74 - - [16/Nov/2020:12:31:56 +0000] "GET /404/ghost HTTP/2.0" 301 178
192.168.1.74 - - [16/Nov/2020:12:31:56 +0000] "GET /404/ghost HTTP/2.0" 301 178
192.168.1.74 - - [16/Nov/2020:12:31:56 +0000] "GET /404/ghost HTTP/2.0" 301 178
etc

and Firefox reports: “The page isn’t redirecting properly”. This loop occurs both for accessing a basicauth-protected page and for an actual 404.

As a reminder of the context, this is my complete Caddyfile:

####################################################################################
# Top Matter
####################################################################################

{
#	debug
	email pwh@cassland.org
}

####################################################################################
# Sites
####################################################################################

#cassland.org, www.cassland.org {
#thrall.cassland.org {
pen2.cassland.org {

#	root * ..\cassland.org\html
	root * /var/www/html/cassland_org
	
	uri /images/QCavi.jpg replace .jpg .php
	uri /images/FireDog.jpg replace .jpg .php

	basicauth /CherwellSingers/* {
		cherwell JDJhJDE0JHBoRjNLQUFJTjNoWTMwSC80eVJBZXVFNHg0cnBRZGowNEZBSTJITFh6eURPLlZxbHVDdzA2 # xxx
	}
	basicauth /Nic/* {
		NBH JDJhJDE0JHo5WGN5eGZ6SkNiRlEwYmtaZjlMMWVMeE1rVWIwZjNCNGhDdXJvRGRjVWRxQ1I5V3ozcWpH # yyy
	}
	basicauth /scores2/* {
		pauls JDJhJDE0JGZaLjhRbHVhUWpZZTVMZjhINHNKTGVpTFcyVEpMNDRZMWxTWXl2aEVJejYxREJVOXIzTVlD # zzzx	}

	encode gzip
	
#	php_fastcgi localhost:9128 #(php7nts)
	php_fastcgi localhost:9000 #(php-fpm)

	@browsedirs {
		path /Album
		path /Album/*
		path /images
		path /images/*
		path /sounds
		path /sounds/*
		path /Nic
		path /Nic/*
		path /scores
		path /scores/*
		path /scores2
		path /scores2/*
		path /Varicam
		path /Varicam/*
		path /TascamMod
		path /TascamMod/*
	}
	file_server @browsedirs browse
	file_server

	handle_errors {
#		Attempt to make basicauth work in Windows:
#		@401 {
#			expression {http.error.status_code} == "401"
#		}
#		header @401 {
#			WWW-Authenticate "Basic realm=\"restricted\""
#		}
#		respond @401 401
        
#		@404 {
#			expression {http.error.status_code} == "404"
#		}
#		rewrite @404 /404.html
#		rewrite @404 /New404.html
#		file_server

#		Use cat server messages:
		rewrite * /{http.error.status_code}
		reverse_proxy https://http.cat
	}

	log {
#		output file .\Logs\CLaccess.log
		output file /var/log/caddy/CLaccess.log
		format single_field common_log
	}
}

I now see this has been reported before (at least, the reverse_proxy version), with what seems to me an inconclusive response: this report here. But note that in my original case the reverse_proxy shouldn’t be invoked; the problem seems more to be that handle_errors is picking up a status set by the basicauth and preventing it being passed on to the client correctly. Equally, the handle_errors directive works until a basicauth is added to the Caddyfile - this stops the handle_errors working, even when the basicauth is not invoked.

Paul

9

Well, I don’t see any interest in the fact that it seems that having both basicauth and handle_errors in the same Caddyfile (or at least the same virtual server) results in both failing to work. But to me it’s the kind of thing that makes Caddy 2 feel not ready for prime time yet.

However, I’ve got an acceptable temporary workaround for my example by using:

try_files {path} {path}/ 404.html

instead of my handle_errors - but I would prefer to be able to use it so that I can handle other errors as well…

Of course, if someone can demonstrate how the directives can be made to not conflict, then I’d be all ears!

Paul

10

Sorry. We work hard every day to continuously improve the free software you benefit from. It very much is ready for “prime time.”

I’m honestly having a hard time reproducing the behavior you’re describing.

Here’s my own Caddyfile based on your last update:

{
	debug
}

localhost

# Using the Caddy site as a test fixture
root * /home/matt/Dev/caddyserver/website/src

@browsedirs {
	path /resources
	path /resources/*

	# I don't have these directories
	# path /Album
	# path /Album/*
	# path /images
	# path /images/*
	# path /sounds
	# path /sounds/*
	# path /Nic
	# path /Nic/*
	# path /scores
	# path /scores/*
	# path /scores2
	# path /scores2/*
	# path /Varicam
	# path /Varicam/*
	# path /TascamMod
	# path /TascamMod/*
}
file_server @browsedirs browse
file_server

basicauth /resources/* {
	Bob JDJhJDEwJEVCNmdaNEg2Ti5iejRMYkF3MFZhZ3VtV3E1SzBWZEZ5Q3VWc0tzOEJwZE9TaFlZdEVkZDhX
}

php_fastcgi localhost:9000

handle_errors {
#	Attempt to make basicauth work in Windows:
#	@401 {
#		expression {http.error.status_code} == "401"
#	}
#	header @401 {
#		WWW-Authenticate "Basic realm=\"restricted\""
#	}
#	respond @401 401

#	@404 {
#		expression {http.error.status_code} == "404"
#	}
#	rewrite @404 /404.html
#	rewrite @404 /New404.html
#	file_server

#	Use cat server messages:
	rewrite * /{http.error.status_code}
	reverse_proxy https://http.cat
}

And as far as I can tell, basicauth is working fine in conjunction with handle_errors.

Can you give me the exact curl command that you’re seeing problems with, along with its full output, and then what you expect the output to be instead? I’m trying to replicate it 1:1 but it’s hard with the information provided here (also this thread is getting quite complex and I’ve had a hard time following it).

1 Like
11

Thanks for the reassurance that it can work. I built up my last test (not shown in this thread) by commenting out my whole Caddyfile and then adding directives one by one until I got a failure. Initially I only got a failure when I added the handle_errors (which was later than the basicauth). At that point I removed everything unnecessary other than those (essentially the PHP handling and all but one basicauth), and still had failures. I then removed the basicauth and the handle_errors worked again.

Still, I can see how this thread is confusing now, so I’ll not add to it, but make a new one when I’ve gone through the simplest possible combinations to find where the failure in my setup is creeping in.

Paul

1 Like
12

This topic was automatically closed after 30 days. New replies are no longer allowed.