Help hiding only one named folder

jfirestorm44 · May 30, 2020, 1:15pm

1. Caddy version (`caddy version`):

v2.0.0 h1:pQSaIJGFluFvu8KDGDODV8u4/QRED/OPyIR+MWYYse8=

2. How I run Caddy:

caddy.service file (not API)
or
caddy start/stop/reload

a. System environment:

Raspberry Pi (Raspbian Buster Lite)

b. Command:

caddy start --config <path to file>

caddy reload --config <path to file>

c. Service/unit/compose file:

N/A

d. My complete Caddyfile or JSON config:

mydomain.duckdns.org {
	log {
		output file /var/log/caddy/portainer.log
		format console
	}
	@internal {
		remote_ip 192.168.0.0/16
	}
	handle @internal {
		reverse_proxy 127.0.0.1:9000
	}
	respond 403
}
jellyfin.mydomain.duckdns.org {
	log {
		output file /var/log/caddy/jellyfin.log
		format console
	}
	reverse_proxy 127.0.0.1:8096
}
omv.mydomain.duckdns.org {
        log {
		output file /var/log/caddy/omv.log
		format console
	}
	reverse_proxy 127.0.0.1:81
}
nas.mydomain.duckdns.org {
	log {
		output file /var/log/caddy/nas.log
		format console
	}
	root * /srv/dev-disk-by-label-HomeDrive/
	@noAccess {
		not path */RESTRICTED/*
	}
	file_server @noAccess browse {
                 hide RESTRICTED Games "TV SHOWS"
}
	encode gzip
	basicauth {
		username (hashbrown64)
	}
	@iPhone {
		not remote_ip xxx.xxx.xxx.xxx
	}
	respond @iPhone 403
}

3. The problem I’m having:

I am trying to figure out the secret recipe for hiding files properly in Caddy v2. I understand that folders/files can be denied by using a ‘named matcher’ with ‘not path’ and calling that out in the file_server directive which I am using for my ‘RESTRICTED’ folder.

Using that method is great but any user can still see the folder (although access is denied) and I would like to have it invisible to all as well as denying permission.

I have tried many variations of using the hide module and it does work very well. My problem is I can’t figure out how to make it only hide one instance of a named folder and not all of them.

4. Error messages and/or full log output:

5. What I already tried:

As you see in my caddyfile I have one folder with not path and then I also hid that folder along with another called Games and one called “TV SHOWS”.

	@noAccess {
		not path */RESTRICTED/*
	}
	file_server @noAccess browse {
                 hide RESTRICTED Games "TV SHOWS"
}

It does exactly what I want it to do except all of my Games folders in every directory are hidden. I only want to hide one Games folder and that’s the one in …Arcade/Games

I have tried all of these:

hide RESTRICTED /srv/dev-disk-by-label-HomeDrive/Arcade/Games "TV SHOWS"
hide RESTRICTED */srv/dev-disk-by-label-HomeDrive/Arcade/Games "TV SHOWS"
hide RESTRICTED */srv/dev-disk-by-label-HomeDrive/Arcade/Games*
hide RESTRICTED */Arcade/Games "TV SHOWS"
hide RESTRICTED /Arcade/Games "TV SHOWS"
hide RESTRICTED /Arcade/Games* "TV SHOWS"
hide RESTRICTED */Arcade/Games/* "TV SHOWS"

And probably a few other things. I have also separated the hidden folders with a semi-colon which works but still doesn’t allow me to hide just one directory. i.e. hide RESTRICTED ; Games ; "TV SHOWS"

I’m sure there’s a way to get the desired results but I just can’t crack the code.

Any help would be greatly appreciated.

6. Links to relevant resources:

francislavoie · May 30, 2020, 2:48pm

The logic that does the hiding is here:

github.com

caddyserver/caddy/blob/95d9446/modules/caddyhttp/fileserver/staticfiles.go#L329-L359

    
      
          // fileHidden returns true if filename is hidden
          // according to the hide list.
          func fileHidden(filename string, hide []string) bool {
          	nameOnly := filepath.Base(filename)
          	sep := string(filepath.Separator)
          
          
	for _, h := range hide {
          		// assuming h is a glob/shell-like pattern,
          		// use it to compare the whole file path;
          		// but if there is no separator in h, then
          		// just compare against the file's name
          		compare := filename
          		if !strings.Contains(h, sep) {
          			compare = nameOnly
          		}
          
          
		hidden, err := filepath.Match(h, compare)
          		if err != nil {
          			// malformed pattern; fallback by checking prefix
          			if strings.HasPrefix(filename, h) {

This file has been truncated. show original

It uses the filepath.Match function which you can see documented here:

I’m not sure what the exact answer is for you but hopefully this points you in the right direction.

jfirestorm44 · May 30, 2020, 3:53pm

Unfortunately I don’t read code very well (at all). I have tried some of the option from the golang site you posted.

Here’s what I know.

hide * hides everything
hide *Games hides all Games folders in all directories
hide */Games hides nothing
hide *Arcade/Games hides nothing
hide */Arcade/Games hides nothing
hide Arcade?Games hides nothing
hide Arcade/Games/ hides nothing
hide */Arcade/Games/* nada
hide */Arcade*Games/* nada
hide *Arcade*Games* nada

I have tried many more variations also with no luck. I don’t really need this function right now but was more experimenting with the hide module. If I get it figured out I’ll update the post with the solution.

Thanks.

francislavoie · May 30, 2020, 4:20pm

Hmm. Could you try ./Arcade/Games? The dot at the front means “current directory” which I hope is read as relative to the root defined.

jfirestorm44 · May 30, 2020, 4:35pm

That’s a no-go also. Doesn’t hide anything.

francislavoie · June 1, 2020, 9:03pm

Okay, I confirmed that if you have a dir structure like this:

.
├── Arcade
│   └── Games
└── Caddyfile

And you make a GET /Arcade request, then only Games is passed to the fileHidden() function, which means any rule that involves the full path doesn’t work at all.

I think we would need to update the code to pass the whole path plus the filename being tested to fileHidden(). This can be tricky though. It’ll take a bit of thinking to figure out the correct solution. It might also break things for existing users.

Here’s a diff that should do what you need but I’m not sure this is the right change to make:

diff --git a/modules/caddyhttp/fileserver/browselisting.go b/modules/caddyhttp/fileserver/browselisting.go
index 9c7c4a20..d27d70f6 100644
--- a/modules/caddyhttp/fileserver/browselisting.go
+++ b/modules/caddyhttp/fileserver/browselisting.go
@@ -18,6 +18,7 @@ import (
        "net/url"
        "os"
        "path"
+       "path/filepath"
        "sort"
        "strconv"
        "strings"
@@ -38,7 +39,7 @@ func (fsrv *FileServer) directoryListing(files []os.FileInfo, canGoUp bool, urlP
        for _, f := range files {
                name := f.Name()

-               if fileHidden(name, filesToHide) {
+               if fileHidden(filepath.Join(urlPath, name)[1:], filesToHide) {
                        continue
                }

That makes it possible to do hide Arcade/Games to only hide that one Games dir.

matt · June 1, 2020, 9:42pm

To clarify, here’s how file hiding works:

If the file to hide is only a filename (i.e. has no path separator as defined by the operating system), then only filename are compared (regardless of path).
The file to hide and the candidate filepath (or filename, according to above rule) is compared using a glob pattern.
If the glob pattern is bad, it just does a naive prefix comparison: if the file to hide prefixes the candidate file, then it is considered hidden.

Hope that helps

jfirestorm44 · June 2, 2020, 1:00am

Ok now that I understand what all of that means the question is how do I make those changes? I found the old browselisting.go file on GitHub but I can’t find it on my server.

I don’t understand how I should be making that change. I installed git and I used the git diff command with the paths you have listed but it couldn’t find them so I’m assuming those are personal?

Is this a change you were saying could be made in the future or that I can make now and try it out? If so how do I change those lines?

Thanks

jfirestorm44 · June 2, 2020, 1:28am

It kind of makes sense to me. I still can’t see why it doesn’t recognize a filepath. I’ve tried it by using * and ? in place of the ‘/’ but that didn’t help. I’m guessing it has to do with the way the code is written that @francislavoie posted above. Like I said It’s not something I need right now but was just trying to become more familiar with how Caddy works.

Whitestrake · June 3, 2020, 12:26am

Are we sure this is the case?

At a glance, the fileHidden function is only ever called with filename or a derivative thereof. filename is a sanitised path join of root and suffix. root is the defined web root or . as fallback; suffix is the entire request URL path.

As far as I can tell, fileHidden is given the whole path every time. It does throw out the path and uses a filepath.Base if it can’t find any instances of a file separator, though.

@jfirestorm44,

I do note that the filepath.Match glob only catches a single path element. If you need to hide every file inside /Arcade/Games/, e.g. /Arcade/Games/foo and /Arcade/Games/bar, but NOT hide /Arcade/Games/foo/bar OR the index itself, the correct parameter should be /Arcade/Games/*.

If you want to hide /foo/Games/ and /bar/Games/ but not /foo/bar/Games/, the correct parameter should be /*/Games. As you’ve noted, using Games hides any directory called Games/ (or specifically you tried *Games, which would also hide /foo/barGames/).

The correct parameter should be /Arcade/Games. Note that this won’t actually hide any files inside of the /Arcade/Games/ directory. If you want to hide the directory and everything in it, you probably just want a responder instead; “invisible” is just a 404 response.

francislavoie · June 3, 2020, 12:38am

Yes I’m 100% sure, I compiled Caddy with log statements and the fileHidden call in browselisting.go only passes each filename in the range of files in the current directory.

Whitestrake · June 3, 2020, 12:48am

Oh, of course, you’re absolutely right.

Yeah. Your diff looks good, but if we can replicate this structure, strictly speaking:

github.com

caddyserver/caddy/blob/e18c37306448c5604bb50618c19d7e247fb8805f/modules/caddyhttp/fileserver/staticfiles.go#L113-L115

    
      
          	root := repl.ReplaceAll(fsrv.Root, ".")
          	suffix := repl.ReplaceAll(r.URL.Path, "")
          	filename := sanitizedPathJoin(root, suffix)

And then call fileHidden(filename, filesToHide), we’d be replicating most accurately the behaviour found in staticfiles.go.

We’ve got our fsrv and repl handy in directoryListing(), could just copy that logic basically straight over.

jfirestorm44 · June 3, 2020, 1:22am

Ok that makes sense. I was under thinking that the way it should work is if Directory is hidden then everything below it automatically becomes hidden also since you can’t see the top directory and therefore can’t access it.

Unfortunately ‘/Arcade/Games/’ or any variation of that doesn’t work. I also have it set up as a ‘not path’ so there’s no access to that folder when I enable that. I just wanted the addition ability to hide it and all of its contents from snoopy people who ask questions if ever needed. I’m the only one using this network right now as this is just a home project to get my feet wet in the networking realm.

Whitestrake · June 3, 2020, 1:36am

Aye. Sorry to “talk over your head” so to speak in your thread here on the forum, but that’s what @francislavoie and I have been going back-and-forth about; the logic is correct in the static file server itself (so it would never serve a hidden directory index file or a regular file if they were hidden), but seems incorrect for the directory listing functionality (i.e. what you get from browse; it seems to show them regardless unless they’re just a filename, not a full path).

What I said should be correct (but isn’t right now), likely will be correct once a PR goes through to fix things up.

system · June 29, 2020, 1:15pm

This topic was automatically closed after 30 days. New replies are no longer allowed.