File_server basics

This Wiki is tailored to the “new guy” that doesn’t have any/much experience with file serving, or maybe you’re new to using a server all together. These are just some basic examples to get started with using Caddy to serve files. This is intended to add some extra examples to the Caddy Documentation.

As a new guy who has barley entered the matrix of hosting my own files and using a server to host my systems programs I could have used some more examples. Seeing something as a whole and not just a piece of the Caddyfile can sometimes really help a person understand how to use the directives. I have seen some of these same questions pop up here on the forum so maybe this Wiki will help someone out.

Please add to this and/or streamline some of these examples if there’s a better method to accomplish the same goal. I know there much more to file serving than this, but wanted to start this with some simple basics.

The simplest form of serving files:

domain.com {
	root * /path/to/directory
	file_server browse
}

This will serve all files in the root directory. If no index fie is being used browse is required to list the files.


To restrict access to certain files or directories you can use a named matcher with the not and path directives:

domain.com {
	root * /path/to/directory
	@NoAccess {
		not path /folder0/ /folder1/subfolder1/*
	}
	file_server @NoAccess browse
}

In the above example there are two paths listed after not path. The first path denies access to the path/to/directory/folder0/ HOWEVER subfolders within folder 0 can still be accessed. Example: If you type domain.com/folder0/user1 you will be able to access the user1 folder although you denied access to folder0.

Adding a * after the path will stop access to all subfolders and files.

not path can also be written line by line if you want your Caddyfile to look like a list.

@NoAccess {
	not path /folder0/
	not path /folder1/subfolder1/*
	not path /folder2/subfolder2/file.txt
	not path */subfolder4/*
}

You can take this a step further and hide files and folders using the hide module:

domain.com {
	root * /path/to/directory
	@NoAccess {
		not path /folder0/ /folder1/subfolder1/*
	}
	file_server @NoAccess browse {
		hide *.txt new file2.odp
	}
}

The hide module cannot be in list form. Only the last ‘hide’ in the list would be hidden. In this case you will have to list everything in line. To hide a file you are required to put the extension. In the above example I have hidden all text files, a folder called new, and a file called file2.odp.


If you are looking for a quick way to access a subdirectory that is buried deep you can use the redirect directive and strip the prefix. Thanks @francislavoie for this one.

domain.com {
	root * /path/to/directory
	@NoAccess {
		not path /folder0/ /folder1/subfolder1/*
	}
	file_server @NoAccess browse {
		hide *.txt
	}
	redir /user1 /user1/
	handle /user1/* {
		uri strip_prefix /user1
		root * /path/to/directory/users/allusers/user1
	}
	redir /user2 /user2/
	handle /user2/* {
		uri strip_prefix /user2
		root * /path/to/directory/users/allusers/user2
	}
}

In the above example we have redirected /user1. We then used a handle to strip the prefix and set the root path. Now you can type domain.com/user1 into a browser and you will be served files from /path/to/directory/users/allusers/user1.

For Caddy v2.1 you can use the new handle_path directive time accomplish the same thing without having to strip the prefix as this directive incorporates that function.

redir /user1 /user1/
handle_path /user1/* {
    root * /path/to/directory/users/allusers/user1
}

You can add password protection to these folders also by using the basicauth directive.

domain.com {
	root * /path/to/directory
	@NoAccess {
		not path /folder0/ /folder1/subfolder1/*
	}
	file_server @NoAccess browse {
		hide *.txt
	}
	redir /user1 /user1/
	handle /user1/* {
		uri strip_prefix /user1
		root * /path/to/directory/users/allusers/user1
		basicauth {
			<username1> <hashed_password_base64>
		}
	}
	redir /user2 /user2/
	handle /user2/* {
		uri strip_prefix /user2
		root * /path/to/directory/users/allusers/user2
		basicauth {
			<username2> <hashed_password_base64>
		}
	}
	basicauth / {
		<username3> <hashed_password_base64>
	}
}

In the above example the basicauth outside of the handles provides security to the main root. This could be someone like an admin that needs access to the main root and all subfolders.

You may also choose to place the basicauth for user 1 and user2 outside of the handle if you prefer to keep all of the basicauth’s grouped together. In that instance you would then need to add the folder after basicauth

basicauth /user1/* {
	<username1> <hashed_password_base64>
}
basicauth /user2/* {
	<username2> <hashed_password_base64>
}
basicauth / {
	<username3> <hashed_password_base64>
}

Like I said before this is very basic stuff to get started with, but hopefully it will help someone who is just getting started with networking/web servers. Like @matt would say “Caddy 2 is no child’s toy”…but you have to start somewhere.

10 Likes

A post was split to a new topic: Help with file server and hide

I was searching and about to write a message asking for help on this and then I saw the file_server directive comes with the hide option.

The examples above were helpful, particularly the example with hide shown with more than one value (since the docs only show one, it wasn’t clear how to add more than one, and I ended up starting to try that by using a comma but then noticed only my last entry was being applied so it didn’t click until I saw the example above).

In my case, I just wanted to make sure that any files/directories starting with a period were hidden automatically since those are all typically the types of files that you wouldn’t want a server to serve (e.g. a .htaccess file that’s been copied from an older Apache setup, accidental .project, .DS_Store, .metadata folders, .git folders, etc.), and then I decided to also exclude any files/directories that start with an underscore as well and you can see the result in the snippet below:

(add_static_file_serving_features) {

	# Allow accessing files without requiring .html:
	try_files {path} {path}.html
	
	# Enable Static File Server and Directory Browsing:
	file_server {
		browse
		hide .* _*
	}
	
	# Enable templating functionality:
	templates
	
	# Enable Compression for Output:
	encode zstd gzip
	
	handle_errors {
		respond "<pre>{http.error.status_code} {http.error.status_text}</pre>"
	}
}
1 Like

3 posts were split to a new topic: File server hiding files

Hi,
Excellent article. One question regarding hiding files and directories using hide. Is it possible to hide items from from the generated list, but still allow direct access to them . For example I do not particularly want to show robots.txt, .well-known, etc, so I hide them, but now those can’t be downloaded instead. Is there a way around it?

There’s no simple way. But what you can do is add a second file_server which doesn’t have hide or browse which you use a path matcher with to handle requests to those specific files you want to allow being served without being hidden.

@hidden path /robots.txt /.well-known/*
file_server @hidden

file_server browse {
	hide robots.txt .well-known
}
3 Likes