File sizes of symlinks in file_server browse display size of symlink instead of size of linked file

[roan]

1. The problem I’m having:

File sizes of symlinks in file_server browse display size of symlink instead of size of linked file.

Not sure if this is an issue on my end or possibly an issue with the new (fancy ) browse template.

Example of symlinks:

Example of original file:

Symlinks created with cp -rs
This was an existing issue that seems to have been solved at some point, see: file_server browse lists size of symlink instead of size of linked file · Issue #4353 · caddyserver/caddy · GitHub

2. Error messages and/or full log output:

N.A.

3. Caddy version:

v2.7.5

4. How I installed and ran Caddy:

a. System environment:

Debian 11
Installed from official package (Install — Caddy Documentation)

b. Command:

(sudo) caddy start

d. My complete Caddy config:

Relevant part:

        handle_errors {
                respond "{err.status_code} {err.status_text}"
        }

        redir /files /files/

        handle_path /files/* {
                root * ./files
                file_server browse
        }

[francislavoie]

Can you run ls -lah in that directory? Just in case there’s something weird about the files you have. Because it works for me. Are the links actually intact? Note the color of the output of la -lah, the files would show up red if the link isn’t pointing to a file that exists.

[roan]

Certainly
Links are fully intact, can view and download them from the web browser just fine.

Based on your message I tried some different files using the same process, which worked fine for me as well.
Did some (limited) testing with filenames and path to see if it might be caused by special characters but nothing decisive for now.

(Excuse the weird image, currently on mobile)

[matt (Matt Holt)]

Huh. Thanks for the added info.

So we don’t know why those symlinks’ file size is wrong, but others are correct, i.e. we haven’t found a reproducer?

That’ll probably be the first step. Wonder if there’s something funky about some symlinks I’ve never heard about before.

[roan]

Yeah it’s certainly weird.
I’ll try to reproduce it this weekend and report back.

[Mohammed90 (Mohammed Al Sahaf)]

What if you use the ln -s command instead of cp -rs?

[roan]

That does work (as in, it shows the correct file sizes)

ln -s behaves differently than cp -rs though, but I don’t really grasp exactly what it does tbh.
The man page doesn’t really explain how it handles recursion through files in the target folder.

It looks like it creates a link to the target directory and the files underneath are copies of the original target files.
But I assume it behaves more like a shortcut to the target folder instead.

And unless I misunderstand cp -rs just copies the directory structure and creates links to each individual file in those newly created folders.

[roan]

Currently trying to troubleshoot the symlinks created with cp -rs
It seems to have something to do with file/ folder nesting, but I am still unable to reliable reproduce.

[roan]

Reproduced! I think

You have to have a nested folder within the symlink that also has spacing in the name (any special character maybe? didn’t try that far)

Quick way to reproduce:

• Create a folder that:
  • Contains a file
  • A folder
  • A folder with a space in the name
  • Both subfolders should also have a file
• Then simply cp -rs fullpathtotargetfolder linkfolder

In my case:

targetfolder/
├── test/
│   └── file2
├── test +spacing/
│   └── file3
└── file1

Result (at least in my case):

• file1 and file2 should display the correct file size
• file3 should display the file size of the symlink

[matt (Matt Holt)]

Thanks; I’m not quite sure I follow though, where is linkfolder in your tree? Are you serving the target folder or the link folder?

[roan]

Targetfolder is a folder somewhere on disk
Linkfolder is the symlink created

I have caddy host that symlink, something like this:

        handle_path /files/* {
                root * ./caddyfolder
                file_server browse
        }

With the symlink linkfolder existing withing that ./caddyfolder

[roan]

If it helps, here is the complete setup outlined:

Caddy config:

	redir /files /files/

	handle_path /files/* {
		root * ./files
		file_server browse
	}

So basically <domain>/files/ serves the files inside the /mnt/data/caddy/files folder

Targetfolder contents:

/mnt/data/targetfolder/
├── test/
│   └── file2
├── test +spacing/
│   └── file3
└── file1

(targetfolder might be a slightly confusing name choice, but the ln man page used it, so I figured it would be clear enough :P)

Creating the symlink
cd inside the /mnt/data/caddy/files folder
Run cp -rs /mnt/data/targetfolder/ ./testingsymlinks

The files folder served by Caddy now contains a symlink to the original folder
Caddy is serving the symlink (and not the original folder).

The issue:
The Caddy file_server browse template displays the file sizes of these symlinks incorrectly in a specific case .

With the example files:
(file 1, 2 & 3 are copies of eachother, so they should all be exactly the same)

• file1 and file2 display the correct file size
• file3 displays the file size of the symlink instead of the original file.

[Mohammed90 (Mohammed Al Sahaf)]

I don’t think the spaces (or special characters) make any difference. The issue is elsewhere. Here’s my shot at reproduction:

cloudshell:~ (gitea-184913)$ tree --du -h ./created-with-cp                                                                                                       
./created-with-cp
├── [ 12K]  srv
│   ├── [  27]  file.txt -> /.../srv/file.txt
│   ├── [4.0K]  test
│   │   └── [  32]  file.txt -> /.../srv/test/file.txt
│   └── [4.0K]  test space
│       └── [  38]  file.txt -> /.../srv/test space/file.txt
├── [4.0K]  test
└── [4.0K]  test space

  24K used in 5 directories, 3 files
cloudshell:~ $ tree --du -h ./created-with-ln
./created-with-ln
├── [  32]  file.txt
├── [4.1K]  test
│   └── [  78]  file.txt
└── [4.2K]  test space
    └── [ 154]  file.txt

 8.3K used in 2 directories, 3 files
cloudshell:~ $ tree --du -h ./srv
./srv
├── [  32]  file.txt
├── [4.1K]  test
│   └── [  78]  file.txt
└── [4.2K]  test space
    └── [ 154]  file.txt

  12K used in 2 directories, 3 files
cloudshell:~ $ cat /etc/os-release 
PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"
NAME="Debian GNU/Linux"
VERSION_ID="11"
VERSION="11 (bullseye)"
VERSION_CODENAME=bullseye
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

My conclusion? Whatever cp -rs does is broken. The content of ./srv (last tree at the bottom) is the original files. I created srv then everything else is linked to it. Notice the files sizes. Now check the sizes of the files linked from created-with-ln vs created-with-cp. The work of ln matches the original sizes. cp is doing its own thing, which is arguably broken.

Caddy is reporting whatever the OS/filesystem reports. I don’t see this as an issue with Caddy. I recommend using ln for more correct behavior.

By the way…

@roan

It looks like it creates a link to the target directory and the files underneath are copies of the original target files.

This is not accurate. ln on its own creates hard links, which creates multiple “copies” of the file without actually copying its content. A change in one syndicates to the others. However, ln -s create symbolic links, aka soft links, which only create pointers, no copies at all.

[roan]

I understand what I said about ln is not accurate, that is what it looks like though

Regarding cp being broken, I’m not sure I agree, I think the way cp handles it is more transparent to the end user.
I’ll try to explain what I mean

I used the same files as before, put them in a caddy/files/original/ folder

• ran cp -rs /mnt/data/caddy/files/original/ cpsymlink
• ran ln -s /mnt/data/caddy/files/original/ lnsymlink
• and then ran ls -l on the root folder of the file server: caddy/files
  and on each of the subfolders/ symlinks

roan@rasNAS:/mnt/data/caddy/files $ ls -lh
total 8.0K
drwxr-xr-x 4 roan roan 4.0K Nov 14 20:23 cpsymlink
lrwxrwxrwx 1 roan roan    9 Nov 14 20:23 lnsymlink -> original/
drwxr-xr-x 4 roan roan 4.0K Nov 13 20:50 original
roan@rasNAS:/mnt/data/caddy/files $ ls -lh original/
total 40M
-rwxr--r-- 1 roan roan  40M Nov 11 20:03  file1
drwxr-xr-x 2 roan roan 4.0K Nov 13 20:50  test
drwxr-xr-x 2 roan roan 4.0K Nov 13 20:50 'test +spacing'
roan@rasNAS:/mnt/data/caddy/files $ ls -lh lnsymlink/
total 40M
-rwxr--r-- 1 roan roan  40M Nov 11 20:03  file1
drwxr-xr-x 2 roan roan 4.0K Nov 13 20:50  test
drwxr-xr-x 2 roan roan 4.0K Nov 13 20:50 'test +spacing'

The symlink created by ln is reported as a symlink by ls, but only the top level lnsymlink/ folder.
The contents of the lnsymlink/ folder display just like any normal files (hence me originally saying it looked like a straight up copy).
This looks confusing to the end user imo (there’s a reason we’re talking about it right now ), I don’t think I agree with saying that this is the more correct approach.

The top level folder of the cp symlink (cpsymlink/) however:

roan@rasNAS:/mnt/data/caddy/files $ ls -lh
total 8.0K
drwxr-xr-x 4 roan roan 4.0K Nov 14 20:23 cpsymlink
lrwxrwxrwx 1 roan roan    9 Nov 14 20:23 lnsymlink -> original/
drwxr-xr-x 4 roan roan 4.0K Nov 13 20:50 original
roan@rasNAS:/mnt/data/caddy/files $ ls -lh cpsymlink/
total 8.0K
lrwxrwxrwx 1 roan roan   36 Nov 14 20:23  file1 -> /mnt/data/caddy/files/original/file1
drwxr-xr-x 2 roan roan 4.0K Nov 14 20:23  test
drwxr-xr-x 2 roan roan 4.0K Nov 14 20:23 'test +spacing'

It doesn’t display as a symlink, looks like any normal folder.
But, the containing files are displayed as symlink, which (personally I think) is more transparent to the end user.

[roan]

Anyways, back to Caddy

Regardless of what cp -rs does is correct or not
The point still stands that Caddy handles file1 and file2 in the example just fine but breaks with file3
So something is broken there.

Either they should all display with the size of the symlink like @Mohammed90 was saying with:

Whatever cp -rs does is broken.
…
Caddy is reporting whatever the OS/filesystem reports

or whatever handles this file information has a bug somewhere.

---

I did start using ln -s as a workaround for now, so do thank you for the earlier suggestion Mohammed

[matt (Matt Holt)]

All Caddy does is check if the OS has the symlink flag set on the file mode. If so, it’s treated as a symlink. If not, it’s not. It says that symlinks are directories if the target of the symlink is a directory. (These are symbolic links, not hard links.)

I don’t really understand the difference between cp -rs and ln -s but it must be messing with one of those properties such that Caddy is getting the “wrong” or “misleading” information from the OS.

If there’s a bug in our code I’ll happy accept a patch.