Why can't unprivileged users nest FUSE mounts, but they can mount FUSE inside NFS with root_squash?

Question

$ mkdir mnt

$ bindfs /tmp mnt
fusermount: option allow_other only allowed if 'user_allow_other' is set in /etc/fuse.conf

$ bindfs --no-allow-other /tmp mnt

$ mkdir /tmp/mnt2
$ bindfs --no-allow-other /tmp mnt/mnt2
fusermount: bad mount point /home/alan/mnt/mnt2: Permission denied

fusermount fails because it runs as a different user.

$ sudo ls mnt/
ls: cannot open directory 'mnt/': Permission denied

fusermount is set-uid root. This is required because unprivileged users cannot use the mount() system call.

$ ls -l $(which fusermount)
-rwsr-xr-x. 1 root root 32848 Feb  7  2018 /usr/bin/fusermount

   ^ set-uid bit

And yet. It is reported that FUSE can be used inside an NFS home directory. Even though the home directory has mode 700 - accessible only by the owning user. And the NFS server defaults to root_squash, which means "the root user will have the same access as user nobody".

Why do these two cases differ?

I am testing on Fedora 28. The reports about NFS are from Ubuntu 18.04. These distributions are quite similar in age, but there might be some differences.

Answer 1

First, consider the implementation of FUSE no_allow_others.

It requires that the effective, real, and saved UIDs (user IDs) all match. (And the same for GID). This is deliberately to stop a set-UID program from accessing the mount.

https://github.com/torvalds/linux/blob/v4.18/fs/fuse/dir.c#L1024

Calling into a user-controlled filesystem gives the filesystem
daemon ptrace-like capabilities over the current process. This
means, that the filesystem daemon is able to record the exact
filesystem operations performed, and can also control the behavior
of the requester process in otherwise impossible ways. For example
it can delay the operation for arbitrary length of time allowing
DoS against the requester.

Now let's trace what fusermount does. We can try looking at

strace -f bindfs ...

and

sudo perf trace -o trace.txt -a sleep 2; sleep 1; bindfs ...

The first one hits a fatal error "Permission denied", because set-UID root does not work when running under strace. The second one succeeds, but cannot show string parameters such as paths. I think the two traces show the same general code path up until the fatal error. This means we can use the strace results to fill in the missing string parameters.

That last call in the strace results is:

[pid 30609] mount("/home/alan-sysop/mnt", ".", "fuse", MS_NOSUID|MS_NODEV, "default_permissions,fd=5,rootmod"...) = -1 EPERM (Operation not permitted)

Interesting! "." means the current directory. So fusermount must already have been running on the mount point... somehow. This trick can sometimes be used to access a directory, that you currently cannot access using its absolute path.

If we scroll up, we can see that fusermount did indeed change into this directory. And it was also dancing with some UID-related (and GID-related) system calls.

[pid 30609] getuid()                    = 1000
[pid 30609] setfsuid(1000)              = 1000
[pid 30609] getgid()                    = 1000
[pid 30609] setfsgid(1000)              = 1000
[pid 30609] openat(AT_FDCWD, "/etc/fuse.conf", O_RDONLY) = 6
...
[pid 30609] lstat("/home/alan-sysop/mnt", {st_mode=S_IFDIR|0775, st_size=4096, ...}) = 0
[pid 30609] getuid()                    = 1000
[pid 30609] chdir("/home/alan-sysop/mnt") = 0
[pid 30609] lstat(".", {st_mode=S_IFDIR|0775, st_size=4096, ...}) = 0
[pid 30609] access(".", W_OK)           = 0
[pid 30609] getuid()                    = 1000
[pid 30609] setfsuid(1000)              = 1000
[pid 30609] setfsgid(1000)              = 1000

The UID results are "wrong" in the strace session. We can see the UID dance part better in the perf trace session. (I've removed the left-most columns for readability).

getuid(                                                               ) = 1000
setfsuid(uid: 1000                                                    ) = 0
getgid(                                                               ) = 1000
setfsgid(gid: 1000                                                    ) = 1000
openat(dfd: CWD, filename: 0xa428e2bc                                 ) = 6
    ...
close(fd: 6                                                           ) = 0
lstat(filename: 0xa63882a0, statbuf: 0x7ffe7bd4f6d0                   ) = 0
getuid(                                                               ) = 1000
chdir(filename: 0xa63882a0                                            ) = 0
lstat(filename: 0xa428eca5, statbuf: 0x7ffe7bd4f6d0                   ) = 0
access(filename: 0xa428eca5, mode: W                                  ) = 0
getuid(                                                               ) = 1000
setfsuid(                                                             ) = 1000
setfsgid(gid: 1000                                                    ) = 1000
getuid(                                                               ) = 1000

The setfsuid() calls are are in the drop_privs() and restore_privs() functions in fusermount.c.

The chdir() call is sneakily hidden in the function called check_perm().

Conclusion

Why does this work on NFS? Answer: because NFS looks at the fsuid (and fsgid), which have been set to the non-root UID.

Why does this not work on FUSE, unless you have allow_others? Answer: because FUSE checks the "real" UID, and not the fsuid.