6

TL;DR

Attempting to format a block device on my FUSE file system fails with EPERM at the open syscall. Permissions are set to 777 and the necessary ioctls are stubbed, but no logs are printed from within the FUSE handler.

Background

I'm writing a program to create virtual disk images. One of my criteria is that it must be able to run with zero superuser access, meaning I can't mount loopback devices, change owners of files or even edit /etc/fuse.conf. For this reason, my approach ends up being fairly long-winded. Specifically, in order to format the various partitions on the disk, I would like to be able to use system tools, because that gives me a far greater range of possible file systems. This involves exposing the various partitions on the VDisk as block devices to the system. However, all the possible methods I've found have required either nbds or loopback devices. Both of which require superuser access.

Implementing FUSE myself

However, implementing block devices in FUSE is not only possible, but supported. Unfortunately, I wasn't able to find much documentation on the matter and since I'm doing all this in Rust, the documentation world for this is even more scarce.

I've implemented the following FUSE methods:

  • init
  • lookup
  • getattr
  • open
  • read
  • write
  • readdir
  • ioctl
    • BLKGETSIZE
    • BLKFLSBUF
    • BLKSSZGET

I can list the contents of the file system and get directory/file information. I'm deliberately ignoring methods which create or modify resources, as this is done through the build process.

The error

As mentioned, I get permission denied (EPERM) error. straceing the mkfs call shows that it's the open call to the block device that fails on the kernel side. Full strace result.

execve("/usr/sbin/mkfs.fat", ["mkfs.fat", "out/partitions/EFI"], 0x7ffd42f64ab8 /* 76 vars */) = 0

    --- snip ---

openat(AT_FDCWD, "out/partitions/EFI", O_RDWR|O_EXCL) = -1 EACCES (Permission denied)
write(2, "mkfs.fat: unable to open out/par"..., 63mkfs.fat: unable to open out/partitions/EFI: Permission denied
) = 63
exit_group(1)                           = ?

For clarity, my directory structure looks like this:

out
├── minimal.qcow2 [raw disk image] (shadows minimal.qcow2 [qcow2 file] with qemu-storage-daemon)
├── partitions
│   ├── EFI [Block device]
│   └── System [Block device]
└── qemu-monitor.sock [UNIX domain socket]

Of course, there are logging functions tracing every method. I do see logs when listing out the partitions, but not when formatting.

As I mentioned, I've found very little documentation on what could actually be causing this error.

Further insights

Thanks to the insights from @orenkishon, I've found some more details that just baffle me.

  1. I found some options in fuser which were interesting:

    • MountOption::Dev Enable special character and block devices
    • MountOption::DefaultPermission Enable permission checking in the kernel
    • MountOption::RW Read-write filesystem (apparently not a default option)

    Unfortunately, no combination of which resolved my issue.

  2. Log functions aren't called immediately. They seem to be tied to some sort of flushing operation. I can run the mkfs.fat command, see one or two logs, switch back to my IDE and see a page worth of logs appear.

    This may be due to the fact that the directory I'm generating the files is within the project's directory, so it is visible to the IDE, but it strikes me as very unusual.

  3. The log in the access function is never visible, but in the statfs function is, but only if mkfs is called from outside the out directory and is the first of any mkfs calls.

    project > cd ./out
    project/out > mkfs.fat partitions/EFI
    mkfs.fat 4.2 (2021-01-31)
    mkfs.fat: unable to open partitions/EFI: Permission denied
    
    # No logs
    
    project > mkfs.fat out/partitions/EFI
    mkfs.fat 4.2 (2021-01-31)
    mkfs.fat: unable to open out/partitions/EFI: Permission denied
    
    # No logs
    project > cargo run ...
    project > mkfs.fat out/partitions
    mkfs.fat 4.2 (2021-01-31)
    mkfs.fat: unable to open out/partitions/EFI: Permission denied
    
    # Logs appear after switching to IDE
    
  1. I'm seeing this log message for the first time today:
[2024-04-21T16:58:24Z DEBUG fuser::mnt::fuse_pure] fusermount: 
[2024-04-21T16:58:24Z DEBUG fuser::mnt::fuse_pure] fusermount: fusermount3: unsafe option dev ignored

There is a MountOption::Dev sepcified, which supposedly adds support for block and character devices. However I can't seem to explain why it's being rejected. I was hopeful that I could use a patched version of libfuse3 but it seems not.

Extra info which may be useful

System Specs

Copied directly from KDE's System Info

  • Operating System: Kubuntu 23.10
  • KDE Plasma Version: 5.27.8
  • KDE Frameworks Version: 5.110.0
  • Qt Version: 5.15.10
  • Kernel Version: 6.5.0-28-generic (64-bit)
  • Graphics Platform: Wayland
  • Processors: 32 × 13th Gen Intel® Core™ i9-13900
  • Memory: 31,1 GiB of RAM
  • Graphics Processor: AMD Radeon RX 7900 XT
  • Manufacturer: ASUS

Generic writing operations also fail

A suggestion was to check whether mkfs fails in case fat32 doesn't support the block device. However this doesn't seem to be the case as formatting with any other filesystem produces the same results.

I'm also using mkfs as a testing platform because I currently don't know of any other readily-available system utilities that write to block devices directly, and mkfs is something I intend to use anyway.

Bad news :(

While reading the manpages I came across this paragraph which made my heart sink:

Most of the generic mount options described in mount are supported (ro, rw, suid, nosuid, dev, nodev, exec, noexec, atime, noatime, sync, async, dirsync). Filesystems are mounted with nodev,nosuid by default, which can only be overridden by a privileged user.

So it looks like this isn't possible. Still, any insights here - any slimmer of hope - would be highly appreciated.

10
  • What's your CPU architecture and your distro? I'm afraid the issue might be that without root it is barely possible to achieve that.. Commented Apr 27 at 8:54
  • Hey thanks for your input. I'm desperately hoping that barely possible doesn't mean impossible. In any case, see edits for system specs
    – J-Cake
    Commented Apr 27 at 13:35
  • You're not willing to use a custom kernel I'm guessing? Commented Apr 28 at 10:52
  • Hm, not really. I'd likely go as far as patches for common libraries, but in the kernel is def beyond my personal skills and crosses the boundary of practicalness. I would like this to be runnable as a regular program downloaded via apt or pacman on hopefully any Linux distro
    – J-Cake
    Commented Apr 28 at 13:11
  • There's also the heavy alternate way: guestfish which specializes in filesystem operations, by running an ephemeral VM for operations. Your user would ideally need permission to use /dev/kvm else qemu will do software emulation if nothing else is available: zero permission needed (but slower).
    – A.B
    Commented Apr 28 at 13:24

2 Answers 2

2
+300

If I understand correctly, you expose a block deice via your FUSE filesystem, which doesn't work for security reasons.

In Unix-like systems, files are much more than just files. Letting an unprivileged user access arbitrary block devices is problematic because, by granting access to a block device corresponding to the rootfs device, the system can be compromised.

In your case, it would probably be best to expose the partitions as regular files, since you are already working with FUSE and mkfs works just as well with these anyway.

PS. If there are other tools that truly require the partition to be a block device, one can emulate this using a similar approach to that of the fakeroot program (i.e. hooking libc calls and emulating their results)

9
  • Hey thanks for the pointers on fakeroot. This seems really promising! One question though is that since the nodev and nosuid options are implied, and require privileged users to override, how this is possible? My understanding is that these overrides take place in the kernel. If this is indeed not the case, then a patched library might be better, since there's no "magic" happening, y'know?
    – J-Cake
    Commented Apr 28 at 13:46
  • @J-Cake These restrictions are indeed in the kernel. What I suggest (again I'm not sure this is 100% helpful) is to not expose a block device at all. Instead running mkfs under myfakeroot which fakes only what is needed for mkfs-style programs to work. I'm not sure what it entails, and if it is even feasible, but I would at least do a quick strace on a regular mkfs run to see what needs to be faked exactly. But, after all, does a block device look that much different than a regular file? Also, specifically mkfs can work just fine on a file. Commented Apr 28 at 14:01
  • My previous debugging using strace on mkfs calls shows that the open call itself is prohibited. I may be completely missing your point here, but I don't see how this can work, if the dev can't even be opened. In any case, using a regular file was my next thought, although a block device is arguably the better fit for the job.
    – J-Cake
    Commented Apr 28 at 14:27
  • Yes, that's what I mean. Using a regular file as if it were a block device. The myfakeroot is just to fake special ioctls that can only be done on block devices. I agree that in an ideal world, using a block device will be better. But you can't do it without root... Commented Apr 28 at 14:40
  • Ah I see. In that case that might be perfect actually. I'll try it, and let you know
    – J-Cake
    Commented Apr 28 at 20:54
1

AFAICT, what you are (or apparently, were, since you now finds a "regular-to-regular" setup alright) trying to do, namely to create a block special file that represents a regular file instead an actual device, is simply impossible "by design".


It's true that the EACCES (not EPERM) is from a really "high level", that is, the mount that contains the block file does not allow a block file to be opened:

https://github.com/torvalds/linux/blob/v6.8/fs/namei.c#L3257 https://github.com/torvalds/linux/blob/v6.8/fs/namei.c#L3230

Because fixing that will not really help in making your goal possible anyway (and I'm not interested in that part either), I'm not gonna investigate why you couldn't get the fix, namely getting rid of nodev, applied. (P.S. I missed your edit about the fact that even this fix itself is not possible either.)


What would really prevent your goal from being accomplished is that, as the name "device file" or "block special file" suggests, for one they must be associated with a device (well, more precisely, set with some major and minor numbers), for two, since they are special in the sense that, they are not expected to need filesystem-specific handling, but device-file-specific handling, which can (or shall) be used generally regardless of what kind of filesystem the block file resides in.

Or in other words, you may think that there's a "trap" or "bypass" that make sure at least some operations on the file will never go to the filesystem implementation, whether it is ext4, btrfs, fuse, doesn't matter. (Here what I really mean is, normally the corresponding kernel driver will make use of the "trap", and fuse is indeed one of the cases, but if you are implementing a kernel filesystem driver yourself, I suppose technically you can opt not to, although what you write will probably never be in-tree, unless there's a legit reason).

https://github.com/torvalds/linux/blob/v6.8/fs/fuse/inode.c#L392 https://github.com/torvalds/linux/blob/v6.8/fs/inode.c#L2315 https://github.com/torvalds/linux/blob/v6.8/block/fops.c#L853 https://github.com/torvalds/linux/blob/v6.8/block/fops.c#L600

As you can see, the "trap" will make sure that when the block file is opened, a special handle that resolves its rdev (which is composed of the "major" and "minor" number you set on the block file) to a device and open the device. In other words, even if whatever library you use seemingly allow you to "map" the block file to a regular file or so, at the end of the day, the major and minor numbers set on the block file determines what it refers to, as long as its type is a block file.

Since most likely the numbers are initialized to some values that does not refer to any actual device in your system, you will get a ENXIO ("No such device or address"):

https://github.com/torvalds/linux/blob/v6.8/block/bdev.c#L840


Probably you are even more familiar with FUSE than me, but either way, this diagram can help in illustrating the idea:

https://commons.wikimedia.org/wiki/File:FUSE_structure.svg

Basically in the case of a block file, the "arrow" will NOT go out to the userspace libfuse part (which is or is equivalent to what leads to your customized filesystem implementation) from the kernel FUSE part, but to perhaps other related kernel part and go back to VFS and go out to the userspace to whatever program (e.g. ls in the diagram) that requested the operation.

4
  • Hi, I must confess I'm not entirely sure what you're saying. My interpretation is an attempt to re-explain the problem. Is this correct?
    – J-Cake
    Commented May 2 at 17:29
  • @J-Cake Well, basically this serves to show you (actually it's more like a learning log for myself) that, even if whatever (userspace) fuse library you use (you haven't really been clear on that) seems to allow you to emulate a block device with some sort of image file as its "backstore", it's just an "illusion" (because of the API is "loose"; I hope you get the sense), because a block special file cannot be made to represent something other than an actual device, not unless the kernel driver that works with the userspace library allows it, and with fuse, it's not the case.
    – Tom Yan
    Commented May 2 at 18:36
  • @J-Cake You probably have to at least look at the lines of kernel code I linked in order to follow what I'm trying to convey, but to be frank, this has steered away from your goal, as in, it's probably not gonna practically help you anyway even if you read and grok the whole thing. I'm merely just sharing what I've learnt regarding a question/doubt spawned inside my head after reading your post.
    – Tom Yan
    Commented May 2 at 18:42
  • I see. Well thanks for your input, ultimately I guess I worked this out the hard way. PS. I did state which fuse library I'm using fuser although it's easy to miss
    – J-Cake
    Commented May 2 at 18:42

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .