create a filesystem as a non root user

Question

I'd like to create a filesystem image as a non-root user, the problem I'm facing is that the way I'm currently doing it, it needs a mount and as a non-root can't do a mount.

This is what I'm now doing:

#!/bin/sh

IMG="app.ext4"
LBL="PIHU_APP"
MNT="/mnt/PIHU_APP"
APP="/home/buildroot/board/arctura/PIHU_APP/application"

#64MB:
dd if=/dev/zero of=$IMG bs=4096 count=16384

mkfs.ext4 $IMG
e2label $IMG $LBL

mkdir -p $MNT
mount -o loop $IMG $MNT
cp -r $APP $MNT
sync
umount $MNT

I have full root access, so I can setup/prepare anything, but the script will be executed from a non-root account.

However, I have a feeling there may be a better way that might not even need mounting, but I'm not sure..

Answer 1

mke2fs -d minimal runnable example without sudo

mke2fs is part of the e2fsprogs package. It is written by the famous Linux kernel filesystem developer Theodore Ts'o who is at Google as of 2018, and the source upstream is under kernel.org at: https://git.kernel.org/pub/scm/fs/ext2/e2fsprogs Therefore, that repository can be considered a reference userland implementation of ext file system operations:

#!/usr/bin/env bash
set -eu

root_dir=root
img_file=img.ext2

# Create a test directory to convert to ext2.
mkdir -p "$root_dir"
echo asdf > "${root_dir}/qwer"

# Create a 32M ext2 without sudo.
# If 32M is not enough for the contents of the directory,
# it will fail.
rm -f "$img_file"
mke2fs \
  -L '' \
  -N 0 \
  -O ^64bit \
  -d "$root_dir" \
  -m 5 \
  -r 1 \
  -t ext2 \
  "$img_file" \
  32M \
;

# Test the ext2 by mounting it with sudo.
# sudo is only used for testing.
mountpoint=mnt
mkdir -p "$mountpoint"
sudo mount "$img_file" "$mountpoint"
sudo ls -l "$mountpoint"
sudo cmp "${mountpoint}/qwer" "${root_dir}/qwer"
sudo umount "$mountpoint"

GitHub upstream.

The key option is -d, which selects which directory to use for the image , and it is a relatively new addition to v1.43 in commit 0d4deba22e2aa95ad958b44972dc933fd0ebbc59

Therefore it works on Ubuntu 18.04 out of the box, which has e2fsprogs 1.44.1-1, but not Ubuntu 16.04, which is at 1.42.13.

However, we can do just like Buildroot and compile it from source easily on Ubuntu 16.04 with:

git clone git://git.kernel.org/pub/scm/fs/ext2/e2fsprogs.git
cd e2fsprogs
git checkout v1.44.4
./configure
make -j`nproc`
./misc/mke2fs -h

If mke2fs fails with:

__populate_fs: Operation not supported while setting xattrs for "qwer"
mke2fs: Operation not supported while populating file system

when add the option:

-E no_copy_xattrs

This is required for example when the root directory is in NFS or tmpfs instead of extX as those file systems don't seem to have extended properties.

mke2fs is often symlinked to mkfs.extX, and man mke2fs says that if you use call if with such symlink then -t is implied.

How I found this out and how to solve any future problems: Buildroot generates ext2 images without sudo as shown here, so I just ran the build with V=1 and extracted the commands from the image generation part that comes right at the end. Good old copy paste has never failed me.

TODO: describe how to solve the following problems:

• create sudo owned files in the image. Buildroot does it.
• automatically calculate the minimal required size. Initial estimate with du for file size and find . | wc for directory structure, min that with 32Mb (smaller fails), then double until the command works, is likely a very decent approach. Buildroot used to do this, but stopped for some reason, but easy to implement ourselves.
• conveniently extract all files from the partition:
  • How can I extract files from a disk image without mounting it?
  • https://superuser.com/questions/826341/access-data-from-a-linux-partition-inside-an-image-without-root-privileges

Multiple partitions in one image file

See this: https://stackoverflow.com/questions/10949169/how-to-create-a-multi-partition-sd-image-without-root-privileges/52850819#52850819

Answer 2

Does it have to be ext4?

If alternatives are possible, you could try your luck with

• mksquashfs, produces a read-only compressed filesystem from userspace
• mkisofs, produces a read-only filesystem commonly used for optical media
• cpio, tar, ... archives that need to be extracted first

Otherwise, to make ext4 in userspace a reality, you have to

• go crazy with debugfs
• boot a kernel in a userspace vm
• find another pure userspace utility that knows to handle ext4 images

Regarding debugfs, most filesystems don't even have such utilities, you lucked out.

User session:

$ truncate -s 64M app.ext4
$ /sbin/mkfs.ext4 app.ext4
Creating filesystem with 65536 1k blocks and 16384 inodes
$ /sbin/debugfs -w app.ext4
debugfs:  mkdir this_is_crazy
debugfs:  cd this_is_crazy
debugfs:  write /proc/config.gz config.gz
Allocated inode: 13

Root session: (just to check if it worked)

# mount -o loop app.ext4 loop/
# cd loop/
# ls
lost+found  this_is_crazy
# md5sum this_is_crazy/config.gz /proc/config.gz 
7b414ad844272a9e3c31931037fe0495  this_is_crazy/config.gz
7b414ad844272a9e3c31931037fe0495  /proc/config.gz

So, it's possible and it works.

However, the most practical option should still be to use root in some fashion anyway.

Answer 3

sudo is made for exactly this type of problem, you need to give the user permission to run the mount command in the /etc/sudoers file

then in your script you can prepend the mount command with sudo:

sudo mount -o loop $IMG $MNT

You can configure the rule so that it does not require a password

Answer 4

You can add it to fstab with the user option, so that it's mountable by a user.

/path/to/app.ext4 /mount/point ext4 loop,user,noauto 0 0

Then your user can create the file and just run:

mount /mount/point

Answer 5

virt-make-fs command can meet all your requirements

sudo yum install -y /usr/bin/virt-make-fs
virt-make-fs -s 64M -t ext4 $APP $IMG --label=$LBL

see also: https://libguestfs.org/virt-make-fs.1.html