Unionfs vs Aufs vs Overlayfs vs mhddfs, which one do I use

Question

I have randomly been reading about union file system which enables a user to mount multiple filesystems on top of one another simultaneously.

However, am finding trouble deciding on which one to use(Unionfs vs Aufs vs Overlayfs vs mhddfs) and why as I have not found concrete information on the subject anywhere. I know for instance that overlayFS has been adopted in the mainstream Linux kernel which means it might get wider adoption. Would appreciate if someone would give me some perspective.

Also I can't find any conceiving use-case for Union file system over something like LVM (as recommended by users in separate question) or RAID setup except in the fact that LVM requires formatting all the drives which might not be desirable if you already have valuable data on the drives.

Answer 1

Here are some thoughts - I am still learning this and will update this as I go.

How to choose the union filesystem

There are two ways to look at this:

• How do the features of each one compare?
• For some common use cases, which one should I choose?

I'll compare unionfs / unionfs-fuse / overlayfs / aufs / mergerfs, the latter being a replacement for mhddfs.

Features of each one

Development status

• aufs seems to be active
• unionfs looks mature, but not under active development?
• unionfs-fuse seems to be active
• mergerfs seems to be active
• overlayfs is active

Distribution / Kernel support

There are kernel mode and usersystem mode filesystems, the latter run on FUSE. Kernel mode ones have less overhead (there is overhead when code switches between user space and kernel space) but the only one currently supported in the Linux kernel is overlayfs. User mode filesystems are easier for distributions to package.

• unionfs and aufs need kernel patches
• unionfs is not distributed by Debian (the rest are)
• unionfs-fuse and mergerfs are based on FUSE, so don't need to additional modules in the kernel
• overlayfs has been part of the kernel since 3.18 (Debian Stretch)

Copy on write

This relates to the Live CD use case below:

• mergerfs does not have copy on write
• The others do

Use cases

Read-only root / The Live CD use case

The idea is to have a read-only CD-ROM/partition of a linux system. The union filesystem makes it look to the user like it is a read-write system so they can make changes. There is a read-write filesystem (for example, a tmpfs RAM disk) which stores the "Delta" of any changes made by the user, but not the full snapshot.

Here any of the union filesystems except mergerfs would do (lack of cow support).

Docker use case

I am aware this is a main use case, but don't know the details - can someone provide guidance on this?

Merging hard disks

For example, you might have two sets of /home directories on different filesystems. Or you might be upgrading your home computer with a second hard disk, and want a single logical volume.

This is where you don't actually want copy-on-write, so possibly mergerfs is the best choice.

Union filesystem versus LVM for disk pooling

I'll list some use cases that can be achieved with union filesystems but not LVM:

If you are upgrading an existing system with a second disk, something like mergerfs might be better because LVM would require you to reformat the first hard disk hence destoying the data on it. A union filesystem would avoid this step.

LVM might split a file over two physical hard disks (assuming RAID 0), so you would lose it if one hard disk fails.

Some users might like, for example, to keep their /home directory on a USB stick that they can take away.

In the use case of one virtual partition on two physical disks, with LVM you wouldn't need to worry about whether files get saved on one disk or the other. With mergefs, the system can automatically choose which one for you depending on how much free space is available.

Answer 2

Bob's answer was very helpful as a primer/overview, I want to add my own conclusions on the situation from a user's point of view.

Historically, the concept of union mount file system can be traced back to the earlier union mounts. Then came unionfs in the early 2000s, which kind of was eclipsed by aufs in popularity, but ultimately was rejected by kernel maintainers in favor of overlayfs. Overlayfs seems to be the only game in town for providing a comparatively simple (at most one read-write layer), in kernel, union file system and is extensively used by things like docker.

The original unionfs seems to have been more featureful than overlayfs (e.g. allowing multiple read-write layers and such). Such complexity is nowadays mainly provided by fuse based union filesystems like unionfs-fuse (an independent project from unionfs) and mergerfs.

Mergerfs seems to be the go to choice at time of writing and enables complex setups like combining multiple plain hard disks and choosing among different policies for how creation of new files/writes are spread among these disks.

By itself, mergerfs can be used as a simple disk based (JBOD) alternative to using LVM/RAID0/BTRFS(single) for combining disks into a single filesystem, with the main advantage being resilience. One dying disk obviously won't bring down the whole filesystem. Mergerfs also combines well with snapraid to give the user a RAID5/6-ish setup (snapraid stores parity information and therefore allows rebuilding a disk, but see the docs for differences to RAID).

TLDR: Use overlayfs for simple layering with one RW layer "on top", mergerfs as an alternative to LVM/RAID0 for combining multiple disks into one logical file system for something like a media collection.