2

I've been looking into using OverlayFS. I'd like to be able to combine a bunch of already formatted, already containing data, hard drives. All hard drives are formatted with ext4.

The goal is to make it so I can use all the storage as one giant location. I currently make use of MergerFS so all the files contained do not overlap anywhere. I would like to move away from MergerFS because I had some issues and OverlayFS seems to be supported in the upstream kernel itself.

But I'm not sure how to configure OverlayFS to do this, is it even possible? On first glance it seems like I can only have one writable location and the other locations are just there to provide the files they have.

1

On first glance it seems like I can only have one writable location and the other locations are just there to provide the files they have.

This is correct, OverlayFS only supports one writable layer at the top. As such, I'd say it's not really suitable for the use case you describe.

The goal is to make it so I can use all the storage as one giant location.

I would say RAID (such as RAID 0 for just striping, or higher levels for redundancy that can withstand disk failures) or a volume manager (such as LVM, which can concatenate disk volumes and can also do striping) are the typical solutions for the problem you describe.

Though you mentioned:

All hard drives are formatted with ext4.

And that's not how these solutions work, they work on block devices such as disk partitions, so you'd end up creating a single filesystem (ext4 or otherwise) on top of the LVM logical volume or RAID device instead.

I'd still recommend using one of these two solutions, since they were made specifically for the use case you describe and they're really stable (having been around for a long time and used in many mainstream products and enterprise deployments.)

| improve this answer | |
  • @filbraden I thought about using a file system that would span across the whole lot of disks but I'm worried about one disk failing and not being able to recover the rest of the data from the still working drives. I know I should have a backup but I want that option just in case. Is it possible to setup that in a way where I can recover the rest of the working drives if one fails? – RansuDoragon May 12 '19 at 11:08
  • @RansuDoragon A RAID with redundancy (such as RAID 5) would be able to do that, in which case if one disk fails you still get to keep 100% of your data. The price to pay is in space, for one of the disks, so if you have 4x 1TiB drives the total available space will be 3TiB (one is for redundancy.) Also, if two disks fail, you lose the whole thing, so only backups would be able to restore any contents at that point. – filbranden May 12 '19 at 11:13
  • @filbraden I wanted to use all of my drives as one whole big storage. I already have a backup drive. – RansuDoragon May 12 '19 at 11:13
  • Maybe MergerFS really is what you want then, though you mentioned you had some issues with it (I'm not surprised that it's not that stable...) You might want to consider btrfs as well, which is a filesystem that blends volume management capabilities into the filesystem itself. You'd have to look into it, but it's possible that btrfs would withstand a disk failure and still give you access to the files in the disks that are still working. I'm not sure about that though, since it'd have to store each file on a single disk and replicate metadata. It's possible it's able to. Take a look. – filbranden May 12 '19 at 11:25
  • Thank you for the suggestion about btrfs, I'll have to look into maybe using it. I've heard about it but never used it personally on anything before. In the meantime I have reported the issue to the MergerFS maintainer github.com/trapexit/mergerfs/issues/616#issuecomment-491985407 – RansuDoragon May 13 '19 at 23:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.