2

My initial google searches say XFS or ReiserFS, but what does the community have to say about this? What are your experiences with 50TB sizes and up?

Edit: After inquiring on IRC I think I would recommend Ext4, and splitting up the storage into 10TB chunks. But I'd still be very interested to hear experiences.

3 Answers 3

3

While I haven't used it myself, ZFS is designed specifically to be able to support large amounts of data with data integrity. If you haven't looked it over already, it may be worth investigating.

2
  • I'm just worried about Linux support for ZFS. stuff like this worries me especially because I was going to use the filesystem on a RHEL OS. Oct 21, 2011 at 11:20
  • Interesting read. Like I said, I haven't used it myself, and I'm not sure if I ever will due to the general lack of support outside of Solaris, but it was specificaly designed to handle large amounts of data. Reiser may be better for your application due to the better support or possibly ext4 like you said. Good luck, I'd be interested to know what you end up using and your general results.
    – sbtkd85
    Oct 24, 2011 at 14:53
2

Imho, its a bad idea to split the 50TB file system into smaller chunks because it introduces problems at a later time: either the partitions are too small, or you have to use something like LVM to be able to resize your partitions later. Note that resizing/shrinking is not supported on all filesystems.

For the fs choice, i would go with ext4.

3
  • 1
    ext4 is currently limited to 16 TB, hence splitting it up. If you are managing that kind of storage, you absolutely should be using LVM.
    – psusi
    Oct 12, 2011 at 19:55
  • Resizing/shrinking can turn a mess! I've lost tons of data (not to mention time the operation took) once I've tried to resize an ext4 partition to the begining of the disk (to claim the unpartitioned space before it) with gparted. Why haven't I had backups? Because that was the disk I've been putting all the backups to and I am not that rich to have a disk for backing up back-ups.
    – Ivan
    Feb 3, 2012 at 20:26
  • EXT filesystems are generally a bad choice when it comes to large structures. The EXT filesystems use linked lists for directory enumeration, which is horrendously slow and inefficient. Though if the directories only have a small number of items, this might not be much of an issue. And IIRC, EXT3/4 can be set to use btree, but it's still nowhere near as efficient as others like xfs or zfs.
    – phemmer
    Nov 20, 2013 at 13:50
0

I have a 50Tb BtrFS on top of raid-5 and LVM (about 80% full after having added 3 disks and extending the raid-5). Works stable.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.