Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

My initial google searches say XFS or ReiserFS, but what does the community have to say about this? What are your experiences with 50TB sizes and up?

Edit: After inquiring on IRC I think I would recommend Ext4, and splitting up the storage into 10TB chunks. But I'd still be very interested to hear experiences.

share|improve this question
add comment

3 Answers

up vote 2 down vote accepted

Imho, its a bad idea to split the 50TB file system into smaller chunks because it introduces problems at a later time: either the partitions are too small, or you have to use something like LVM to be able to resize your partitions later. Note that resizing/shrinking is not supported on all filesystems.

For the fs choice, i would go with ext4.

share|improve this answer
1  
ext4 is currently limited to 16 TB, hence splitting it up. If you are managing that kind of storage, you absolutely should be using LVM. –  psusi Oct 12 '11 at 19:55
    
Resizing/shrinking can turn a mess! I've lost tons of data (not to mention time the operation took) once I've tried to resize an ext4 partition to the begining of the disk (to claim the unpartitioned space before it) with gparted. Why haven't I had backups? Because that was the disk I've been putting all the backups to and I am not that rich to have a disk for backing up back-ups. –  Ivan Feb 3 '12 at 20:26
    
EXT filesystems are generally a bad choice when it comes to large structures. The EXT filesystems use linked lists for directory enumeration, which is horrendously slow and inefficient. Though if the directories only have a small number of items, this might not be much of an issue. And IIRC, EXT3/4 can be set to use btree, but it's still nowhere near as efficient as others like xfs or zfs. –  Patrick Nov 20 '13 at 13:50
add comment

While I haven't used it myself, ZFS is designed specifically to be able to support large amounts of data with data integrity. If you haven't looked it over already, it may be worth investigating.

share|improve this answer
    
I'm just worried about Linux support for ZFS. stuff like this worries me especially because I was going to use the filesystem on a RHEL OS. –  Stefan Midjich Oct 21 '11 at 11:20
    
Interesting read. Like I said, I haven't used it myself, and I'm not sure if I ever will due to the general lack of support outside of Solaris, but it was specificaly designed to handle large amounts of data. Reiser may be better for your application due to the better support or possibly ext4 like you said. Good luck, I'd be interested to know what you end up using and your general results. –  sbtkd85 Oct 24 '11 at 14:53
add comment

I have a 50Tb BtrFS on top of raid-5 and LVM (about 80% full after having added 3 disks and extending the raid-5). Works stable.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.