4

I have a giant 218 TB RAID6, across LVM. It is actually 2 separate RAID6s joined together through LVM to form one giant RAID6. df -h shows all space was available, and I mounted to /share.

I wanted to test that all of the 218 TB was actually usable, so I started running fallocate -l 1TB in a loop, and when it got to 75 TB it stated that the device was out of space. Typing df -h, it showed that there was still 144 TB free on /share.

I looked this up and a lot of people recommended typing df -i to see if all the inode space was taken up, but only 1% used.

Is there a hidden limitation to how much space can actually be allotted to one RAID6? Or is it because I have it configured with LVM?

Any advice is greatly appreciated.

4
  • 1
    If your issue is like this one centos.org/forums/viewtopic.php?t=21766 , you may need to mount with the -o inode64 option. Jun 30, 2015 at 19:13
  • Yes, I am using LVM2.
    – Kamran
    Jun 30, 2015 at 20:44
  • That issue looks similar to what I am experiencing, but when I try to mount with the inode64 option and create files, my terminal kinda freezes, waiting for the process to complete, and then it spits out kernel errors.
    – Kamran
    Jun 30, 2015 at 20:44
  • I tried to reproduce your issue using a 218T loop device in tmpfs. The first time I tried I thought I could reproduce it (fallocate stopped after the 114th 1TB file). Subsequent tries it happily fallocates 217+ TB files. Even works with inode32. Never could reproduce why it didn't work the first time though and now I'm unsure if I imagined it... ;) Jul 2, 2015 at 17:45

1 Answer 1

5

I'm not sure if this is the one-and-done solution, but it worked for me. I had to create my "LVM" with striping options.

lvcreate -L 217T -i2 -I64 -n lv_share VolGroup

See 4.4.2. Creating Striped Volumes.

Then I had to mount it with the -o inode64 option, as Mark mentioned.

See 8.2. Mounting an XFS File System.

2
  • Isn't inode64 the default? Jul 2, 2015 at 17:22
  • @frostschutz apparently not, cf. note about it in mount(8) Feb 23, 2016 at 15:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.