1

I have six 8TB drives on a BTRFS RAID10 that I installed yesterday on Rockstor (centos 7). df reports the size correctly but space available is 5.7 TiB short of what I expected... what gives?

A metadata rebalance had almost no effect.

[root@big ~]# df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/sdf         22T  3.3T   13T  22% /mnt2/main

[root@big ~]# btrfs fi show /dev/sdf
Label: 'main'  uuid: 738d116b-ce5d-46cb-8c40-abb4a03b2071
    Total devices 6 FS bytes used 3.30TiB
    devid    1 size 7.28TiB used 1.10TiB path /dev/sdf
    devid    2 size 7.28TiB used 1.10TiB path /dev/sdc
    devid    3 size 7.28TiB used 1.10TiB path /dev/sdb
    devid    4 size 7.28TiB used 1.10TiB path /dev/sdh
    devid    5 size 7.28TiB used 1.10TiB path /dev/sdd
    devid    6 size 7.28TiB used 1.10TiB path /dev/sdg

[root@big ~]# btrfs fi df /mnt2/big
Data, RAID10: total=3.30TiB, used=3.30TiB
System, RAID10: total=7.88MiB, used=320.00KiB
Metadata, RAID10: total=6.09GiB, used=3.63GiB
GlobalReserve, single: total=512.00MiB, used=0.00B

According to df -h, if I've only used 3.3TiB out of 22TiB, I should have ~18.7TiB Available, but it's reporting to the entire system that I have 13TiB available... 5.7TiB are missing in action

UPDATE: As i've been using it more, it seems to have grown by 2.2TiB. here is my latest df

Filesystem      Size  Used Avail Use% Mounted on
/dev/sdf         22T  6.5T   12T  37% /mnt2/big
0
0

Tl;Dr.

btrfs fi usage /mnt2/mountpoint is the ONLY way to get a somewhat accurate space available estimate.


Free space in Btrfs is very complicated, as such most of the tools normally used to get free space are inaccurate. Unfortunately the best documentation on this issue is found spread around multiple FAQ's in the wiki, starting here https://btrfs.wiki.kernel.org/index.php/FAQ#How_much_free_space_do_I_have.3F

In my case i got:

[root@big ~]# btrfs fi usage /mnt2/big
Overall:
Device size:          43.66TiB
Device allocated:         18.97TiB
Device unallocated:       24.69TiB
Device missing:          0.00B
Used:             12.82TiB
Free (estimated):         15.42TiB  (min: 15.42TiB)
Data ratio:               2.00
Metadata ratio:           2.00
Global reserve:      512.00MiB  (used: 0.00B)

Notice how the Used / Data ratio + Free (estimated) adds up to 21.83TiB, exactly the theoretical maximum space you'd get out of six 8TB drives in RAID 10. Meanwhile df is still reporting my array has 3.5TiB unusable, which is normal.

Filesystem      Size  Used Avail Use% Mounted on
/dev/sdf         22T  6.5T   12T  37% /mnt2/big

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.