I know filesystem supporters war is behind the corner and there are plenty of fine tuning guide for each filesystem in the wild, so please stick to the specifications: I'm asking what would you suggest to go FAST (R/W) with LARGE files, while reliability is not prioritary.

Real world scenario: copy of large VMs images (2++ TB) from backup target storage to local (sata) storage.

Already tried: ZFS(onlinux) on backup target, but pools are getting fragmented and heavily slowed down I/O (both R/W) after a few months of work.

Next in row for testing: XFS over LVM for target storage, seems like the obvious choice... Or not? Any good advice about that? Thank you!


Ok, so i think i figured it out. Speaking about performance, BTRFS did the best job.

I tried multiple linux based NAS, made all with the same hardware but with different filesystem.

I used FIO (http://freecode.com/projects/fio) to run benchmarks as suggested by a Veeam Software white paper since I'm using their backup product (https://www.veeam.com/wp-veeam-backup-replication-v8-designing-planning-backup-repository-performance.html) and these were the results, using consumer level hard disks (SATA WD RED 5400rpm 64mb cache). I believe this results to be solid and inspiring for further tests, just do some math for pretty self-explanatory percentages.

900GB non-zero generated data:
ZFS pool 2x1TB:   bw=27300KB/s, iops=53, runt=3600053msec
BTRFS vol 2x1TB:  bw=41541KB/s, iops=81, runt=3600033msec
BTRFS vol 2x1TB*: bw=48223KB/s, iops=94, runt=3600024msec

1800GB non-zero generated data:
ZFS pool 2x1TB:   bw=19528KB/s, iops=38, runt=3600077msec
BTRFS vol 2x1TB:  bw=31593KB/s, iops=61, runt=3600031msec
BTRFS vol 2x1TB*: bw=40911KB/s, iops=79, runt=3600023msec

5x10GB seq zeroed files with free space < 10%
ZFS pool 2x1TB:   111 MB/s 114 MB/s 112 MB/s 112 MB/s 124 MB/s
BTRFS vol 2x1TB:  155 MB/s 162 MB/s 145 MB/s 126 MB/s 95,4 MB/s
BTRFS vol 2x1TB*: 293 MB/s 296 MB/s 290 MB/s 283 MB/s 281 MB/s (LZO affects)

*Mount options autodefrag, compress=LZO, nodatacow.

**Ext4 and XFS resulted 20% or more inefficient compared even to zfs, that was the minimum entry level for my benchmark.

Please note this is just a contribute to "real life" benchmarking, I'm not stating X is always better than Y, but in my specific case of course it is.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.