btrfs performance

Question

I decided to mess around with btrfs and found some weird performance anomalies. The tests I've done are with with two external hard drives one formatted with ext4 the other btrfs.

ext4 performance result is as follows:

$ sudo dd if=/dev/zero of=/media/Backup/out.img count=1024 bs=1024k
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 24.1959 s, 44.4 MB/s
$ sudo dd if=/dev/zero of=/media/Backup/out.img count=1024 bs=1024k
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 24.1619 s, 44.4 MB/s

The 44.4MB/s is typical for me, but now btrfs (note: I always seem to get that high 3.1GB/s IO right after deleting the file):

$ sudo dd if=/dev/zero of=/tmp/test/out.img count=1024 bs=1024k
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 0.347933 s, 3.1 GB/s
$ sudo dd if=/dev/zero of=/tmp/test/out.img count=1024 bs=1024k
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 1.1732 s, 915 MB/s
$ sudo dd if=/dev/zero of=/tmp/test/out.img count=1024 bs=1024k
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 1.21539 s, 883 MB/s
$ sudo dd if=/dev/zero of=/tmp/test/out.img count=1024 bs=1024k
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 1.25093 s, 858 MB/s

My question is what's going on here with btrfs? Why is the performance IO much higher? I've even ran a sync right after and it completes instantly.

I am running the latest stable Kernel.

Answer 1

I also advise you to use other tools for benchmarking I/O than dd. Brtfs is not a traditional filesystem and being a copy-on-write and transactional filesystem, most of the operations are done in memory and not directly on the hdds.

So when you issue the deletes and recreate the file, I believe it will just reuse what is has in memory. Let's not forget that you are creating a file initialized only with zeros. BRTFS only writes to disk after a timeout or when enough pages haven been reached.

I advise you to take a look at this document (PDF). It provides a deeper explanation and in more detail what might be the explanation for you question.

Answer 2

This behaviour is because BTRFS supports sparse files. Basically, any sufficiently long string of "empty space" (0 bits) will be stored as metadata saying "from this point to this other point is all 0s" instead of actually writing the 0s to disk. In this case, as your input stream is /dev/zero, your entire file is 0s, and thusly its actual size on disk is just a tiny bit of metadata, more or less.

Not sure why you're seeing that much apparent increase in throughput after deleting a file first, though.