Does BTRFS guarantee data consistency on power outages?

Question

As ZFS states exclusively, ZFS is claimed to be invulnerable ZFS accepts that it might be vulnerable to power failures.

I couldn't find such a statement for BTRFS. Is it (or designed/planned to be) durable between power outages?

Answer 1

I've asked the question on #btrfs IRC, they said should be ok if your hw isn't "buggy" where not-"buggy" means your hw has correct flush/barrier semantics.

TL;DR: This means that btrfs is protected against data corruption due to power loss in a similar way as ZFS.

Here is why: The general idea behind ZFS and btrfs is similar. Both use Merkle trees as a data structure. Writes might require multiple blocks on the disk(s) to be updated. The file system is handling this by writing the new data to empty blocks (even if an existing file is being modified, thus it doesn't need to modify blocks that reflect the old state) and building a new updated tree. Once all the heavy lifting is done and data + the updated tree have been written to the disk the head pointer gets updated to the new tree making the change visible.

Here is how things are supposed to behave when writing to a file:

1. Write data to free blocks on the disk.
2. Make a copy of the Merkle tree*, update it according to the changes written in (1).
3. Ask hardware to flush data to disk - hardware writes all pending data.
4. Update head pointer to new Merkle tree.
5. Free old blocks that aren't needed anymore.

If power is lost after (4) the transaction is complete. If power is lost during steps (1) to (3) the file system will come up with the old state (data written in step (1) is lost but the file system is consistent). Note that there is no need to check for file system errors which means the file system is available immediately which is a big advantage (checking large file systems can take very long!).

Here is an example how things can go wrong with "buggy" hardware:

1. Write data to free blocks on the disk.
2. Make a copy of the Merkle tree*, update it according to the changes written in (1).
3. Ask hardware to flush data to disk - hardware confirms completion but doesn't flush all the way (e.g. the data might remain in the disk's write-back cache).
4. Update head pointer to new Merkle tree. This data gets written to disk before other pending data (e.g. because the head of the disk happens to be at the right location).
5. Data written in steps (1) and (2) gets written to disk.
6. Free old blocks that aren't needed anymore.

The file system will become inconsistent if power is lost between (4) and (5) or while performing step (5). As a consequence the Merkle tree and/or the data might only be partially written causing the file system to become inconsistent.

In practice you have to be particularly careful when using RAID controllers. They usually disable the write-back caches on the disk and use their own write-back cache instead. There are two common ways for things to go wrong here:

• The RAID controller's cache is RAM. If it's not backed by a battery or some similar technology you might lose a significant amount of data.
• Some disks don't disable their write-back caches when asked to do so. This is more likely to happen with cheap desktop drives than with server drives designed for RAIDs.

*I'm simplifying things here. It's actually not necessary to copy the whole tree. Only the parts that changed need to be added - the remaining parts can be shared between the old and the new tree.