impact of nodatacow on BTRFS snapshots

Question

I'm running Manjaro ARM with BTRFS on a Raspberry Pi 4 which running off an SD card, which are known to be a problem when performing too much read/write operations.

For that reason I would like to avoid such an operations, without sacrificing the possibility of having live snapshots, the sole reason I'm running BTRFS, so that I can then backup those snapshots somewhere else.

I've recently found that it is possible to have snapshots when mounting a BTRFS subvolume with the nodatacow mount options ^{[1] [2]}, which disables COW. I also know that, for technical reasons, this option must be provided system-wide ^[3].

From my understanding disabling COW means that we will get a behavior of a "normal" filesystem, expect for the metadata ^[4], meaning that the files content always get overwritten on new writes. However, from my interpretation of ^[2], if we are to perform a snapshot the current version of each file is saved, while a new place is allocated for the subsequent writes, which will inherent the nodatacow and overwrite on new writes, meaning that I will only have two versions of my system: the current version and the snapshot (which can be easily restored as usual).

Further disadvantages include the disabling of checksum and compression ^[3].

Is my understanding on the matter at hand correct?

What would be the impact on the size, fragmentation and performance (IO operations) of the filesystem and snapshots when using the nodatacow mount option?

Answer 1

Just came accross this question while researching something unrealated about btrfs. I think I can answer some of your questions:

How does nodatacow affect actual fragmentation and performance on sd-cards?

Not much. The reason is that with reasonable write sizes (as close to the erase block size as practically possible) the fs level COW will just replace the block device level COW that would happen otherwise.

Key to that is of course knowing the actual erase block size of the sd-card. But even if your filesystem block size is smaller than the erase block size of the media (as will often be the case) you can still expect reasonably good performance.

Notice that both, the block device level COW / dynamic wear-leveling (done by the controller chip in the sd-card) and the fs level COW, can create fragmentation, but this is generally not considered a problem with sd-cards and SSDs because they lack seek times.

Why is the erase block size important for in-place writes?

From the raspi forums: https://forums.raspberrypi.com/viewtopic.php?t=11258

As you can see the erase block size can be quite large (up to 16M in the examples from that thread). That means when you do an in-place / nocow write to the sd-card, the card might have to erase 16M and then write up to 16M regardless of how much data you wrote. This is a worst case scenario though. The reason why this might happen is because the controllers in sd-cards and SSDs try to do their own version of COW to help with wear leveling where they remap the blocks you write to, to a relatively less used and previously erased block of the flash medium.

How does the size on disk change because of nodatacow?

Again, not much. Deletions are done async for btrfs anyway, meaning that just because you deleted a file does not mean the space on disk is usable for something else right away. Instead another process is checking how often the data is referenced in the filesystem (normal volume and snapshots alike) and removes it when the count reaches 0. This is identical for datacow and nodatacow. The only difference you might find is if you have a non-snapshotted file that is very busy with writes and at the same time you are checking the remaining free space: you might notice a larger lag for the disk space to go up to the correct value because the cowed and busy file might temporarily use more space on the media than it actually contains.

What i recommend

use the noatime mount option to minimize unnecessary writes, make sure the partition is properly aligned to write block size and even better would be if it is also aligned to erase block size (to avoid write amplification) and then just try it out. All these other optimizations for SSDs and flash media are seldom necessary if you have fairly standard fs use (busy databases and VM images are the exception).