btrfs — Is it dangerous to defragment subvolume which has readonly snapshots?

Question

If you open the defragment section of btrfs-filesystem(8), you will see the following ominous inscription left by the developers:

Warning: Defragmenting with Linux kernel versions < 3.9 or ≥ 3.14-rc2 as well as with Linux stable kernel versions ≥ 3.10.31, ≥ 3.12.12 or ≥ 3.13.4 will break up the ref-links of COW data (for example files copied with cp --reflink, snapshots or de-duplicated data). This may cause considerable increase of space usage depending on the broken up ref-links.

That sounds terrible. A selling point of btrfs is its ability to create snapshots without copying everything. I mostly create readonly snapshots.

Do the files of readonly snapshots also count as “COW-data” or will parent subvolume deduplication survive without making disk space bloat?

Answer 1

Btrfs defrag won't break all reflinks

Just the particular instances you point it at. So, if you have subvolume A, and snapshots S1 and S2 of that subvolume A, then running defrag on just subvolume A will break the reflinks between it and the snapshots, but S1 and S2 will still share any data they were originally with each other. If you then take a third snapshot of A, it will share data with A, but not with S1 or S2 (because A is no longer sharing data with S1 or S2).

Given this behavior, you have in turn three potential cases when talking about persistent snapshots:

1. You care about minimizing space used, but aren't as worried about performance.
   In this case, the only option is to not run defrag at all.
2. You care about performance, but not space usage. In this case, defragment everything.
3. You care about both space usage and performance. In this balanced case, I would personally suggest defragmenting only the source subvolume (so only subvolume A in the above explanation), and doing so on a schedule that coincides with snapshot rotation. The idea is to defragment just before you take a snapshot, and at a frequency that gives a good balance between space usage and performance. As a general rule, if you take this route, start by doing the defrag on either a monthly basis if you're doing daily or weekly snapshots, or with every fourth snapshot if not, and then adjust the interval based on how that impacts your space usage.

Source: Btrfs mailinglist, as referenced by Spacedog.

Btrfs defragment readonly snapshot

From my trial and error experience, btrfs defragmenting snapshots (to use the new zstd compression) resulted in 100% Exclusive and 0.00 bytes of shared data.

Before btrfs defragment:

# btrfs filesystem du -s /mnt/btrfs/Backups.backupdb/d2/readonly-snapshot/
     Total   Exclusive  Set shared  Filename
   1.41GiB     6.27MiB     1.41GiB  /mnt/btrfs/Backups.backupdb/d2/readonly-snapshot/

After btrfs defragment:

# btrfs filesystem du -s /mnt/btrfs/Backups.backupdb/d2/readonly-snapshot/
     Total   Exclusive  Set shared  Filename
   1.42GiB     1.42GiB       0.00B  /mnt/btrfs/Backups.backupdb/d2/readonly-snapshot/

Shared data goes down to 0.00B

Answer 2

Yes, files in a readonly snapshot count as COW-data and will contribute to disk space bloat caused by defragmenting.

When defragmentation happens, data is copied from the old extents into fewer new extents. The new extents are distinct from the old extents. All other copies of the file (in snapshots, for instance) still point to the old extents. Therefore, you have data bloat.

There's a long thread about defragmenting on the mailing list starting here that has some interesting points.