3

The arch wiki recommends that images have Copy-On-Write disabled on the directory when using btfs. I do see that it would be a good idea if you had a lot of file read/writes. This question explores this idea. I know that VMWare will grow into different files and will write snapshots out which could be problematic when using Copy-On-Write.

For QEMU the file is left open for the VMs existance so there is a potential problem in the write after the VM is shutdown, but I would see slow I/O after a VM is shutdown not being an issue. What pitfalls would I be avoiding by performing this step for QEMU.

Additionally: I have assumed that the image is raw for this question. Is there a possible stability issue as qcow2 already has copy-on write.

| improve this question | |
8

The performance degradation with VM images on BTRFS is not merely due to a lot of file writes; as far as BTRFS is concerned the writes are to the same file. The issue arises from a lot of random writes to the same file. These are writes which occur throughout the file.

In a nutshell, random writes interfere with BTRFS' COW resulting in file fragmentation, which in turn causes read performance degradation. If you have an image file on hand, you can check for file fragmentation with filefrag.

Note this is not an issue with just VM images. It affects any file written to at random file offsets, such as the SQLite databases used by Firefox.

Solutions/work-arounds

There are a few things you can do about file fragmentation on BTRFS. Choose one of the following:

  1. Mount the filesystem with nodatacow, which disables COW through the entire filesystem. Although really, it avoids using COW unless it absolutely has to (such as creating a snapshot).
  2. Use chattr to disable COW on the directory containing the files in question, and then recreate the files since the chattr doesn't apply to existing files.
  3. Periodically run btrfs fi defrag against the files in question.
  4. Mount the filesystem with autodefrag to automatically defrag the filesystem.

The first two options disables COW while the last two allows COW but cleans things up after-the-fact. The BTRFS COW and the QEMU COW should not interfere, it'd just be extra slow :)

My personal experience

In my experience with SQLite database files ...

  1. nodatacow - Haven't tried it.
  2. chattr - I ended up with fragmented files anyway.
  3. btrfs fi defrag - I did this for a while to test the concept.
  4. autodefrag - I've been using this quite successfully.

For QEMU images, I use LVM volumes instead of image files. So I don't deal with the COW issue at all.

Resources

Recommended reading to get a better sense of how COW works on BTRFS.

| improve this answer | |
  • The current documentation of btrfs mount options says this about "autodefrag": "Not well suited for large database workloads." When you said you used it successfully, was that for database workloads? To me the comment suggests that it may not be such a great idea to use autodefrag with databases, maybe even using btrfs for databases altogether. Since they do journaling on their own it might actually be a good idea to use ext4 with journal switched off rather than btrfs. – Robert Klemme Jul 21 '18 at 14:27
  • 1
    While I do have some databases, in particular Firefox's SQLite, I do not, nor would I, use BTRFS to store databases. At some point after answering this question I stopped using autodefrag and switched to a job which runs btrfs fi defrag because I noticed autodefrag was hurting my throughput; And that was just on everyday Desktop workloads. As for ext4 without journaling, it depends on what's being journaled. I recommend keeping meta-data journaling so that if the OS goes down an extensive fsck won't be needed to bring it back up. – Emmanuel Rosa Jul 21 '18 at 14:47
  • The usual tradeoffs apply: you pay for the fast recovery with throughput during operation. Often different file systems and mount options are chosen for the volume that stores TX log and the volume(s) that store data files (not only for reliability reasons but also access patterns and volumes). PostgreSQL for example makes sure WAL writes are persisted (using fdatasync()) and thus would not require file system level journal. – Robert Klemme Jul 21 '18 at 15:28
  • 4
    Please edit your answer to point out that the defragment process will break most reflinks: unix.stackexchange.com/q/400225/65781 so it's not an option (manual or automatic) at all with lots of snapshots at hand. – ceremcem Oct 30 '18 at 8:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.