The performance degradation with VM images on BTRFS is not merely due to a lot of file writes; as far as BTRFS is concerned the writes are to the same file. The issue arises from a lot of random writes to the same file. These are writes which occur throughout the file.
In a nutshell, random writes interfere with BTRFS' COW resulting in file fragmentation, which in turn causes read performance degradation. If you have an image file on hand, you can check for file fragmentation with filefrag
.
Note this is not an issue with just VM images. It affects any file written to at random file offsets, such as the SQLite databases used by Firefox.
Solutions/work-arounds
There are a few things you can do about file fragmentation on BTRFS. Choose one of the following:
- Mount the filesystem with
nodatacow
, which disables COW through the entire filesystem. Although really, it avoids using COW unless it absolutely has to (such as creating a snapshot).
- Use
chattr
to disable COW on the directory containing the files in question, and then recreate the files since the chattr
doesn't apply to existing files.
- Periodically run
btrfs fi defrag
against the files in question.
- Mount the filesystem with
autodefrag
to automatically defrag the filesystem.
The first two options disables COW while the last two allows COW but cleans things up after-the-fact. The BTRFS COW and the QEMU COW should not interfere, it'd just be extra slow :)
My personal experience
In my experience with SQLite database files ...
nodatacow
- Haven't tried it.
chattr
- I ended up with fragmented files anyway.
btrfs fi defrag
- I did this for a while to test the concept.
autodefrag
- I've been using this quite successfully.
For QEMU images, I use LVM volumes instead of image files. So I don't deal with the COW issue at all.
Resources
Recommended reading to get a better sense of how COW works on BTRFS.