How to debug/audit which devices cache writes?

Question

I store documents on btrfs partition built upon sparse dm-crypt device located on ext4 main partition on physical harddrive.

When kernel panics (and this happens on daily basis on my ASUS P53E with 3.6 kernel :-( ) I loose recently modified files (always the files' contents got replaced with zeros).

One way of preventing damage to files it is to disable the write cache.

For this I would need to disable write caching for the btrfs partition, the dm-crypt device that backs the partition, and for the sparse file where dm-crypt device lives.

1. How can I check the cache-write status for the drives?
2. How can I disable it?

I use Mint 13 Maya with 3.6.8 mainline kernel.

Answer 1

I'm not sure the disk drives write cache is going to fix the issue for you as it sounds like you are using a loop device. So there is still the page cache/file in between your Btrfs filesystem and the actual disk. The same type of issue exists for journaling filesystems detailed here for loop-AES. So when data is synced to your loop device, it may not be on real disk yet just in a cache waiting to be re ordered and written out.

ext4 doesn't support the sync mount option ext2/3 did to disable caching. due to the layers in between I'm not sure even that would get you an effective recovery. I don't know enough about the internals unfortunately, at least more data would make it to disc.

In the same way you might be able to limit the issue by tuning the page cache so the system writes out to disk more often. The linux page cache reports it's values in /proc/meminfo under

"Dirty" - for pages that are currently dirty
"Writeback" - for dirty pages that are being written out to disk.

There are files in /proc/sys/vm/ that report status and control the flush threads that write data back to disk.

You could put a small value (greater than 8096 or 2 pages) in /proc/sys/vm/dirty_background_bytes to make the background pdflush process run more actively or /proc/sys/vm/dirty_bytes to make a process trigger flush more actively (For a performance penalty, or at least more disk writes overall).

I think it's unlikely the hardware write cache is where your main issue lies. If you were only going direct to the device via dm-crypt then I'd look there first. In any case IDE and SATA write caches can be disabled with hdparm -W0 /dev/xdx.

Also as your using a, technically, experimental file system in a more edge case way than most you might have better luck with a more mature fs where people have stumbled into the issues already. If you need Btrfs, the best bet would be to do the encryption to a physical partition.

Answer 2

So far I have found a way of setting write-cache behavior for physical partition (/dev/sda6 in my case):

sudo hdparm -W /dev/sda6 for inspecting the write behavior and sudo hdparm -W0 /dev/sda6 for disabling the write-cache.

But it is an overkill in my situation; I want only to disable write-cache for the file backing the dm-crypt, not the entire partition.