6

When writing content to removable devices on Linux (USB sticks/HDDs, SD cards, etc), I often see incredible write speeds in the first few seconds, sometimes in the order of GB/s (filling of the write buffer) followed by several minutes of quiet (buffer actually being written to the device).

It's misleading. It makes it hard to judge the actual write speed. It's annoying (can't interrupt a program while it's flushing buffers). It can cause FS damage, and at the very least unwritten content when the device is unplugged while it's still being written.

The write buffer appears to be over a GB in my case. Is there a way to control it? I'd like to reduce it to a more sensible value.

Note: This is not about filesystem cache, this effect also appears on raw devices

Note: I do not want to disable write buffers entirely

Thanks!

  • 1
    never unplug the removable device without umounting an fs (in GUI use 'eject' button on mounted fs node in the tree shown by file manager). The umount effectively completes any pending IO and flushes any outstanding fs cashes. – Serge Jun 25 '16 at 13:21
  • (and when writing the device directly e.g. with dd, you must make sure to use something like sync before you remove the device) – sourcejedi Oct 9 '18 at 9:34
2

I found the answer. 64-bit Linux maintains a large write buffer (20% of available memory!) by default. (Interestingly, 32-bit Linux limits itself to at most 180MB) To change the dirty buffer size to e.g. 200MB, one can use

echo 200000000 > /proc/sys/vm/dirty_bytes

OR to use a percentage of RAM, e.g. 1%:

echo 1 > /proc/sys/vm/dirty_ratio

More information: https://lwn.net/Articles/572911/

  • Both 32-bit and 64-bit Linux kernels use default dirty_ratio of 20. However, 32-bit kernel has maximum available kernel RAM limit of 1 GB due to limits of 32-bit x86 architecture. As a result, maximum default write cache size on 32-bit system is around 180-200 MB depending on hardware (at least PCI devices with DMA access will reduce usable RAM limit). – Mikko Rantalainen May 17 '17 at 7:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.