My main desktop has 64GB of RAM. This seems to cause the dirty write cache to often exceed 10GB during disk-intensive operations such as file copies or network transfers. (I gathered this by observing the
Dirty value in
/proc/meminfo before a transfer - either 0 or very close to it - and during a transfer - over 10GB) I boot from an SSD, but work with large data transfers on external hard drives. One example external drive can probably write at around 100MB/sec average, but I'm connected to my ZFS-array-based NAS via 10Gbit networking, so >400MB/sec bandwidth via the LAN is common.
The problem with this is that if I am multitasking, and if another task executes a
sync operation (one example is I think
dpkg and similar execute a
sync right before completing), it can hang the entire process for several minutes. Additionally, the writing process is still writing, so the sync can take even longer than just the time to empty the dirty cache. This also is evident on the system in other areas - random lags and hangs occur in applications during the transfer.
10GB is quite an insanely large write cache even for a hard drive. It might not be so bad with my NVMe boot SSD that can achieve >1GB/sec writes, but on a relatively slow hard drive it can cause the sync time to exceed 2 minutes assuming no new writes are occurring. If a very large ongoing transfer is occurring I've seen the sync command stall for >10 minutes!
Is there any way to tell the Linux kernel to limit how much data can be outstanding to a given device? (e.g. if I could tell the kernel to only allow, say, 512MB of data to be in cache prior to forcing data to start flushing to disk, that would help the situation immensely.) Failing that, I could maybe live with lowering the systemwide write cache, but also not sure how to do that.