I would like to use the free memory for write cache. Linux does that automatically, but it has some weird flushing algorithm that I would like tuned for my use.
I have recently used
ddrescue to copy a disk. It reads a block and writes the block to disk.
What I would like is to have the reading read as much as possible until there is no more free then block until some space is freed, while at the same time the writing writes as fast as possible freeing up memory.
So if the writing disk is faster than the reading disk no memory will be used, and if the writing disk is slower than the reading disk then all the memory will stay used until the copying is complete.
In both scenarios the memory usage remains constant (unless other programs take some of the free memory).
But what I see is this:
The reading fills up the cache (while little to no writing is done) and when the cache hits a limit (or after er period of time) then the reading is blocked while all the cache is written to disk. It is almost as if a troll is running 'sync' every now and then.
I have tried changing the /proc/sys/vm/dirty_* to no avail: The only change I have been able to provoke is how tall the tops are. So an answer that mentions /proc/sys/vm/dirty_* will have to provide tested values that measurably flattens out the saw tooth graph.
A tuning that will tell Linux to start writing ASAP but never block reading or writing if there is free space, will most likely suit my purpose.
I am aware of tools like
mbuffer. They will flatten out the graph, but they are not dynamic (i.e. releasing memory when the free mem is low) and they do not allow for seeks, so they will not work with programs like
ddrescue (which has to seek back and forth).