How can you configure Linux to both read from a local disk/filesystem and write to a network share at the same time, as opposed to reading while no data is going over the network, then sending that data over the network while the local disk is idle?
It is much faster to read and write at the same time instead of only performing one operation and then the other in an alternating fashion.
I am moving a large amount of data from local disks on a Linux machine to a NAS device.
I am using
rsync to basically copy
/mnt/nas, which is a CIFS mount.
It started off well, reading at 100MB/sec and writing to the NAS at 100MB/sec (limit of gigabit network), with both reading and writing happening simultaneously.
However now, a few hours later, I am finding that it is reading from the local disk, then stopping the read while it writes to the NAS, then when there is no more data to write to the NAS, it resumes reading from the disk again. The network is idle while the disk is being read, and the disk is idle while the network is in use.
Needless to say, reading 200MB then writing 200MB takes much longer than reading and writing that 200MB at the same time.
How can I configure the kernel such that it sticks to the earlier behaviour of reading and writing at the same time, rather than alternating between reading then writing, performing only one operation at a time?
Some observations: When the local disk reads at 100+MB/sec everything seems to happen in parallel just fine, but once the disk slows down (seems to be going at only 20MB/sec now for some reason) that's when this read/write switching seems to happen.
I can also run
sync manually every few seconds to get the writes happening in parallel with the reads (though obviously at the reduced speeds) however putting
sync in a
while loop so that it runs every five seconds doesn't seem like the right solution...
The kernel seems to cache about 1GB of data and then write it out over the network as fast as possible - which is fine - I just don't understand why the slow disk needs to stop being read while the data is being sent out over the network.