First, let me address your concerns.

I'm copying from one slow drive, to another (different) slow drive. Blocking for reads on one drive until I write to another leaves a lot of room for improvement.

I developed a small cpp/h file that does this for me. It improved speed in my use case by ~40%. https://github.com/pauldotknopf/AsyncStreamCopyCPP/blob/master/examples/main.cpp

I'd like to know if there is something equivalent, but as a command. Preferably one that is similar to how the "cp" command operates.

# Use async read/write threads and shared/re-used buffers between the two.
cp /slowdrive1/file1.txt /slowdrive2/file2.txt
  • Did you flush the system buffers before testing the speed? echo 3 > /proc/sys/vm/drop_caches – wurtel Feb 21 '19 at 13:30
  • // pretend this function takes a while! - did you test without pretending? The Linux kernel implements async readahead. – sourcejedi Feb 21 '19 at 13:42
  • Relevant: Multithreaded cp on linux?. – Stephen Kitt Feb 21 '19 at 16:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.