2

I'm working on a bash script that duplicates files from one "source" pendrvie to multiple others (also formatting drives to FAT32 if they don't mount).

I'm copying the files with rsync.

I've noticed that rsync finishes pretty quickly, sometimes reporting ridiculous write speeds like 500 MB/s, but usually staying at 15 MB/s.

The catch is that afterwards - unmounting the drives can take as long as minutes, and I figured out that it's because of the system disk cache, and asynchronous I/O.

I wonder if mounting the drives in "sync" instead of the default "async" mode would allow the rsync processes to show real writing speeds, and spare the lengthly unmouning, that can't be easily monitored.

I guess it'd be easier to track the real progress and produce a sensible ETA, but I'm afraid I'll hurt the performance somehow, and disable some cool features (for example forcing each copy to be re-read from the source drive instead of the RAM cache).

What benefits and drawbacks can sync produce over async is this particular usecase?

  • It depends on how many files you're copying and how large they are. With many small files sync would hurt a lot. With a few large files it wouldn't make much difference. – Satō Katsura Mar 3 '17 at 15:58
  • @SatoKatsura you ought to expand that a little, it'd make for a good answer (and post it as an answer, of course). – derobert Mar 3 '17 at 16:09
1

Yes, it's because of the disk cache: Before unmounting, all pending writes have to be finished.

Mounting it with sync won't improve the total time needed: What needs to be written needs to be written, no matter if rsync already exited or not. In fact, mounting with sync will likely force some blocks to be unnecessarily written several times (e.g. directories), and that will slightly decrease the lifespan of the stick, and it will also increase total time needed.

If you want to monitor the actual writing speed, just use a different tool like atop or iotop, which will show the writing speed for a particular device.

Monitoring or scripting the complete transaction is also easy: Just wait until the final umount returns, and never mind when rsync finishes.

So I'd advise against mounting with sync.

  • Not only that, but mounting with sync could severely hurt a pendrive's lifetime, as cheap flash media has limited numbers of writes. – Gilles Mar 3 '17 at 23:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.