I use rsync 3.1.1 to keep in sync two discs, one of which is on the network and mounted as a samba share on
/mnt/ROUTER_WD_2TB/. This is the source, with a speed of 30-40MB/s maximum due to network limitations. The other, the destination, is mounted locally (peak 110MB/s) and is
I use the following rsync command:
rsync -haAXi --quiet --append-verify --delete /mnt/ROUTER_WD_2TB/* /mnt/BACKUP_HITACHI_2TB/.
The disks contain many files, most of them small.
The problem is that rsync takes quite a long time (10-20m) before starting to move any files, I guess because it has to compute file lists for a very large number of small files. During this period, the newtwork utilization sits at a low 200-500KB/s, while when transferring files the speed is about 40MB/s.
It usually happens that rsync takes about 15m to finally find something it has to copy, then takes 5 seconds to copy it, then continues checking for some other files to copy for another 5 minutes. All in all, a 5 second file copy ends up lasting 20 minutes!
Is there any measure I could take to avoid this long period prior to copying the files, other than excluding folders? Any kind of "cache" I can implement so that rsync does not have to rebuild all the file list from scratch?