On a fresh Ubuntu 18.04 system I decided to give Deja-Dup a go. It's just a GUI for Duplicity (which uses rsync). About 850GB of data needed to be backed up. Source SSD was NVMe. Destination SSD was SATA. Initial (full) backup took about 7 hours.
The next day I did nothing more than check mail and install an application — which added ~230MB — before running Deja-Dup again.
Deja-Dup ran for ~39 minutes loading a single core at 35–70% for the entire duration.
Duplicity was invoked three times:
- The Scanning phase pegged a core at 100% for 18 minutes.
- The Backing Up phase pegged a core at 100% for 16 minutes.
- The Verifying phase pegged a core at 100% for 5 minutes.
This is on a new computer with plenty of RAM. The backup was not encrypted.
Now, I expect initial (full) backups to take a while, and that's fine. What I don't expect is for ~230MB of new data to take ~39 minutes to be backed up (and consume over a collective core-hour of CPU time).
Is something wrong or broken? Should incremental backups of a couple of hundred megabytes take that much time to perform? I was expecting something under 5 minutes, not ~39. Why is it taking so long?
(If I had a spare 1TB SATA SSD I'd hook it up and just rsync the data straight across to see if that is noticeably faster — but unfortunately I do not.)
Update 1: I ran a manual backup after negligible changes (a few KB of new mail) and the time taken was the same (~39 minutes). Thus the time taken seems to have little to do with the amount of new data that needs to be backed up.
Update 2: Monitoring with iotop revealed that the Scanning phase reads 7.36GB from the drive. Now that's obviously not the whole 850GB... but it's not far removed from the number you get if you multiply the number of files on the source drive (1174000) by the block/cluster size (4096) — i.e. 4.81GB. Not sure how to account for the remaining 2.5GB though, if this were the case.