I have CCTV server that is backed up daily to a remote ftp server using an lftp command. The CCTV server saves videos to a new folder each day, the backup runs once a day at 1am, so each backup only affects 2 folders. After 28 days the local copies are all deleted.
The command I currently use is:
mirror --reverse --use-cache --allow-chown --allow-suid --no-umask --verbose
The file transfer rate when moving files has been consistent for 2 years, but the wait between folders is slowly increasing. This means that while the file transfer rate is 1Mb/s if you take the wall clock time that backing up 2GB takes each day, the average speed after 2 years is now down to 0.5Mb/s.
Is anything in my command causing the process to bloat?
--use-cache for example?
Could I have the mirror command to run one thread per folder so that it can get on with loading folder contents while it's uploading other files?