0

I would like to backup some directory on my laptop to an external hard drive.

The destination in the external hard drive already has an older backup of the directory on my laptop.

  1. I was wondering how I can update the backup on my external hard drive? For example, will the following command work as expected?

    rsync -a --delete  /path/to/source/dir  /path/to/dest/dir
    

    does it work the same as I first rm -r /path/to/dest/dir/*, and then rsync -a /path/to/source/dir /path/to/dest/dir?

  2. How can I make rsync create the new backup faster without making mistakes, by taking advantage of the older backup on the destination (some files might have been changed since the last backup, but more files haven't)? Another reason for taking advantage of the older backup is that my external hard drive doesn't have enough space to hold a new backup without removing anything.

Thanks.

5
  • Do you just want to run a backup again with only files and directories which have changed since the last backup? May 14, 2018 at 2:27
  • Not sure what you mean. I want to backup all the current files and directories in an efficient way
    – Tim
    May 14, 2018 at 2:42
  • If you just want to back up that directory to an external HD then you can just cp -rv -f /laptop/directory /external/hdd/. That will overwrite any older files whether they've changed or not without prompting. May 14, 2018 at 2:58
  • But cp is not efficient, and can't take advantage of the older backup.
    – Tim
    May 14, 2018 at 4:08
  • Then use a proper backup software such as borgbackup.
    – Kusalananda
    May 14, 2018 at 8:15

2 Answers 2

2

Yes, rsync is a good solution, but better use (note the ending slashes):

rsync -a --delete  /path/to/source/dir/  /path/to/dest/dir/

To quote man rsync:

A trailing slash on the source changes this behavior to avoid creating an additional directory level at the destination. You can think of a trailing / on a source as meaning "copy the contents of this directory" as opposed to "copy the directory by name"[...]

With --delete you get mirroring (exact same files), without it you get additive backup (files deleted in the source are kept in the copy).

The danger of --delete is that if you target the wrong directory, many files can be deleted, so best restrict its use to well-tested scripts, or use -n/--dry-run to check first what would happen.

3
  • Or you could include --max-delete 50 to (say) limit the number of deleted files to 50 per invocation. May 14, 2018 at 20:45
  • Yes, but as far as my experiments go it still erases 50 files and only stops when it hits the 51st... And if you lower the threshold --delete is no longer very useful.
    – xenoid
    May 14, 2018 at 22:05
  • Of course. It's a safety valve on --delete "if you target the wrong directory". May 14, 2018 at 22:54
1
  1. How can I make rsync create the new backup faster without making mistakes, by taking advantage of the older backup on the destination (some files might have been changed since the last backup, but more files haven't)?

Backing up with rsync does this by default, if you pass -a (and get the paths right, as per @xenoid's answer). And you want to pass -a anyway, so it's all good. The specific reason is a bit subtle:

Rsync finds files that need to be transferred using a "quick check" algorithm (by default) that looks for files that have changed in size or in last-modified time.

If rsync thinks a file doesn't need to be transferred, then it doesn't transfer it. And this works because -a implies this option:

    -t, --times                 preserve modification times

so the last-modified time will be the same on the backup file as it is on the original file.

without making mistakes

Just don't fake rsync out by modifying the contents of files and then resetting their last-modified time :-). Otherwise that file will be overlooked.

I haven't heard of this being a problem, so I wouldn't worry about it. It's common to use rsync for basic backups in this way.

I expect it's more likely that you'd run into other limitations. I.e. if you back up $HOME without using filesystem snapshots, then your firefox profile is probably being modified at the same time as the backup runs, and you will not get a consistent working firefox profile in your backup. Or that you'll want to recover a deleted/overwritten file, but since you're not keeping any older backups, it could already have been deleted/overwritten on the backup.

10
  • Thanks. I wonder if it is faster or slower using -u than not using -u? Does -u not make any mistake in determining which file to actually transfer? Also see unix.stackexchange.com/questions/443829/…
    – Tim
    May 15, 2018 at 2:12
  • @Time It will not be faster. This is why I say it is subtle :-) - if you do not use -a / -t, then rsync will copy all files from scratch, unless you use -u. Do you see why? I would avoid using -u when you don't need to, as there are more cases where it would make a mistake. It can even skip files that are different sizes, if the source file has been dated backwards in time.
    – sourcejedi
    May 15, 2018 at 8:12
  • 1
    Bad idea to use -u for this... If you ever roll back a file to something of the same size with an earlier date, your backup will keep the most recent version, which is no longer the one you have.
    – xenoid
    May 15, 2018 at 8:44
  • @xenoid no, worse. -u says "This forces rsync to skip any files which exist on the destination and have a modified time that is newer than the source file." It doesn't matter if they are a different size or not.
    – sourcejedi
    May 15, 2018 at 9:55
  • Thanks. If -u decides a file required for transfer, does the quick check algorithm also decides the same?
    – Tim
    May 15, 2018 at 12:35

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .