22

I usually do rsync -n before doing an actual rsync. The thing is, how could I automate the second run to reuse exactly the list generated with --dry-run?

I mean, if I run rsync -n then I get a simulation of what rsync would do, my idea would be to take that output, process it then somehow make rsync only --include-from that output, so the full list has to be calculated only once.

If this doesn't even make sense, please, let me know.

By the way, I tend to use rsync -hva --progress --stats --delete --exclude-from "$EXCLUDEFILE". If you also have a suggestion on what options would you use, please, let me know. I'm trying to duplicate the contents of an external hard drive (that only contains images, videos and documents) to another external hard drive.

  • You could try redirecting the output from the --dry-run command into a file (e.g., rsync --dry-run -avz SOURCE DEST > include.txt), removing all lines that aren't files/paths from include.txt, and running rsync --include-from include.txt. – ThatGuy Feb 9 '16 at 1:06
  • That last command should be rsync --include-from include.txt --exclude=* SOURCE DEST. See stackoverflow.com/a/19296415/1765910 for details. – ThatGuy Feb 9 '16 at 1:18
  • Yes, that's the idea, but I don't know how to actually do that (plus I would like the approval from someone that actually knows what (s)he's doing). I just use rsync but don't know anything from shell scripts / bash… – Manuel Feb 9 '16 at 16:34
17

It is certainly possible to use the file list generated during a dry run as an include file. Ideally, however, the extra lines at the top and bottom of the dry-run output should be deleted.

Example output:

sending incremental file list
[LIST OF FILES]

sent 226 bytes  received 34 bytes  520.00 bytes/sec
total size is 648,373,274  speedup is 2,493,743.36 (DRY RUN)

To remove these lines, pipe the rsync output into head and into tail. Then, save the processed output to an include file:

rsync --dry-run -avz source/ destination/ | head --lines=-3 | tail --lines=+3 > include.txt

Finally, tell rsync to use the generated include file (adding additional options as desired):

rsync --include-from=include.txt --exclude=* source/ dest/

  • The problem with this is that it doesn't cope with files that should be deleted. And is that method of tail/head robust enough to ensure having only the list of files? – Manuel Feb 10 '16 at 22:18
  • Adding --delete causes files deleted from the source directories to be deleted from the destination directories as well, at least on the computer I tried it on. You can leave out the tail/head part entirely. rsync interprets each line in the include file as a file name and simply ignores non-existing files, so nothing will happen if you keep the extra lines (unless you happen to have files named "sending incremental file list", etc., in which case these files will be copied as well). – ThatGuy Feb 11 '16 at 1:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.