Recently I needed to delete a large number of files (over 1 million) and I read that doing:

rsync -av --delete `mktemp -d`/ ~/source && rmdir ~/source

Was one of the most optimized ways to do that, and I can vouch that it's faster than rm -rf.

I'm not an expert on the matter, but from my understanding the reason for rsync performance has something to do the way it lists files (LIFO instead of FIFO I suppose). Now, the problem is, I also need to to move a large number of files in a efficient way. After searching a bit, I found this:

rsync -av --ignore-existing --remove-source-files ~/source ~/destination

While this deletes all the moved files in ~/source, the directories remain there. Since I have a "round-robin"-like directory structure the number of files/directories is very close to 1, so I am forced to run the first command again to get rid of the directory entirely:

rsync -av --ignore-existing --remove-source-files ~/source ~/destination && \
rsync -av --delete `mktemp -d`/ ~/source && rmdir ~/source

A straight mv would finish virtually instantly, but my ~/destination directory has files that should be kept, so mv is not an option. I found the --prune-empty-dirs and --force rsync options, but neither seem to work as I would expect:

--force                 force deletion of directories even if not empty
--prune-empty-dirs      prune empty directory chains from the file-list
--remove-source-files   sender removes synchronized files (non-dirs)

Is there a way to mimic a move with rsync in one go?

up vote 7 down vote accepted

I found this thread over on stackoverflow titled: Deleting folders with rsync “move”?, which is asking essentially the same question. One of the answers suggested doing the rsync in 2 commands since it appears there isn't a single command that can accomplish the move/remove of the files and the source directories.

$ rsync -av --ignore-existing --remove-source-files source/ destination/ && \
  rsync -av --delete `mktemp -d`/ source/ && rmdir source/

Alternatively you can do it using this command:

$ rsync -axvvES --remove-source-files source_directory /destination/ && \
  rm -rf source_directory

Not ideal but does the job.

  • Yes, it was me who answered that question (and the same "solution" is also on this question). =P – Alix Axel Jun 29 '13 at 3:34
  • @AlixAxel - Ha, sorry I didn't even notice you answered that one. Oh well. Shall I delete this answer then? – slm Jun 29 '13 at 8:19
  • 3
    Using rsync to prune directories doesn't feel right and there is always danger with rm -rf. I'd recommend the 2nd step to be: find source/ -d -type d -exec rmdir {} \; – zany Dec 13 '13 at 14:10
  • 2
    I just feel compelled to add, never use -delete and --remove-source-files in a single rsync command (the above examples are fine, this is tangential). If you interrupt then re-execute what I'm describing, you'll lose transferred files. Which I've done before :( – user7000 Jul 16 '15 at 19:03
  • When cleaning up with find, I prefer this: find source/ -type d -empty -delete It gives me a weird sense of reassurance that it won't just clobber everything. – greyfade Apr 5 at 22:58

From zany's comment to slm's answer (Move files and delete directories with rsync?) I would recommend these 2 commands as an answer:

rsync -av --ignore-existing --remove-source-files source/ destination/ && \
find source/ -depth -type d  -empty -exec rmdir "{}" \;

The advantage is, like zany said, there is still some danger involved in using rm -rf if you don't get it right or for beginners.

I added 2 options, -depth and -empty and while I am not sure if this is really necessary, it makes the 2nd command more portable for other situations and even more safe (it still does the right thing if some directories are not empty and starts removing from the deepest point in a directory tree)

  • why not just -delete instead of -exec rmdir {} \;? – flying sheep Jan 10 '17 at 9:24
  • @flying_sheep see user7000's comment in sim's answer: when rsync ist interrupted you could loose files – mit Feb 8 '17 at 15:50
  • When rsync is interrupted, the find never starts or not? So the -delete in a second rsync shouldn't be a problem. But -delete also remove files, instead rmdir never delete files, only directories. Both (-delete and rmdir) check the directories if they are empty before deleting. – benba Apr 15 '17 at 19:59
  • Furthermore the man pages says u should use -execdir instead of -exec ... – benba Apr 15 '17 at 20:19

This does the job in one step. Note the trailing/ slash/ on both the source and target paths.

rsync \
    -ruval \
    --ignore-existing \
    --remove-source-files \
    --prune-empty-dirs \ 
    /source/path/ /target/path/

I echo the caution from user7000 not to use --delete and --remove-source-files together in the same call to rsync. If the operation fails or is interrupted and the same call is repeated, you will lose data. If in any doubt, use the --dry-run option to see what would be done.

  • -ruval seems redundant. -a is equivalent to -rlptgoD, which includes both -r and -l. – greyfade May 28 at 1:33

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.