Move files and delete directories with rsync?

Question

Recently I needed to delete a large number of files (over 1 million) and I read that doing:

rsync -av --delete `mktemp -d`/ ~/source && rmdir ~/source

Was one of the most optimized ways to do that, and I can vouch that it's faster than rm -rf.

I'm not an expert on the matter, but from my understanding the reason for rsync performance has something to do the way it lists files (LIFO instead of FIFO I suppose). Now, the problem is, I also need to to move a large number of files in a efficient way. After searching a bit, I found this:

rsync -av --ignore-existing --remove-source-files ~/source ~/destination

While this deletes all the moved files in ~/source, the directories remain there. Since I have a "round-robin"-like directory structure the number of files/directories is very close to 1, so I am forced to run the first command again to get rid of the directory entirely:

rsync -av --ignore-existing --remove-source-files ~/source ~/destination && \
rsync -av --delete `mktemp -d`/ ~/source && rmdir ~/source

A straight mv would finish virtually instantly, but my ~/destination directory has files that should be kept, so mv is not an option. I found the --prune-empty-dirs and --force rsync options, but neither seem to work as I would expect:

--force                 force deletion of directories even if not empty
--prune-empty-dirs      prune empty directory chains from the file-list
--remove-source-files   sender removes synchronized files (non-dirs)

Is there a way to mimic a move with rsync in one go?

Answer 1

From zany's comment to slm's answer (Move files and delete directories with rsync?) I would recommend these 2 commands as an answer:

rsync -av --ignore-existing --remove-source-files source/ destination/ && \
find source/ -depth -type d  -empty -exec rmdir "{}" \;

The advantage is, like zany said, there is still some danger involved in using rm -rf if you don't get it right or for beginners.

I added 2 options, -depth and -empty and while I am not sure if this is really necessary, it makes the 2nd command more portable for other situations and even more safe (it still does the right thing if some directories are not empty and starts removing from the deepest point in a directory tree)

Answer 2

I found this thread over on stackoverflow titled: Deleting folders with rsync “move”?, which is asking essentially the same question. One of the answers suggested doing the rsync in 2 commands since it appears there isn't a single command that can accomplish the move/remove of the files and the source directories.

$ rsync -av --ignore-existing --remove-source-files source/ destination/ && \
  rsync -av --delete `mktemp -d`/ source/ && rmdir source/

Alternatively you can do it using this command:

$ rsync -axvvES --remove-source-files source_directory /destination/ && \
  rm -rf source_directory

Not ideal but does the job.