Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

Recently I needed to delete a large number of files (over 1 million) and I read that doing:

rsync -av --delete `mktemp -d`/ ~/source && rmdir ~/source

Was one of the most optimized ways to do that, and I can vouch that it's faster than rm -rf.

I'm not an expert on the matter, but from my understanding the reason for rsync performance has something to do the way it lists files (LIFO instead of FIFO I suppose). Now, the problem is, I also need to to move a large number of files in a efficient way. After searching a bit, I found this:

rsync -av --ignore-existing --remove-source-files ~/source ~/destination

While this deletes all the moved files in ~/source, the directories remain there. Since I have a "round-robin"-like directory structure the number of files/directories is very close to 1, so I am forced to run the first command again to get rid of the directory entirely:

rsync -av --ignore-existing --remove-source-files ~/source ~/destination && \
rsync -av --delete `mktemp -d`/ ~/source && rmdir ~/source

A straight mv would finish virtually instantly, but my ~/destination directory has files that should be kept, so mv is not an option. I found the --prune-empty-dirs and --force rsync options, but neither seem to work as I would expect:

--force                 force deletion of directories even if not empty
--prune-empty-dirs      prune empty directory chains from the file-list
--remove-source-files   sender removes synchronized files (non-dirs)

Is there a way to mimic a move with rsync in one go?

share|improve this question
2  
If you don't get an answer here, try posting on the rsync list. They're very helpful. lists.samba.org/mailman/listinfo/rsync –  Joe Jun 7 '13 at 19:36
add comment

1 Answer

up vote 1 down vote accepted

I found this thread over on stackoverflow titled: Deleting folders with rsync “move”?, which is asking essentially the same question. One of the answers suggested doing the rsync in 2 commands since it appears there isn't a single command that can accomplish the move/remove of the files and the source directories.

$ rsync -av --ignore-existing --remove-source-files source/ destination/ && \
  rsync -av --delete `mktemp -d`/ source/ && rmdir source/

Alternatively you can do it using this command:

$ rsync -axvvES --remove-source-files source_directory /destination/ && \
  rm -rf source_directory

Not ideal but does the job.

share|improve this answer
    
Yes, it was me who answered that question (and the same "solution" is also on this question). =P –  Alix Axel Jun 29 '13 at 3:34
    
@AlixAxel - Ha, sorry I didn't even notice you answered that one. Oh well. Shall I delete this answer then? –  slm Jun 29 '13 at 8:19
    
Using rsync to prune directories doesn't feel right and there is always danger with rm -rf. I'd recommend the 2nd step to be: find source/ -d -type d -exec rmdir {} \; –  zany Dec 13 '13 at 14:10
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.