CentOS 5.x

I apologize if this is a repeat question. I've seen a lot of similar questions (regarding deleting files) but not exactly the same scenario.

I have a directory containing hundreds of thousands of files (possibly over a million) and as a short-term fix to a different issue, I need to move these files to another location.

For the purpose of discussion, let's say the these files originally reside in /home/foo/bulk/ and I want to move them to /home/foo2/bulk2/

If I try mv /home/foo/bulk/* /home/foo2/bulk2/ I get a "too many arguments" error.

Mr. Google tells me that an alternative for deleting files in bulk would be to run find. Something like: find . -name "*.pdf" -maxdepth 1 -print0 | xargs -0 rm

That would be fine if I was deleting stuff but in this case I want to move the files... If I type something like find . -name "*" -maxdepth 1 -print0 | xargs -0 mv /home/foo2/bulk2/ bash complains about the file not being a directory.

What's the best command to use here for moving the files in bulk from one directory to another?

up vote 13 down vote accepted

Taking advantage of GNU mv's -t option to specify the target directory, instead of relying on the last argument:

find . -name "*" -maxdepth 1 -exec mv -t /home/foo2/bulk2 {} +

If you were on a system without the option, you could use an intermediate shell to get the arguments in the right order (find … -exec … + doesn't support putting extra arguments after the list of files).

find . -name "*" -maxdepth 1 -exec sh -c 'mv "$@" "$0"' /home/foo2/bulk2 {} +
  • Brilliant. Thanks. – Mike B Nov 28 '13 at 0:24
  • You can use find ... -exec sh -c 'blah "$@" blah' sh {} +" (the "shell trick") to deal with issues around argument ordering. There are several examples of this in the findutils (info) documentation; search for "sh -c". – James Youngman Dec 9 '13 at 23:12

Consider mving the parent directory instead of the files:

mv /home/foo/bulk /home/foo2/bulk2 && mkdir /home/foo/bulk

(But it might cause problems if /home/foo/bulk must exist at every moment.)

  • 1
    I would add to that, just in case your destination directory exists, move the files in the destination directory to your source directory (the big one) and rename the directory. – sbenitezb Nov 28 '13 at 3:57
  • Thanks - great suggestion. In this case, the directory must exist at every moment. Great idea though. – Mike B Nov 28 '13 at 20:20
  • I have to admit I don't understand what this answer does. Why mkdir? And what about that dir 'existing at every moment' or not? Why would this speed things up? – Monica Heddneck Jun 2 at 0:48

Just for variety, I'm fond of using cpio for some cases like this.

find tmp |cpio -v  -p --make-directories --sparse tmp2
  • 1
    If the source and destination are on the same file system then this approach could be very inefficient (unnecessary copying). Also it is always better to use -print0 and -0 options. – pabouk Nov 28 '13 at 9:11

With GNU mv:

find . -mindepth 1 -exec mv -t /tmp {} +

On systems like OS X that don't have mv -t but do have -print0 and xargs -0:

find . -mindepth 1 -print0|xargs -0 -I, mv , /tmp

Or if the file names don't contain spaces, quotes, backslashes, or newlines (with OS X's xargs) or quotes, backslashes, or newlines (with GNU xargs):

ls|xargs -I, mv , /tmp

For a specified time try the following:

find . -name "*" -type f -mtime +7 -exec mv -t /folder/to/transfer/files {} +

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.