9

If I want find command to stop after finding a certain number of matches, how do I do that?

Background is that I have too many files in a folder, I need to put them into separate folders randomly like:

find -max-matches 1000 -exec mv {} /path/to/collection1 \+; 
find -max-matches 1000 -exec mv {} /path/to/collection2 \+; 

is this possible to do with find alone? If not, what would be the simplest way to do this?

  • I have an issue with the "randomly". Of course you can choose a random destination, but that means that when looking for a file that you know the name of you would have to test several (maybe all) of the subdirectories before finding the file. – Law29 Dec 19 '19 at 22:01
17

As you're not using find for very much other than walking the directory tree, I'd suggest instead using the shell directly to do this. See variations for both zsh and bash below.


Using the zsh shell

mv ./**/*(-.D[1,1000]) /path/to/collection1    # move first 1000 files
mv ./**/*(-.D[1,1000]) /path/to/collection2    # move next 1000 files

The globbing pattern ./**/*(-.D[1,1000]) would match all regular files (or symbolic links to such files) in or under the current directory, and then return the 1000 first of these. The -. restricts the match to regular files or symbolic links to these, while D acts like dotglob in bash (matches hidden names).

This is assuming that the generated command would not grow too big through expanding the globbing pattern when calling mv.

The above is quite inefficient as it would expand the glob for each collection. You may therefore want to store the pathnames in an array and then move slices of that:

pathnames=( ./**/*(-.D) )

mv $pathnames[1,1000]    /path/to/collection1
mv $pathnames[1001,2000] /path/to/collection2

To randomise the pathnames array when you create it (you mentioned wanting to move random files):

pathnames=( ./**/*(-.Doe['REPLY=$RANDOM']) )

You could do a similar thing in bash (except you can't easily shuffle the result of a glob match in bash, apart for possibly feeding the results through shuf, so I'll skip that bit):

shopt -s globstar dotglob nullglob

pathnames=()
for pathname in ./**/*; do
    [[ -f $pathname ]] && pathnames+=( "$pathname" )
done

mv "${pathnames[@]:0:1000}"    /path/to/collection1
mv "${pathnames[@]:1000:1000}" /path/to/collection2
mv "${pathnames[@]:2000:1000}" /path/to/collection3
|improve this answer|||||
  • yes, being able to do it without find is even better. – David Dai Dec 19 '19 at 22:05
14

You can implement new tests for find using -exec:

seq 1 1000 |
find . -exec read \; -exec mv {} /path/to/collection1 +

will move the first 1000 files found to /path/to/collection1.

This works as follows:

  • seq 1 1000 outputs 1000 lines, piped into find;
  • -exec read reads a line, failing if the pipe is closed (when seq’s output has been consumed);
  • if the previous -exec succeeds, -exec mv ... performs the move.

-exec ... + works as you’d expect: read will run once per iteration, but find will accumulate matched files and call mv as few times as possible.

This relies on the fact that find’s -exec succeeds or fails based on the executed command’s exit status: when read succeeds, find continues processing the actions given above (because the default operator is “and”), and when it fails, find stops.

If your find supports the -quit action, you can use that to improve the efficiency:

seq 1 1000 |
find . \( -exec read \; -o -quit \) -exec mv {} /path/to/collection1 +

Without that, find will test every single file, even though it will only keep 1000 for mv.

I’m assuming that read is available as an external command, and implements the POSIX specification for read; if that’s not the case, sh -c read can be used instead. In both cases, find will start a separate process for each file it checks.

|improve this answer|||||
  • to do find -exec read I need to have read as an external command right? the bash built-in wouldn't work? – David Dai Dec 19 '19 at 9:53
  • In Archlinux, I could only find a read external command in the packages plan9port and 9base. It works for your application, but it doesn't seem like a typical thing to have installed. I don't know on what systems it would be usual to find it. I also don't understand what you mean with "which uses the shell's built-in". I guess it could, but at least the one from plan9port doesn't, and I don't see why it would instead of re-implementing what it could in C. The benefits of it being a shell-builtin like assigning variables can't be obtained from an external anyways. – JoL Dec 19 '19 at 17:12
  • 1
    @JoL in Fedora, the bash package ships a /usr/bin/read script which runs builtin read "$@". I’ve updated my answer to address this particular issue, hopefully without the approximations of my comments! And yes, it’s one of the commands which doesn’t make much sense outside a shell context (like cd). – Stephen Kitt Dec 19 '19 at 17:17
10

I don't think it can be done with just find. You can use something like:

find [... your parameters ...] -print0 | head -z -1000 | xargs -0 mv -t /path/to/collection

-print0, -z, and -0 work together to make sure everything works even with linefeeds in filenames.

|improve this answer|||||
  • Thank you! I was well aware of -print0 and xargs -0, but not of the "-z" option (which is apparently everywhere and I somehow missed it all these years... Just saw that it's there in cut, sed, tail and probably all coreutils... As for the -t option of mv, I guess it was tailor-made for xargs - with the more general (not mv-specific) solution for this something along the lines of xargs -i somecommand {} targetFolder/. – ttsiodras Dec 25 '19 at 14:15
  • @ttsiodras -z is a recent discovery for myself too... – xenoid Dec 25 '19 at 14:19
4

I think, it is not possible directly with find, but you can use a pipe with head and xargs like:

find ... | head -1000 | xargs -i mv "{} /path/to/collection1"

This moves the first 1000 files to collection1.

|improve this answer|||||

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.