Suppose there is a directory holding 300 data files. I want to randomly select 200 of those files and move them into another directory. Is there a way to do that under Unix/Linux?

  • R can probably do this in the twinkling of an eye with list.files()... – sr_ May 10 '12 at 14:29
  • 4
    I'd vaguely plug together shuf and head (or just use shuf -n, should've read the man page...) – Ulrich Schwarz May 10 '12 at 14:41

If your system has shuf, you can use this quite conveniently (even handling ugly file names):

shuf -zen200 source/* | xargs -0 mv -t dest

If you don't have shuf but have a sort that takes -R, this should work:

find source -type f -print0 | sort -Rz | cut -d $'\0' -f-200 | xargs -0 mv -t dest
| improve this answer | |
  • 7
    Ah yes, because where else would one look for shuffling than in a tool for sorting. (At least shuf isn't called tros because it does the opposite of sorting.) – Ulrich Schwarz May 10 '12 at 16:52
  • 2
    There's no such thing as the opposite of sorting (in the same sense as there's no such thing as "no weather"). Random is still sorted, it's just sorted randomly. – Plutor May 10 '12 at 20:43
  • 1
    What is the "-zen200"? That's not in any of the documentation for shuf, or anywhere on the Internet, but your example doesn't work without it. Quite mystical. – SigmaX Feb 17 '15 at 21:56
  • 2
    @SigmaX Indeed, quite zen, isn't it. Hint: it's 3 separate flags. – Kevin Feb 17 '15 at 22:05
for (( i=0; i<200; i++ )); do
    rnd=$(( RANDOM % ${#keys[@]} ))
    mv "${files[$key]}" "$otherdir"
    unset files[$key]
| improve this answer | |

Put all filenames into an array named "files" in bash:

files=( * )

size of array:

echo ${#files[@]}

define 2/3 of them as sample size:


for i in $(seq 1 $take)
    echo ${files[r]}

This will select duplicates, and isn't tested with filenames with blanks and such.

The simplest way to avoid duplicates is, to iterate over all files, and pick each one with 2/3 chance, but this will not necessarily lead to 200 files.

This will remove a file if it was chosen from the list and fulfill your requirements:

files=( * )
# define 2/3 of them as sample size:

while (( i < $take ))
    if [[ -n $f ]]
        echo ${files[r]}
        unset files[r]    
| improve this answer | |
  • You might select the same file more than once. – glenn jackman May 10 '12 at 14:43
  • Very nice shell script. To get around your problem of not getting 200 files, you probably want to use Reservoir Sampling: en.wikipedia.org/wiki/Reservoir_sampling I'm going to be weak and not include a shell script example of this. – Bruce Ediger May 10 '12 at 14:51
  • @glennjackman: I wrote so, yes. Needed some minutes to figure out, how to remove entries from the array. – user unknown May 10 '12 at 14:53
  • Minor caveat: $RANDOM can only have values 0 through 32767, so this will not work properly if you have more than 32768 files. Also, fetching is biased towards the first files. – l0b0 May 10 '12 at 15:02
  • @l0b0: Requirements where, to pick 200 from 300. If the files aren't in the current directory, but on a file server, it will not work too. Different requirements, different answer. – user unknown May 10 '12 at 15:05

If this needs to be statistically random, you shouldn't use RANDOM % ${#keys[@]}. Consider:

  1. $RANDOM has 32768 unique values
  2. The first selection is 1 out of 300 elements
  3. 32768 = 109 * 300 + 68

Thus, when selecting the first item, there's a 110/32768~=0.33569% chance for each of the 68 first elements, and 109/32768~=0.33264% chance for each of the other 232 elements to be selected. Picking is repeated several times with different chances, but biased towards the first elements whenever 32768 % ${#keys[@]} -ne 0, so the error compounds.

This should be unbiased, and works with any filename:

while IFS= read -r -d '' -u 9
    mv -- "$REPLY" /target/dir
done 9< <(find /source/dir -mindepth 1 -print0 | shuf -n 200 -z)
| improve this answer | |

Kevin's solution works great! Something else I've used a lot because it find it easier to remember off the top of my head is something like:

cp `ls | shuf -n 200` destination
| improve this answer | |

One liner in bash:

ls original_directory/|sort -R|head -number_of_files_to_move|while read file; do cp "new_directory/"$file test; done
| improve this answer | |
  • Please elaborate; U&L is a knowledge base. – countermode Feb 22 '17 at 9:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.