14

Sometimes I have a folder full of jpg's and I need to randomly choose 8 or so of them. How could I automate this so my account randomly chooses 8 jpg's from the folder and copies them to another destination?

My question is simple really, instead of using cp and giving it a file name then destination file name, I want to build a script that randomly chooses 8 of the .jpgs in the folder, and copies those to another folder.

1
  • 1
    you can use a combination of crontab and mv.
    – vfbsilva
    Jul 22 '15 at 19:55
33

You could use shuf:

shuf -zn8 -e *.jpg | xargs -0 cp -vt target/
  • shuf shuffles the list of *.jpg files in the current directory.
  • -z is to zero-terminate each line, so that files with special characters are treated correctly.
  • -n8 exits shuf after 8 files.
  • xargs -0 reads the input delimited by a null character (from shuf -z) and runs cp.
  • -v is to print every copy verbosely.
  • -t is to specify the target directory.
3
  • What if some of the file names start with -? I tried shuf -zn8 -e *.jpg | xargs -0 cp -vt -- {} target/ to no avail.
    – havakok
    Oct 3 '19 at 10:58
  • For anyone who has found this answer and has seen @havakok's question, they also asked the question here and obtained an answer: unix.stackexchange.com/a/544902/372726 Oct 13 '20 at 19:51
  • This solution does not work with a large number of files. Halavus answer solves the problem in that case.
    – Manu CJ
    Dec 22 '20 at 9:06
5

The best answer absolutely didn't worked for me, because -e *.jpg doesn't actually look into the working directory. It's just an expression. So shuf doesn't shuffle anything...

I found the following improvement based on what I learned in that post.

find /some/dir/ -type f -name "*.jpg" -print0 | xargs -0 shuf -e -n 8 -z | xargs -0 cp -vt /target/dir/
3
  • 3
    The -e *.jpg expects a set of matching files in the current directory. If there are no matches it will (usually) return the single literal *.jpg to shuf, which then has only one element to consider.
    – roaima
    Dec 17 '17 at 13:58
  • 1
    Thank you for this solution; it works with a large number of files, as opposed to the accepted solution. Jan 19 '20 at 16:41
  • If you are on MacOS, first install coreutils for the shuf command (brew install coreutils), then use: find /some/dir/ -type f -name "*.jpg" -print0 | xargs -0 shuf -e -n 8 -z | xargs -0 -J % cp -v % /your/target/dir
    – Phlogi
    Dec 31 '20 at 9:36
2

You can also do this with Python.

Here is a python scscript I use to move a random percent of images that also gets associated label datasets typically required for CV image datasets. Note this moves the files because I do not want my test training dataset in my training dataset.

I use the below for Yolo training sets as labels and images are in the same directory and the labels are txt files.

import numpy as np
import os
import random

#set directories
directory = str('/MauiData/maui_complete_sf_train')
target_directory = str('/MauiData/maui_complete_sf_test')
data_set_percent_size = float(0.07)

#print(os.listdir(directory))

# list all files in dir that are an image
files = [f for f in os.listdir(directory) if f.endswith('.jpg')]

#print(files)

# select a percent of the files randomly 
random_files = random.sample(files, int(len(files)*data_set_percent_size))
#random_files = np.random.choice(files, int(len(files)*data_set_percent_size))

#print(random_files)

# move the randomly selected images by renaming directory 

for random_file_name in random_files:      
    #print(directory+'/'+random_file_name)
    #print(target_directory+'/'+random_file_name)
    os.rename(directory+'/'+random_file_name, target_directory+'/'+random_file_name)
    continue

# move the relevant labels for the randomly selected images

for image_labels in random_files:
    # strip extension and add .txt to find corellating label file then rename directory. 
    os.rename(directory+'/'+(os.path.splitext(image_labels)[0]+'.txt'), target_directory+'/'+(os.path.splitext(image_labels)[0]+'.txt'))

    continue
1

You could retrieve files in this way:

files=(/tmp/*.jpg)
n=${#files[@]}
file_to_retrieve="${files[RANDOM % n]}"
cp $file_to_retrieve <destination>

make a loop 8 times.

1
  • So essentially rather than an answer you provide a list of variable names.
    – gented
    Dec 18 '18 at 22:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.