2

I have a directory containing around 15K images, with x number of sequences

scene1_000000.png
scene1_000001.png
scene1_000002.png
scene1_000003.png
scene1_000004.png
scene1_000005.png
scene1_000006.png
scene1_000007.png
scene1_000008.png
       .
       .
       .
       .
scene2_000300.png
scene2_000000.png
scene2_000001.png
scene2_000002.png
scene2_000003.png
scene2_000004.png
scene2_000005.png
scene2_000006.png
scene2_000007.png
scene2_000008.png
       .
       .
       .
       .
scene2_000300.png
       .
       .
       .
       .
scene50_000000.png
scene50_000001.png
scene50_000002.png
scene50_000003.png
scene50_000004.png
scene50_000005.png
scene50_000006.png
scene50_000007.png
scene50_000008.png
       .
       .
       .
       .
scene50_000300.png

I want to keep from each sequence the first 150 sequences, and delete the rest. So I will have for every scene, the sequences from 000000 to 000150

1
  • 2
    Another technique, is to move the files that you want to keep, to another place. Then check, then remove the remnant. Jul 10 '18 at 17:09
4

With a recent version of bash, you can use brace expansion for this:

rm scene*_{000151..000300}.png

Bash's brace expansion deals with leading 0s:

$ echo {000000..000005}.png
000000.png 000001.png 000002.png 000003.png 000004.png 000005.png

So you can use {000151..000300} to generate the list of files you need.

8
  • Ok thanks a lot, I will try it and give you a feedback and what if the scenes do not have the same number of sequences for example some have 300 sequences and some more or less but anyways I need just the first 150, regardless of how many sequences are there Jul 10 '18 at 16:26
  • @MostafaHussein that isn't a problem. Because of the * in scene*, the shell will expand this to the list of all matching files. Since the files you mention don't exist, they will not be included in the command.
    – terdon
    Jul 10 '18 at 16:37
  • so this means that I just check what is the largest number of sequences I have (let's say 500) and do like this rm scene*_{000151..000500}.png Jul 10 '18 at 16:43
  • @MostafaHussein yes. But try ls scene*_{000151..000500}.png first, to be sure.
    – terdon
    Jul 10 '18 at 16:57
  • okay I tried it with ls first and it works fine, but there is only 1 problem. I have to know the largest number of sequences available in the directory. To elaborate, let's say that the greatest scene here has 601 sequences, and all the others are below 300, if I did that ls scene*_{000151..000601}.png, it will work fine without caring about the other sequence numbers, however, if I did that ls scene*_{000151..000602}.png, it will give an error /bin/ls: cannot access '*_000602.png': No such file or directory. Jul 11 '18 at 5:28
4

Using find:

find . -maxdepth 1 -type f \
         -name 'scene*.png' \
       ! -name '*_0000[0-9][0-9].png' \
       ! -name '*_0001[0-4][0-9].png' \
       ! -name '*_000150.png' -print -delete

This would find all the files that you'd like to delete in the current directory (only).

The various -name flags do the following:

  1. Only select the ones matching scene*.png.
  2. Filter out (remove from selection) filenames that are in the range 000000.png to 000099.png.
  3. Filter out filenames that are in the range 000100.png to 000149.png.
  4. Filter out filenames that end in 000150.png.

... then delete the file if it's still considered. The 2nd, 3rd and 4th -name tests are negated to stop find from deleting those files (these are the ones we'd like to keep).

The names of the files deleted will also be printed before the actual deletion occurs.

I suggest that you run this with -delete removed first.

4
  • but will this delete the first 150 and keep the rest ? what I want is keep the first 150, and delete the rest Jul 10 '18 at 16:44
  • 1
    @MostafaHussein That is what this is doing. The ! -name bits are filtering out the names that we'd like to keep. The rest is removed. It will keep the ones with numbers 150 or less.
    – Kusalananda
    Jul 10 '18 at 16:44
  • I tried something like that find . -maxdepth 1 -type f -name '*.png' ! -name '*_0000[0-1][0-1].png' -print, and this is assumed to print 4 images from each scene, which are _000000.png , _000001.png , _000010.png , _000011.png multiplied by 50 scenes so 200 images in total, however, it printed 10979 images, which have sequence numbers ending with 300's or 200's Jul 11 '18 at 5:36
  • 1
    @MostafaHussein No, you are still misunderstanding. It will print the image filenames that matches *.png but not the ones ending in 00.png, 01.png, 10.png or 11.png. In my code, we select all scene*.png images, then we remove (from that selection) the images we want to keep. Then we delete (from disk) the images that are still selected.
    – Kusalananda
    Jul 11 '18 at 5:51
2

In zsh, the <m-n> construct matches strings that are numbers between m and n. Both are optional. Thus:

rm scene<->_<151->.png

(you could start the wildcard with scene*_ as well, but <-> would be a safeguard against * matching something unintended if there are files named according to a different pattern)

0

Maybe a little script :

echo '#!/bin/bash
# Test argument
if [ $# = 1 ]; then
  if [ -d $1 ]; then
    cd $1
  else
    echo "$1 is not a directory" >&2
    exit 1
  fi
fi

declare f      # filname
declare -i fc  # file counter
declare -i fcs # file conter for a scene
declare sn     # scene number
declare csn    # current scene number
declare -i dfc # delete file counter
declare -i pfc # preserve file counter

# Open logfiles
exec file-delete>>&3
exec file-preserve>>&4

for f in *; do
  ((fc++))
  sn=${f#scene}
  sn=${sn%_*}
  if [ "$sn" = "$csn" ]; then
    ((fcs++))
  else
    # New number scene
    fcs=1   
    csn=$sn
    dfc=0
    pfc=0
    echo
  fi
  if ((fcs > 150));then
    # Candidate for deleting
    echo "$f" >>&3
    ((dfc++))
  else
    # Preserve file
    echo "$f" >>&4
    ((pfc++))
  fi
  # Display in console
  echo -en "\r scene $csn preserve:$pfc delete:$dfc"
done
echo

# Closing logfile
exec 3>&-
exec 4>&-

echo "Files scaned : $fc"
wc -l file-delete
wc -l file-preserve
' > ~/scanfile

For run it :

bash ~/scanfile [<pathdir>]

If is omitted, it works in current directory.

It does not delete anything, it writes two a file in scaned directory (file-delete and file-preserve).

After the check, the files to be deleted can be moved with :

[cd <pathdir>]
mkdir tmp
while read; do mv -v $REPLY tmp; done < file-delete

If the result is good, then

rm -rf tmp
rm file-delete file-preserve

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.