7

I have a set of files, all that are named with the convention file_[number]_[abcd].bin (where [number] is a number in the range 0-size of drive in MB). i.e there is file_0_a.bin, file_0_b.bin, file_0_c.bin and file_0_d.bin and then the 0 would become a 1 and so on.

The number of files is figured out at run-time based on the size of the partition. I need to delete all of the files that have been created, but in a pseudo-random manner. in blocks of size that I need to be able to specify, i.e where there is 1024 files, delete 512, then delete another 512.

I have the following function for doing it currently, which I call the required number of times, but it will get progressively less likely to find a file that exists, to the point where it might never complete. Obviously, this is somewhat less than ideal.

What is another method that I can use to delete all of the files in a random order?

deleteRandFile() #$1 - total number of files
{
    i=$((RANDOM%$1))
    j=$((RANDOM%3))
    file=""

    case $j in
    0)
        file="${dest_dir}/file_${i}_a.bin";;
    1)
        file="${dest_dir}/file_${i}_b.bin";;    
    2)
        file="${dest_dir}/file_${i}_c.bin";;
    3)
        file="${dest_dir}/file_${i}_d.bin";;
    esac

    if ! [[ -f $file ]]; then
        deleteRandFile $1
    else
        rm $file
    fi

    return 0;
}

Edit: I'm trying to delete in random order so that I can fragment the files as much as possible. This is part of a script that begins by filling a drive with 1MB files, and deletes them, 1024 at a time, then fills the 'gap' with 1 1GB file. Rinse and repeat until you have some very fragmented 1GB files.

3
  • Maybe it would help if you can explain why it matters in what order you delete the files. Commented Oct 8, 2014 at 21:25
  • @NateEldredge Sure, uh...I'm trying to fragment a hard drive and kill a file system. I have my reasons.
    – Yann
    Commented Oct 8, 2014 at 21:37
  • In zsh, you would use the *.bin(o+functionName) notation as in unix.stackexchange.com/a/9831
    – ignis
    Commented Oct 9, 2014 at 7:06

2 Answers 2

13

If you want to delete all the files, then, on a GNU system, you could do:

cd -P -- "$destdir" &&
  printf '%s\0' * | # print the list of files as zero terminated records
    sort -Rz |      # random sort (shuffle) the zero terminated records
    xargs -r0 rm -f # pass the input if non-empty (-r) understood as 0-terminated
                    # records (-0) as arguments to rm -f

If you want to only delete a certain number of those matching a regexp you'd insert something like this between the sort and xargs:

awk -v RS='\0' -v ORS='\0' -v n=1024 '/regexp/ {print; if (--n == 0) exit}'

With zsh, you could do:

shuffle() REPLY=$RANDOM
rm -f file_<->_[a-d].bin(.+shuffle[1,1024])
10
  • I don't suppose you'd mind explaining a bit more fully?
    – Yann
    Commented Oct 8, 2014 at 10:41
  • I need to learn to type faster, or ignore the review queues to get an answer in ;-)
    – Anthon
    Commented Oct 8, 2014 at 10:42
  • This looks pretty good, do you mind going into a bit more detail about what the xargs -r0 flag does?
    – Yann
    Commented Oct 8, 2014 at 10:48
  • Thanks, the last thing is how would I need to alter this to make it not delete the lot at once, but delete, say, 1024 at a time? Would piping it through something to get the first 1024 entries that match a regex work?
    – Yann
    Commented Oct 8, 2014 at 10:55
  • 1
    @OlivierDulac, no. printf '%s\0' * formats each argument as %s\0 so is like find except that it excludes dot files, sorts the list, doesn't fork a process. And you can do printf '%s\0' file_*_[a-d].bin. find may be better if you only want regular files though you could also use zsh and its globbing qualifiers. Commented Oct 9, 2014 at 7:48
11

Here's a potential alternative using find and shuf:

$ find $destdir -type f | shuf | xargs rm -f

This will find all the files in $destdir and then use the shuf command to shuffle their order, and then pass the list on to xargs rm -f for deletion.

To gate how many files are deleted:

$ find $destdir -type f | shuf | head -X | xargs rm -f

Where -X is the number of files that you want to delete, for example, head -100.

13
  • It's a good answer, but it doesn't let me specify how many files to delete. (also I don't have shuf on the system, but I didn't specify what I had, so that doesn't stop this being a good answer)
    – Yann
    Commented Oct 8, 2014 at 12:32
  • I'm confused, why do you need to specify a number to the delete function?
    – slm
    Commented Oct 8, 2014 at 12:35
  • 1
    @Yann4 I'm confused but curious about that "doesn't involve re-compiling a kernel" - is that some kind of meme I missed? Commented Oct 8, 2014 at 17:12
  • 2
    This answer is unsafe as written, at least in general. find outputs literal strings separated by newlines, and xargs reads a shell-quoted, whitespace-delimited list of names as input. A malicious name in the input can trick it into deleting something very different from what you intended to delete. Commented Oct 8, 2014 at 23:08
  • 2
    @R - look at the requirements for the format of filenames that the OP is using. This is perfectly safe given that!
    – slm
    Commented Oct 8, 2014 at 23:09

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .