0

I need a modification for the below perl command:

perl -wE 'say for ((sort { -s $b <=> -s $a } </tmp/?>)[0..9]);'

Requirement:

  1. It should scan through all the sub-directories inside the target directory.
  2. List down the top 10 files with their size and path.
2

This perl script will print exactly what you need, it is using File::Find to traverse recursively. I have used -f to make sure only files are pushed into the hash

Hash %files has filepath as the key and size as its value. Then sorted it on basis of values and printed the top 10 results

#!/usr/bin/perl

use strict;
use warnings;
use File::Find;

my %files;
my $counter=0;
find( \&wanted, '<target directory>');
for my $file  ( sort {$files{$b} <=> $files{$a}}  keys%files){
        print "$file : $files{$file}\n";
        $counter++;
        if ($counter == 10){
                last;
        }
}


sub wanted {
  $files{"$File::Find::name"}=-s $File::Find::name if -f;
  return;

}

Or simply using an array to get it working

#!/usr/bin/perl

use strict;
use warnings;
use File::Find;

my @files;
my $counter=0;
find( \&wanted, '<target directory>');
for my $file  ( sort { -s $b <=> -s $a}  @files){
        my $size = -s $file;
        print "$file : $size\n"
        $counter++;
        if ($counter == 10){
                last;
}
sub wanted{
  push @files,$File::Find::name if -f;
  return;
}
  • in your above code second snippet @files[0..9] won't give top sorted file names , but will only run sort on first 10 entries of @files array – mkmayank May 18 '18 at 11:25
  • thanks a lot for suggestion, i have edited my answer to get the correct result – Arushix May 18 '18 at 11:45
  • it worked charm for my lower size directories. However for a bigger size, im getting an "Out of Memory" message – Anoop Kumar KR May 18 '18 at 14:11
  • @Arushix... any suggestion to avoid the Out of Memory message? – Anoop Kumar KR May 21 '18 at 9:04
2

Use File::Find to recursively walk the directory tree:

perl -MFile::Find -wE '
    find(sub { push @all, $File::Find::name }, "/tmp");
    say for (sort { -s $b <=> -s $a } @all)[0..9]'

If there are too many files and you're getting Out of memory, return the sizes and use external sort and head to limit the output:

perl -MFile::Find -wE 'find(sub { say -s $_, " $File::Find::name" }, "/tmp")' \
| sort -nr | head -n10
  • thanks for the response. I got the file names, how can i get the size of the files too along with the file names. – Anoop Kumar KR May 18 '18 at 10:58
  • 1
    add | xargs du -h to the above perl command output – mkmayank May 18 '18 at 11:01
  • I found this for the file size: perlmaven.com/how-to-get-the-size-of-a-file-in-perl – Kramer May 18 '18 at 11:06
  • i see the size details came out are different with the perl command and with the unix find command – Anoop Kumar KR May 18 '18 at 13:01
  • hpauto@st2ba1301:/tmp$ p"); say for (sort { -s $b <=> -s $a } @all)[0..9]' 2>/dev/null | xargs du -s < 343800 /tmp/Spectra/spectre_meltdown_fix.tar 38336 /tmp/lftp_latest/gcc-4.8.3-1.aix7.1.ppc.rpm – Anoop Kumar KR May 18 '18 at 13:02
1
zsh -c 'ls -ldS /tmp/**/?(DOL[1,10])'

To list the 10 Largest single-character (?) files in /tmp and subdirs (**/), ordered by Size.

With perl, and to avoid storing the whole file list in memory when you only want the 10 largest ones:

perl -MFile::Find -e '
  find(
    sub {
      if (length == 1 && $_ ne ".") {
        @s = sort {$b->[0] <=> $a->[0]} [-s, $File::Find::name], @s;
        splice @s, 10
      }
    }, "/tmp"
  ); printf "%16d %s\n", @{$_} for @s'

(the length == 1 && $_ ne "." is to match on single-byte file names like your /tmp/? suggests you want to do).

Instead of printf "%16d %s\n", @{$_}, you could also run ls like in the zsh solution with exec "ls", "-ldS", map $_->[1], @s

0

easier with bash I would say

find ${PATH_TO_PARENT_FOLDER} -type f -exec du -ahx {} + | sort -rh | head -10

Essentially we are using find to locate all files inside a folder,

  • -type either f for file or d for dir
  • -exec executes a command using the output of find and places find result as argument at {}

then executing du to calculate the size of each element found through find and adding

  • -a, --all write counts for all files, not just directories
  • -h, --human-readable print sizes in human readable format (e.g., 1K 234M 2G)
  • -x, --one-file-system skip directories on different file systems

Then we pipe it to sort it form bigger to smaller and we use head to present the top 10 only

sort to sort the output result

  • -h, --human-numeric-sort compare human readable numbers (e.g., 2K 1G)
  • -r, --reverse reverse the result of comparisons

head to just read the top results

  • -<N> to set the amount of lines you want to see from 1 to N
  • I have a directory with more than 300000 files, find command takes around 15mins or it goes in hung state.. !! :( – Anoop Kumar KR May 18 '18 at 11:00
  • you could combine the solution from @choroba with mine, read the FS using perl, then apply for each of the found results the commands I shared with you :) – Kramer May 18 '18 at 11:02
  • choroba's command is failing with "Out of memory" – Anoop Kumar KR May 18 '18 at 13:23
  • ?????????????????? – Anoop Kumar KR May 22 '18 at 8:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.