Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I'd like to know the distribution of file sizes under a certain directory.

Please note: distribution of file sizes, not size of a directory. That means I want to know there are 25 files of 60 bytes, 50 files of 12587 bytes, 2 files of 57kbytes, and so on.

Bonus points if the data could be gathered via command line (eg. on a remote system) in a format easily useable to produce graphs.

share|improve this question
add comment

3 Answers 3

List the files, extract the size in bytes from the list, sort it and count the occurrence of every size:

find /my/directory -type f -exec ls -l {} + | cut -d' ' -f5 | sort -n | uniq -c
  • not terribly efficient
  • if there are many many files it may be better to save intermediate results in a temp file, sort it to another temp file, then "uniq" it
  • here I use numeric sort so the output will be ordered by ascending file size (nice), but any sort will do as long as equal lines are grouped together
  • pipe the results in awk '{ print $1 "," $2 }' to get a CSV file to be used in your graphing tool of choice (even spreasheet tools will do)
share|improve this answer
    
I think you forgot the -l option to ls. You may want to replace \; with + to make it more efficient. –  Stéphane Chazelas Nov 7 '12 at 15:08
add comment

A variant of Luke404's with GNU find:

find . -type f -printf '%s\n' | sort -n | uniq -c
share|improve this answer
add comment

This Perl code may help:

@files = grep {-f} glob "*"; #List files in the current directory   
%files;  
for(@files)  
{  
  chomp (my $size = \`du -h \"$_\"`);  
  $size=~ s/\s+.*//;  #Remove the file name from the output of du  
  $files{$size}++;  #  Add an entry to the hash  
}  
print "Size,Count\n"; # Print a header  
print "${\_},$files{$_}\n" for(keys %files); # Print info in CSV format  

Note the following:

  • This code does not attempt to sort the files by size (this would probably need a subroutine by itself)
  • I'm using du rather than the -s operator of Perl in order to have human readable output.
  • If you want to list the contents of a directory other than the current one, replace glob "*" by glob "$ARGV[0]/*" and supply the name of your required directory as a command line argument.
  • If you want to list the contents of several directories in a batch job, you can save this code as list_dir.pl and have a bash wrapper that does something like this:

    list_dir.pl dir1 > dir1_list.csv  
    list_dir.pl dir2 > dir2_list.csv
    
  • Alternatively, the code can be further modified to accept several directories as arguments and fork a process for each directory it's searching.

I'm assuming you want files in the current directory only one level deep (no recursion).

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.