Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I did a website scrape for a conversion project. I'd like to do some statistics on the types of files in there -- for instance, 400 .html files, 100 .gif, etc. What's an easy way to do this? It has to be recursive.

Edit: With the script that maxschelpzig posted, I'm having some problems due to the architecture of the site I've scraped. Some of the files are of the name *.php?blah=blah&foo=bar with various arguments, so it counts them all as unique. So the solution needs to consider *.php* to be all of the same type, so to speak.

share|improve this question
up vote 32 down vote accepted

You could use find and uniq for this, e.g.:

$ find . -type f | sed 's/.*\.//' | sort | uniq -c
   16 avi
   29 jpg
  136 mp3
    3 mp4

Command explanation

  • find recursively prints all filenames
  • sed deletes from every filename the prefix until the file extension
  • uniq assumes sorted input
    • -c does the counting (like a histogram).
share|improve this answer
I have a similar script. Simple and fast. – Rufo El Magufo Aug 10 '11 at 19:10
Some of the files are of the name *.php?blah=blah&foo=bar with various arguments, so it counts them all as unique. How can I modify it to look for *.php*? – user394 Aug 11 '11 at 13:35
You can try to use a different sed expression, e.g. sed 's/^.*\(\.[a-zA-Z0-9][a-zA-Z0-9][a-zA-Z0-9]\).*$/\1/' – maxschlepzig Aug 11 '11 at 14:35
@maxschlepzig awesome, thanks so much! :) – user394 Aug 11 '11 at 16:04
@bela83, the prune variants rely on short-circuit evaluation - thus, my first version find -name '.*' -prune -o -type f -print evaluates like: if directory entry matches .* then prune it, otherwise if it is a file then print it. Since .* also matches ., i.e. the CWD, everything is pruned, i.e. find does not even descend into the first directory. Perhaps, 2 year old versions of find behaved differently - or it was just an oversight of me, back then. Anyhow, find -name '.*' -not -name . -prune -o -type f -print fixes this. – maxschlepzig May 4 '15 at 20:16

With zsh:

print -rl **/?*.*(.:e) | uniq -c |sort -k1n

The pattern **/?*.* matches all files that have an extension, in the current directory and its subdirectories recursively. The glob qualifier . selects only regular files. The history modifier retains only the file extension. print -rl prints one match per line. uniq -c counts consecutive identical items (the glob result is already sorted). The final call to sort sorts the extensions by use count.

share|improve this answer
This works awesome – JackLeo Sep 6 '11 at 16:59

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.