I had a command which would work through a text file, count all the occurrences of the words and print it out like this:

user@box $˜ magic-command-i-forgot | with grep | and awk | sort ./textfile.txt
66: the
54: and
32: I
16: unix
12: bash
5:  internet
3:  sh
1: GNU/Linux

So it does not search line-by-line, but word by word, and it does it for all the words, not just for 1 word. I'd found it somewhere on the internets a long time ago, but I cannot find or remember it..

6 Answers 6


I would use tr instead of awk:

echo "Lorem ipsum dolor sit sit amet et cetera." | tr '[:space:]' '[\n*]' | grep -v "^\s*$" | sort | uniq -c | sort -bnr
  • tr just replaces spaces with newlines
  • grep -v "^\s*$" trims out empty lines
  • sort to prepare as input for uniq
  • uniq -c to count occurrences
  • sort -bnr sorts in numeric reverse order while ignoring whitespace

wow. it turned out to be a great command to count swear-per-lines

find . -name "*.py" -exec cat {} \; | tr '[:space:]' '[\n*]' | grep -v "^\s*$" | sort | uniq -c | sort -bnr | grep fuck

  • 1
    Might want to use tr -s to handle multiple spaces, especially when encountering indentation.
    – Arcege
    Commented May 21, 2012 at 0:02
  • @Arcege: Good point. Although it wont change the result it may fasten script a little bit.
    – seler
    Commented May 21, 2012 at 7:10
  • -g (--general-numeric-sort) option of sort may be preferable in some cases. E.g. sort -n will keep 10\n1 4 as is, treating 1 4 as 14, while sort -g will treat it correctly as 1 4\n10. Commented Sep 6, 2015 at 12:04
  • nice command, really deserve a vote up :)
    – Noor
    Commented Nov 4, 2015 at 13:07
  • 1
    @seler I think you can make the grep and tr part even simpler like below: echo "Lorem ipsum dolor sit sit amet et cetera." | tr ' ' '\n' | grep -v "^$" | sort | uniq -c | sort -bnr Note that I am a beginner, so I may be wrong, please feel free to advise.
    – smc
    Commented Oct 26, 2019 at 19:10
  1. Split the input into words, one per line.
  2. Sort the resulting list of words (lines).
  3. Squash multiple occurences.
  4. Sort by occurrence count.

To split the input into words, replace any character that you deem to be a word separator by a newline.

<input_file \
tr -sc '[:alpha:]' '[\n*]' | # Add digits, -, ', ... if you consider
                             # them word constituents
sort |
uniq -c |
sort -nr
  • This is a nice answer because it can handle the case where the words are directly next to some non-word punctuation that you want to ignore. Commented Mar 27, 2017 at 18:09

Not using grep and awk but this seems to do what you want:

for w in `cat maxwell.txt`; do echo $w; done|sort|uniq -c
  2 a
  1 A
  1 an
  1 command
  1 considered
  1 domain-specific
  1 for
  1 interpreter,
  2 is
  1 language.
  1 line
  1 of
  • 1
    This won't work if the input contains shell wildcards (you need to add set -f), and treats punctuation as part of words (which can be fixed awkwardly by adding punctuation characters to IFS — good luck trying to support non-ASCII character sets). This won't be good with very large input files, as it stores the whole file in memory (sort is smarter). Commented May 21, 2012 at 0:21

Using awk/sort/uniq solution:

awk '{for(w=1;w<=NF;w++) print $w}' ~/textFile.txt | sort | uniq -c | sort -nr

I believe you're after something like this?

$ perl -n -e 'foreach ${k} (split(/\s+/)){++$h{$k}};END{foreach $l (keys(%h)){print "$h{$l}: ${l}\n"}}' /path/to/your/file | sort -n -k 1

of course you can do the same with awk as well :)

for word in $(sed 's/[^A-Za-z]/ /g' $file | tr " " "\n" | sort -u)
  echo -n "$word "
  grep -c $word $file
done | sort -k2 -n 

sorts ascending after splitting the file into words.

The simple grep will find fish in fisheye, so you have to enhance the grep command to prevent partial matches.

Takes about 3s for a 25k text file on a machine in the ages, classical HDD (IDE).

For bigger files or often performed operations a hash map approach would be better, but for a seldom run job or smaller files only, it might be sufficient.

You must log in to answer this question.