Identify the number of unique values and then the number of occurrences of those values in the file

Question

I have a data file which contains 15000 lines, but only 400 unique values. I am looking for a way to identify the number of unique values and then the number of occurrences of those values in the file. I came up with the following but it is very very slow. Any thoughts?

for value in `cat mylist.txt | uniq`
do
    counter=`grep $value mylist.txt |wc -l`
    echo $value $counter
done

Answer 1

Just use sort and uniq:

sort mylist.txt | uniq | wc -l

That will give you the number of unique values. To get the number of occurrences of each unique value, use uniq's -c option:

sort mylist.txt | uniq -c

From the uniq man page:

   -c, --count
               prefix lines by the number of occurrences

Also, for future reference, grep's -c option is often useful:

 -c, --count
              Suppress  normal  output;  instead  print  a  count  of
              matching  lines  for  each  input  file.   With the -v,
              --invert-match option (see below),  count  non-matching
              lines.  (-c is specified by POSIX.)