23

Say I have a file which contains:

A
A
A
B
CC

I want to have the output like this:

A 3
B 1
CC 1
23

I figured it out; one of uniq's options is -c, for "prefix lines by the number of occurrences":

$ uniq -c
  • 1
    Note that puts the numbers first. If you were fussy about the order, you could do: uniq -c filename.txt | sed 's/[^0-9]*\\([0-9]\+\\) \\(.*\\)/\2 \1/' – frabjous Jan 7 '11 at 3:28
  • 12
    Also note that it only counts adjacent repeat lines. A common idiom is sort | uniq – Steven D Jan 7 '11 at 3:54
  • 4
    uniq also puts the count in front of the datum. The original question would actually need something like this: sort filename | uniq -c | awk '{print $2, $1}' – Bruce Ediger Jan 7 '11 at 13:25
  • In case it isn't clear from the above comments, you must ensure the data is sorted first to achieve your goal. If it is not sorted, you will have repeat entries. For example, if your original file was instead A \ A \ A \ B \ A \ CC, the output of just uniq -c would show A 3 and later show A 1. Sorting first, will guarantee all identical lines are grouped together – drootang Jan 12 '17 at 16:31
16

I just came here with a similar problem. From this, I managed to put together a slightly more advanced command, which I hope is useful for others.

As Steven D said in the comments above uniq only counts adjacent repeat lines, so you need to sort the lines first. After that we find the unique lines then sort again so the most occurring lines are on top.

sort file.txt | uniq -c | sort -nr > output.txt

Output is redirected into the file output.txt. If you just want to view results on the command line, remove the redirection and change the last command to sort -n so that the most common line will be at the bottom, i.e. definitely still on screen.

  • 4
    You can replace cat file.txt | sort with just sort file.txt. :) – mattdm Mar 7 '11 at 21:21
  • 1
    @mattdm: the downside of that formulation is that you can't quickly replace the cat with something more interesting. Since, you know, there is no cat. – SamB Nov 18 '11 at 21:34
  • 1
    @SamB Then write it as < file.txt sort | uniq -c. This is easy to edit, and still avoids the unnecessary cat. – hvd May 12 '15 at 9:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.