Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

As input, we have some ids which are strings. How do we calculate what identifier has appeared most frequently and how many times?

Example

Input:

cat 
dog 
cat
dog
dog
dog 
dog 
fly
spider
spider
cat
cat

Output:

dog 5
share|improve this question

3 Answers 3

You can use the commands sort & uniq -c to count the occurrences of all the strings like this:

$ sort sample.txt | uniq -c
      4 cat
      5 dog
      1 fly
      2 spider

If you just want the one string "dog" you can use grep either before or after.

after

$ sort sample.txt | uniq -c | grep dog
      5 dog

before

$ grep dog sample.txt  | uniq -c
      5 dog

How it works

The command uniq -c will count all the unique strings that it's presented, but the list needs to be in sorted order first. Hence why we use the sort command prior to doing the tally.

The command grep is for selecting things in the output that are of interest. You can thing of grep as a filter that can act either inclusively or exclusively depending on what you want from the output.

The sort command can sort data based on rules. In this case we're letting it behave in its default mode, so it's sorting the data based on an alphanumeric set of sorting rules.

share|improve this answer
1  
I almost got it, well played –  Creek Jun 4 at 11:21
    
For small file sizes it doesn't matter, of course, but I guess you agree that it's kind of crazy in the general case to sort a file in order to count entries. Especially if the sorting is used for the output formatting only... –  Hauke Laging Jun 4 at 11:31
    
@HaukeLaging - this appears to be homework, so the approach I suggested would be the aim of the instructor (I'm guessing) to solve this Q. But for larger files yes you're absolutely right. –  slm Jun 4 at 11:36
    
You'll need to force the locale to C for both sort and uniq for that to work properly. Try printf '%s\n' ① ② | sort | uniq -c in a typical GNU system deployment for instance. –  Stéphane Chazelas Jun 4 at 13:09
1  
Dang, slm. I saw the question and thought uniq -c! Oh well, well done. –  mikeserv Jun 4 at 21:22

If you are looking for one string only:

grep -cxF dog

or

echo "dog: $(grep -cxF dog)"

An efficient solution for a complete list:

> awk '{a[$0]++}; END {for (val in a) print val ": " a[val];}' file
spider: 2
cat: 4
fly: 1
dog: 5
share|improve this answer
    
Note that that awk one will not necessarily be faster than LC_ALL=C sort | LC_ALL=C uniq -c (in my tests on a 5400000 lines file, 650644 unique, on a GNU system, it isn't) and may use-up the memory and/or reach a limitation in awk depending on the awk implementation. –  Stéphane Chazelas Jun 4 at 13:21
    
@StéphaneChazelas I cannot comment on the quality of certain awk implementations but the idea that awk which needs memory according to the number of unique lines whereas sort has to store the whole file, seems a bit strange to me. –  Hauke Laging Jun 4 at 13:33
3  
sort works in batches saved in temp files to avoid filling up the memory. It may fill up the tmp filesystem, but it will not cripple the system. –  Stéphane Chazelas Jun 4 at 13:35

Rearrange the data with sort and count the matching occurences using uniq -c

sort < input.txt | uniq -c
share|improve this answer
    
You'll need to force the locale to C for both sort and uniq for that to work properly. Try printf '%s\n' ① ② | sort | uniq -c in a typical GNU system deployment for instance. –  Stéphane Chazelas Jun 4 at 13:10

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.