0

So I have a large file which is in a columns comprised of cities. I want to write a program that will find the total number of instances of each city and display it in a new file. How should I go about doing that.

This is an example, there are way more values than this

If the file is like this

City           
Manhattan   
Cork       
Manhattan  
Chennai
Chennai

How output should look

City
Manhattan 2 
Cork      1
Chennai   2
3
1

Do it with datamash:

datamash -g1 -s -H count 1 < infile

Output:

GroupBy(City)   count(City)
Chennai 2
Cork    1
Manhattan   2
0

Use Linux built-in commands sort and uniq:

cat DATAFILE | sort | uniq --count

This will give you something like:

  2 Chennai
  1 City           
  1 Cork       
  2 Manhattan

Explanation: This uses the command uniq which normally removes repeated lines, keeping only one instance. With the option "--count" it issues an additional count of the number of repetitions. In order for uniq to work, lines need to be sorted - otherwise duplicates would not be in consecutive lines as necessary. sort does exactly that, sort lines by alphabet.

Do you need these in a different column order? Does the first line have to be ignored? If so, please also let us know whether city names may have more than one word in them.

Not the answer you're looking for? Browse other questions tagged or ask your own question.