0

I stumbled on this anomaly regarding the sort command and I can't find an explamation to why it happens -

running the following command:

cat ~/.bash_history | uniq -c | sort -h -r | head

Should output the top most used 10 command sequences I ran in the system, but in reality it fails to do so, the output shows as if I ran all commands once.

but upon running:

cat ~/.bash_history | sort | uniq -c | sort -h -r | head

It seems to work, and I can't understand why, can anyone please explain?

2 Answers 2

3

uniq only filters adjacent lines. Say you have a file like this:

$ cat foobar.txt
foo
bar
foo

with two non-adjacent foo lines. Then uniq -c counts each of them once, while sorting the file first means uniq -c sees them as adjacent and counts them as one line that appears twice:

$ uniq -c foobar.txt
  1 foo
  1 bar
  1 foo
$ sort foobar.txt | uniq -c
  1 bar
  2 foo
1
  • Ah! this explains it, thank you so much!
    – Danus
    Commented Nov 3, 2019 at 20:31
2

uniq is a POSIX utility. You used uniq -c and it works like this:

The uniq utility shall read an input file comparing adjacent lines, and write one copy of each input line on the output. The second and succeeding copies of repeated adjacent input lines shall not be written. The trailing of each line in the input shall be ignored when doing comparisons.

Repeated lines in the input shall not be detected if they are not adjacent.

[…]

-c
Precede each output line with a count of the number of times the line occurred in the input.

(source, emphasis mine)

This means that non-adjacent lines are treated as if they were different. In your use case you need to sort first to make identical lines adjacent. Only then uniq will do what you want.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .