Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Suppose there is a column of numeric values like following:



I want the output:


That is, only the repeated lines. Is there any command line tools to find this out in Linux? (NB: The values are numerically sorted).

share|improve this question
See man uniq. – jasonwryan Oct 22 '12 at 5:29
up vote 46 down vote accepted

You can use uniq(1) for this:

uniq -d file.txt

This will print out the duplicates only. The input file needs to be sorted such that all duplicates are consecutive (which they appear to be), so run it through sort first if it is not.

share|improve this answer
what if I want the triplicates only to be printed? – MiNdFrEaK Oct 22 '12 at 7:55
@MiNdFrEaK sort | uniq -c | grep '^\s*3\s' | sed 's/^\s*[0-9]*\s*//' for triplicates; replace "3" with any N for N-plicates – full.stack.ex Oct 22 '12 at 8:10
@MiNdFrEaK sort | uniq -c | sed -n 's/^[[:blank:]]*3[[:blank:]]\{1,\}//p' for triplicates – user24222 Oct 22 '12 at 11:03
@camh can you do this on csv files as well? only values of a certain column? – NumenorForLife Jun 4 '15 at 12:20

Using uniq and awk:

cat File1  | uniq -c | awk '$1 > 1 { print $2 }'
share|improve this answer
This work, but I don't see why you pipe the output of cat? – Bernhard Oct 22 '12 at 17:52

Execute this: perl -ne 'print if $a{$_}++' filename.txt

share|improve this answer
It gives 3\n3\n4\n\4n for the input File1 which is obviously wrong. – yaegashi Jul 10 '15 at 0:03
the perl snip i find myself revisiting provides the number of incidences of each line so it can be piped, sorted, and filtered as needed: perl -ne '$a{$_}++; END { while(($k,$v)=each %a){printf "%d\t%s", $v,$k}}' filename – Theophrastus Jun 2 at 22:00

protected by Stephen Kitt Jun 5 at 12:13

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.