3

I have a file with multiple columns and want to identify those where specific column values (cols 3-6) have been duplicated.

The following code finds the duplicates but I want to display both instances, not just the second. The other column values (cols 1, 2 and 7+) can be different between the 2 lines hence the need for me to view both instances.

awk 'seen[$3, $4, $5, $6]++ == 1' filename
2
  • awk 'seen[$3, $4, $5, $6]++ >= 1' filename
    – cuonglm
    Aug 25, 2015 at 17:06
  • 1
    Still only shows the second instance - not both
    – Bob
    Aug 25, 2015 at 17:15

1 Answer 1

9

uniq is the correct tool for that:

uniq -D -f2 file

Where:

  • -D - prints all duplicates
  • -f2 - avoid comparing the first 2 fields

Edit: If the fields 7 and above are not to be compared, you need awk:

awk 'n=x[$3,$4,$5,$6]{print n"\n"$0;} {x[$3,$4,$5,$6]=$0;}' file
  • The array item x[] (columns 3-6) is checked. If it's already set run the part in {...} (in the same statement the n variable is set to the value of that array item)
  • In the brackets {...}: The n variable and the current line $0 are printed.
  • Then we set the x[] array item for the next iteration to the current line contents, for later comparsion.
3
  • Wouldn't -f2 still compare columns beyond col6? Also, what if I need to compare non-consecutive columns?
    – Bob
    Aug 25, 2015 at 17:03
  • Doesn't it presuppose that the file is sorted on fields 3 to 6 also? Aug 25, 2015 at 17:16
  • The edited version has done the job. Thanks @chaos!
    – Bob
    Aug 25, 2015 at 17:31

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .