I have a comma separated file with multiple fields.



So, does anyone know the answer?

  • 3
    What do you mean by unique line? As far as I can tell, each of your lines is unique. Could you show us your expected output? Do you want only one of the ATTG lines?
    – terdon
    Jul 3, 2013 at 14:23
  • 1
    I agree. Please include what you would like to see being outputted from the above and explain why.
    – Oli
    Jul 3, 2013 at 14:43

2 Answers 2


Say you need to print unique lines based on column 1 (candidate1, candidate2 etc). Note that this prints the first instance of a line with candidate1 etc. To print unique lines based on column 2, change $1 to $2 below.

awk -F, '!($1 in arr){print} {arr[$1]++}' file_name

If all the fields are comma separated like this one

1, candidate1,12,56,ATTG
2, candidate2,45,90,ATTG 
3, candidate3,90,140,ATTG 
4, candidate4,2,36,AGCT

Then you do something like this one

$ grep -E 'candidate1|candidate3' file_name | awk -F, '{print $0}'
1, candidate1,12,56,ATTG
3, candidate3,90,140,ATTG

another example for a single line

$ cat filename | egrep 'candidate3' | awk -F, '{print $0}'
3, candidate3,90,140,ATTG
  • I deleted you mention of awk taking comma as default since it certainly does not on my system and I have never heard of it doing so. Perhaps it does for your awk implementation but is is not a general rule.
    – terdon
    Jul 3, 2013 at 14:22
  • I am downvoting this answer because your awk is doing nothing besides print the line. You are using grep to find presence in the line, but the question is about particular column values (which awk is well suited for).
    – jordanm
    Jul 3, 2013 at 15:32
  • to jordanm, the question was to print unique line based on column value. So, my answer is correct.
    – mezi
    Jul 3, 2013 at 15:57
  • 1
    Yes, but @jordanm's question still stands, what is your awk supposed to be doing? It does not change the output of grep in any way. In any case, you are not looking for anything unique, you are just printing ALL lines that contain "candidate3", combined with a useless use of gawk and a useless use of cat. Your solution is basically grep 'candidate1\|candidate3'.
    – terdon
    Jul 3, 2013 at 16:43
  • @mrahman, No it does not print unique lines based on column value. It prints lines matching a regex pattern. What if he wanted 90 from column 3? If you replaced your regex with "90" you would get lines 2 and 3.
    – jordanm
    Jul 3, 2013 at 16:46

Not the answer you're looking for? Browse other questions tagged .