Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have a comma separated file with multiple fields.



So, does anyone know the answer?

share|improve this question

closed as unclear what you're asking by slm, Hauke Laging, jasonwryan, Renan, Evan Teitelman Jul 3 '13 at 23:10

Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question.If this question can be reworded to fit the rules in the help center, please edit the question.

What do you mean by unique line? As far as I can tell, each of your lines is unique. Could you show us your expected output? Do you want only one of the ATTG lines? – terdon Jul 3 '13 at 14:23
I agree. Please include what you would like to see being outputted from the above and explain why. – Oli Jul 3 '13 at 14:43

Say you need to print unique lines based on column 1 (candidate1, candidate2 etc). Note that this prints the first instance of a line with candidate1 etc. To print unique lines based on column 2, change $1 to $2 below.

awk -F, '!($1 in arr){print} {arr[$1]++}' file_name
share|improve this answer

If all the fields are comma separated like this one

1, candidate1,12,56,ATTG
2, candidate2,45,90,ATTG 
3, candidate3,90,140,ATTG 
4, candidate4,2,36,AGCT

Then you do something like this one

$ grep -E 'candidate1|candidate3' file_name | awk -F, '{print $0}'
1, candidate1,12,56,ATTG
3, candidate3,90,140,ATTG

another example for a single line

$ cat filename | egrep 'candidate3' | awk -F, '{print $0}'
3, candidate3,90,140,ATTG
share|improve this answer
I deleted you mention of awk taking comma as default since it certainly does not on my system and I have never heard of it doing so. Perhaps it does for your awk implementation but is is not a general rule. – terdon Jul 3 '13 at 14:22
I am downvoting this answer because your awk is doing nothing besides print the line. You are using grep to find presence in the line, but the question is about particular column values (which awk is well suited for). – jordanm Jul 3 '13 at 15:32
to jordanm, the question was to print unique line based on column value. So, my answer is correct. – mezi Jul 3 '13 at 15:57
Yes, but @jordanm's question still stands, what is your awk supposed to be doing? It does not change the output of grep in any way. In any case, you are not looking for anything unique, you are just printing ALL lines that contain "candidate3", combined with a useless use of gawk and a useless use of cat. Your solution is basically grep 'candidate1\|candidate3'. – terdon Jul 3 '13 at 16:43
@mrahman, No it does not print unique lines based on column value. It prints lines matching a regex pattern. What if he wanted 90 from column 3? If you replaced your regex with "90" you would get lines 2 and 3. – jordanm Jul 3 '13 at 16:46

Not the answer you're looking for? Browse other questions tagged or ask your own question.