Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have a gene list file. Some thing like this

    SWT21
    SSA1
    NRP1
    EFB1
    TFC3
    MDM10

I have another file which also contains the names of these genes in my list along with other essential information about them. The second file looks like this:

chrI    147593  151166  YAL001C -   TFC3
chrI    143706  147531  YAL002W +   VPS8
chrI    142173  143160  YAL003W +   EFB1
chrI    140759  141407  YAL004W +   YAL004W
chrI    139502  141431  YAL005C -   SSA1
chrI    137697  138345  YAL007C -   ERP2
chrI    136913  137510  YAL008W +   FUN14
chrI    135853  136633  YAL009W +   SPO7
chrI    134183  135665  YAL010C -   MDM10

I want to extract those lines in the 2nd file which have gene names as are present in first file.

share|improve this question
1  
None of the genes in your gene list is present in your gff-like file. Could you give a better example please? –  terdon Mar 24 at 20:32
    
@terdon I changed the list –  user3138373 Mar 24 at 20:37
    
I really recommend you do not use the answer you accepted. It will fail for cases where one gene's name is included in another. –  terdon Mar 24 at 20:47
    
@terdon Thanks. Your solution works fine as of now for me. –  user3138373 Mar 24 at 20:50

4 Answers 4

All you need is a simple grep:

grep -Fwf gene_list.txt gene_info.txt

The options used are:

  • -w : Search for whole words, this ensures that the gene name ERK1 will not match the gene ERK12 (-w is not a standard option but is fairly common)
  • -f : Read the patterns to be searched for from a file. In this case gene_list.txt.
  • -F : Treat the patterns as strings, not as regular expressions. This ensures that a gene name like TOR* (if such a thing existed) would not match TORRRRRR.

NOTE: This assumes that there are no spaces around the gene names in your list. If there are, you will need to remove them first (here with GNU sed):

sed -i 's/ //g' gene_list.txt
share|improve this answer

From an example on the awk wiki:

awk 'FNR==NR {arr[$0];next} $6 in arr' gene_list info_list

Explanation copied from the wiki entry:

  • FNR == NR: This test is true when the number of records is equal to the number of records in the file. This is only true for the first file, for the second file NR will be equal to the number of lines of file1 + FNR.
  • arr[$0]: This is a classic technique to create an array element index by the whole line. This will create an array with the first names of file1.
  • next: this will skip to the next record so no more processing is done on file1.
  • $6 in arr: This will only happen on the records of file2 because of the next. If $1 is present in arr, ie in file1, the default action will be executed and the line will be printed.
share|improve this answer

For more robustness you could alter the gene list into regular expressions to match at the end of the line. This should work (and be POSIX compatible):

sed 's/[[:space:]]*$/[[:space:]]*$/' gene_list | grep -wf - gene_info

Or with leading space in the gene_list (as per the example in the question):

sed 's/$/[[:space:]]*$/; s/[[:space:]]//' gene_list |
  grep -wf - gene_info

Of course this won't work if there are characters included in the gene names which are interpreted by regular expressions. Atlhough, if this is the case then you need to add extra replace expressions to sed to escape them, eg s:\[:\[:.

share|improve this answer
    
@terdon, added the -w option to grep, this will fix it. –  Graeme Mar 24 at 20:51
    
Thanks, downvote retracted, upvote added. this is still a bad solution for this particular issue hoewever. Gene names can contain . and other meta characters, you really don't want to use regexes. No way you can know this from the question of course, but that's another reason why I recommended the OP not use this. I used to use a similar approach in my pipelines and it took me ages to figure out why I was getting unexpected results. –  terdon Mar 24 at 22:05
    
@terdon, jasonwryan's solution is probably better if you want to match a in particular column I guess. That said you can always something like `grep '[^[:alnum:][:space:]]' to check for potentially problematic characters. –  Graeme Mar 24 at 22:29
1  
Take my word for it. There is basically no guessing what can be in a gene name. I have literally spent hours of my life trying to sort out errors like this before discovering grep's -F option. But yes, the awk approach can indeed be better, as it often is when there are columns involved. It can also be problematic though when you are working through several gigabytes worth of data, since the array needs to be kept in memory. –  terdon Mar 24 at 22:34

Assuming the gene list is list.txt and the info file is info.txt, as well as that the gene names never contain whitespace:

for item in $(cat list.txt); do grep "\s$item$" info.txt ; done

Output:

chrI    139502  141431  YAL005C -   SSA1
chrI    142173  143160  YAL003W +   EFB1
chrI    147593  151166  YAL001C -   TFC3
chrI    134183  135665  YAL010C -   MDM10
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.