Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

list.txt

 GETID_17049_knownids_1/2_Confidence_0.625_Length_2532
 GETID_9248_knownids_6/10_Confidence_0.439_Length_2474
 GETID_11084_knownids_3/3_Confidence_0.600_Length_1451
 GETID_15916_knownids_10/11_Confidence_0.324_Length_1825

sample1.txt

>GETID_17049_knownids_1/2_Confidence_0.625_Length_2532
sampletextforsample1
sampletextforsample1
sampletextforsample1
>GETID_18457_knownids_1/2_Confidence_0.625_Length_2532
sample2textforsample1
sample2textforsample1
sample2textforsample1
sample2textforsample1

sample2.txt

>GETID_11084_knownids_3/3_Confidence_0.600_Length_1451
sampletextforsample2
sampletextforsample2
>GETID_67838_knownids_3/3_Confidence_0.600_Length_1451
sample2textforsample2
sample2textforsample2

sample3.txt

>GETID_17049_knownids_1/2_Confidence_0.625_Length_2532
sampletextforsample3
sampletextforsample3
sampletextforsample3
>GETID_15916_knownids_10/11_Confidence_0.324_Length_1825
sample2textforsample3
sample2textforsample3

output.txt

>GETID_17049_knownids_1/2_Confidence_0.625_Length_2532
sampletextforsample1
sampletextforsample1
sampletextforsample1
>GETID_17049_knownids_1/2_Confidence_0.625_Length_2532
sampletextforsample3
sampletextforsample3
sampletextforsample3
>GETID_11084_knownids_3/3_Confidence_0.600_Length_1451
sampletextforsample2
sampletextforsample2
>GETID_15916_knownids_10/11_Confidence_0.324_Length_1825
sample2textforsample3
sample2textforsample3

I want to read each line from list.txt (by capturing bracketed values (GETID_{17049}knownids{1/2}_Confidence_1.0_Length_{2532}) and compare with sample1.txt, sample2.txt, sample3.txt has multiple line and print the contents (output.txt) from these files when it is matched with list.txt. The output should contain exact match to list.txt. Any help in awk/sed/perl is appreciated.

share|improve this question
1  
Just curious, how close does a grep -F -A 1 -f list.txt sample[123].txt | grep -vx '--' get to what you want? –  jw013 Aug 29 '12 at 18:43
    
duplicate of this? –  jw013 Aug 29 '12 at 18:45
    
I tried grep and awk script from this previous post.But it does not work because I have to compare with three sample files –  jack Aug 29 '12 at 18:58
add comment

2 Answers

up vote 1 down vote accepted

If you slightly modify the solution provided by Gilles in this question (also referred to by jw013), you can get the effect you ask for, except that the order is based on the input sequence and not identical to output.txt listed in your question:

awk -v patterns_file=list.txt '
BEGIN {
  while (getline < patterns_file)
    patterns_array[">" $0] = 1
  close(patterns_file)
}
$0 in patterns_array { print; getline; print }
' sample[1-3].txt

Output:

>GETID_17049_knownids_1/2_Confidence_0.625_Length_2532
sampletextforsample1
>GETID_11084_knownids_3/3_Confidence_0.600_Length_1451
sampletextforsample2
>GETID_17049_knownids_1/2_Confidence_0.625_Length_2532
sampletextforsample3
>GETID_15916_knownids_10/11_Confidence_0.324_Length_1825
sample2textforsample3

Edit

To make this work with multi-line records use an appropriate record separator (RS), in your case setting it to: greater-than at beginning of file (^>) or new-line followed by greater-than (\n>) or new-line at the end of the file (\n$) seem like good choices based on the input provided.

Something like this should work:

awk -v patterns_file=patterns.txt '
BEGIN {
  while (getline < patterns_file) 
    patterns_array[$0] = 1
  close(patterns_file)
  RS="^>|\n>|\n$"
}
$1 in patterns_array { print ">" $0 }
' sample[1-3].txt

Edit 2

To output each record only once, delete it from patterns_array after output:

awk -v patterns_file=patterns.txt '
BEGIN {
  while (getline < patterns_file) 
    patterns_array[$0] = 1
  close(patterns_file)
  RS="^>|\n>|\n$"
}
$1 in patterns_array { print ">" $0; delete patterns_array[$1] }
' sample[1-3].txt
share|improve this answer
    
See edit to the answer. –  Thor Aug 30 '12 at 15:08
    
now it works.The ouput prints without '>' symbol. Also the output should contain exact count as in the list.txt. –  jack Aug 30 '12 at 15:42
    
Forgot to re-add >, fixed. What do you mean by exact count? –  Thor Aug 30 '12 at 15:55
    
say if count for '>' in the list.txt, the output should also contains same count for '>', even there is repeat in list or sample –  jack Aug 30 '12 at 16:06
    
Just delete them from patterns_array after output. I'll add an edit. –  Thor Aug 30 '12 at 16:17
add comment

Here's a PERL solution. It will work for any number of files and expects the first file to be the list. It will also append the file name to the FASTA header.

#!/usr/bin/perl -w
use strict;
my $list=shift;
open(A,$list); 
my %k;
while(<A>){
    ## Remove trailing newline
    chomp;
    if ( /(\d+?)_knownids_(.+?)_.+?(\d+)$/){ 
      ## Concatenate the patterns and save in a hash
      my $pp=join("-", $1,$2,$3);
      $k{PAT}{$pp}=$_;
    }
}
close(A);
## Read each input file
my $name;
for my $f (@ARGV) {
    open(F,$f);
    while(<F>){
       ## Skip empty lines
       next if /^\s*$/;
       ## Is this a FASTA header?
       if ( /^\s*>/){
           ## If this id is in the list, keep it for this file
           if(/(\d+?)_knownids_(.+?)_.+?(\d+)$/){ 
              $name=join("-", $1,$2,$3);
           }
           ## Skip the sequences we are not interested in
           else{$name="foo"}
       }
       ## Collect the sequence
       else {
           if (defined($k{PAT}{$name})) {
           $k{$f}{$name}.=$_;
           }   
       } 
    }
    close(F);
}
## For each unique pattern found in list.txt
foreach my $pat (keys(%{$k{PAT}})) {
    ## For each of the files passed as arguments
    foreach my $file (@ARGV) {
    ## If the pattern was found in that file, print
    if (defined($k{$file}{$pat})) {
          print ">$k{PAT}{$pat}_$file\n";  
          print "$k{$file}{$pat}"
        }
    }
}

If the script is saved as compare.pl, do:

$ ./compare.pl list.txt sample1.txt sample2.txt sample3.txt sampleN.txt

The output is:

> GETID_11084_knownids_3/3_Confidence_0.600_Length_1451_sample2.txt
sampletextforsample2
> GETID_17049_knownids_1/2_Confidence_0.625_Length_2532_sample1.txt
sampletextforsample1
> GETID_17049_knownids_1/2_Confidence_0.625_Length_2532_sample3.txt
sampletextforsample3
> GETID_15916_knownids_10/11_Confidence_0.324_Length_1825_sample3.txt
sample2textforsample3
share|improve this answer
    
there is some warning:Use of uninitialized value in join or string at line 26. –  jack Aug 30 '12 at 15:02
    
@jack, that warning probably means you have a FASTA header that does not contain "knownids". I have edited the script to avoid these errors. Make sure ALL your FASTA headers conform to the pattern you gave in you question. –  terdon Aug 30 '12 at 15:23
    
Thanks.now it works.I need exact counts as in the list.txt even samples contains identical ids. –  jack Aug 30 '12 at 15:41
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.