1

Hi have a tabular file (separator =\t) A.txt :

205 toto patho
207 tata benign
208 titi likely_patho

and B.txt:

210 lolo patho
211 lala benign
212 lili benign

And I would like to have a file that print the value of the second column if the third column are "patho" or "likely_patho". With this example, we have :

A toto;titi
B lolo

In order to do that, I've done :

for bla in *.txt; do
r="$(basename -s ".txt" $bla)"
awk -v filename=$r '($3=="patho" || $3=="likely_patho") {print filename"\t"$2}' $bla >> result.txt ;
done

But this code give me one line per match in the event that a file contain several match (like A.txt) :

A toto
A titi
B lolo

How can I have the proper output ? Thanks

1
  • 1
    Please edit your question to add a 3rd input file that has no patho or likely_patho values to the example so we can see what should be output for that case (no file name? a file name on it's own? something else?). Also tell us what the output should be if an input file was empty.
    – Ed Morton
    Jun 30 at 14:31

5 Answers 5

2

Using any awk:

$ cat tst.awk
BEGIN {
    split(t,tmp)
    for ( i in tmp ) {
        tgts[tmp[i]]
    }
    FS = OFS = "\t"
}
FNR == 1 {
    fname = FILENAME
    sub(/\.[^.]*$/,"",fname)
}
$3 in tgts {
    hits[fname] = (fname in hits ? hits[fname] ";" : "") $2
}
END {
    for ( fname in hits ) {
        print fname, hits[fname]
    }
}

$ awk -v t='patho likely_patho' -f tst.awk *.txt
A       toto;titi
B       lolo
1
  • Note that this answer will store most of the output in one array. It will consume a huge amount of memory for a huge output list. Jul 6 at 2:22
1

Try :

awk '  FNR==1 {  f=FILENAME;
                 sub(/\.[^.]*$/,"",f);
                 printf "%s%s\t",aline,f;
                 aline="\n";
                 s=""
       }
       ($3=="patho" || $3=="likely_patho"){
                 printf "%s%s",s,$2;
                 s="; "
       }
       END{print ""}
    ' ./*.txt
0
0

try (can be one lined)

awk -v filename="$r" 'BEGIN { string=filename "\t" }
          ($3=="patho" || $3=="likely_patho") {printf string $2; string=";" }
          END { printf "\n" } ' $bla >> result.txt ;

I use string as a prefix, it will print filename first, then semicolon after.

You need to add final newline.

1
  • 2
    Quote your shell variables, see mywiki.wooledge.org/Quotes, and always do printf "%s", $<anything> instead of printf $<anything> for any input data as the latter will fail when the input contains printf formatting characters like %s. Also, print "" is better than printf "\n" as the former uses whatever value ORS has while the latter hard-codes the value you hope/assume that ORS has. This approach requires a surrounding shell loop, though, which is slow and unnecessary.
    – Ed Morton
    Jun 30 at 14:39
0
grep -HoP '[^\t]+(?=\t(?:likely_)?patho$)' -- *?.txt|
sed 's/\.[^.]*:/\t/'|
awk -F '\t' '
NR==1||prev!=$1{
  printf "%s%s", sep, $0
  prev=$1;sep=ORS
  next 
}
{ printf ";%s", $2 }
END {print ""}'

A   toto;titi
B   lolo
0

Using perl (with extra linefeeds for readability):

$ perl -MFile::Basename -F'\t' -le '
$f = fileparse($ARGV, qw(.txt)) if $. == 1;
if ($F[2] =~ /^(likely_)?patho$/) {
  push @{ $files{$f} }, $F[1]
};
close(ARGV) if eof;   # close each input file and reset the line counter $. at eof

END {
  foreach (sort keys %files) {
    print "$_\t", join(";",@{ $files{$_} })
  }
}' A.txt B.txt
A       toto;titi
B       lolo
  • -MFile::Basename tells perl to load the File::Basename module. This is a core perl module and is included with perl.

  • -F sets the field separator (tab) and also enables both perl's -a auto-split mode (splits each input line into an array called @F, similar to awk), and perl's -n option to iterate over its input (behaves similarly to both sed -n and awk).

  • -l enables automatic processing of line-endings (\n by default). In short, it removes newline characters from the end of each input line (using perl's chomp() function) and adds them to the end of every print statement.

  • All perl command-line options are documented in man perlrun.

The script iterates over each input line and, whenever it finds a match in the third field ($F[2] - perl array indices start from zero) it adds the second field to a Hash-of-Arrays (HoA) called %files. This is a hash (associative array) where the key is the base filename and the value is an array of 2nd-field strings. See the man pages for perldata, perllol, and perldsc for more information about perl data structures.

When all the input has been read and processed, it output the data in the requested format, sorted by filename.


NOTE: The ... if $. == 1; in conjunction with the close(ARGV) if eof line ensures that the base filename is only extracted on the first line of each new file. This isn't required, just a minor optimisation that would only be beneficial if you have very large input files. If you prefer a slightly shorter one-liner or if performance isn't an issue, delete the close(ARGV) ... line and the if $. == 1 conditional.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.