Count appearance of a variable in another file

Question

I do have the following situation:

File1 looks like:

A  
B  
C  

File2 looks like:

chr1 \t 1234523 \t A  
chr3 \t 1234231 \t A  
chr6 \t 121233 \t A  
chr1 \t 1126685 \t B  
chr1 \t 15834523 \t B  
chr4 \t 12345647 \t C  
chr12 \t 1456523 \t C  

I would like to get the output:

A \t 3 
B \t 2  
C \t 2  

I know that I could do it like

grep A File2 | wc -l

However, than I would need to do it for each single line in File1 (which are 700).

How can I automate that?

(Sorry, I am a total beginner)

Answer 1

Assuming that \ts are actually tabs, and that the occurrences are on the same column, and that A only matches A, not AA. Let a be the file with A,B,C and b the file where you want to count matches on (the second one you provided).

• First, you need to get only the possible matches from b, ignoring everything else. This is the third column of b, so we can use cut that is, well, intended to cut parts of a file

  cut -f 3 b

• Then, you need to turn this into a list of occurences and their counts: you can sort and use uniq to count those, on the output of cut

  sort | uniq -c

• Finally, you did this for all values in b, but you only want those from a. You can use join which joins two different files on common fields (in this case, the first and only field of a (it seems to do that by default) and the second field (2) of b, which is the second file (-2)

  join -2 2 a result-from-b

You can chain this in several different ways, a possible way is using named pipes from bash's process substitution:

join -2 2 a <(cut -f 3 b | sort | uniq -c)

This should at least be better than individual greps, as you only process b thrice (remove other columns, sort, and uniq) and then I suppose the join will only read each file once, as it requires the inputs to be sorted. Of course this relies on the assumptions I made (and you also have to sort a, but that's just <(sort a) instead of a if it was not sorted before.

Answer 2

It looks from your sample input that you want to count each distinct value in the last field of tab-separated records. Here's an awk snippet that does this.

awk -F '\t' '
     {++a[$NF]}
     END {for (x in a) {print x "\t" a[x]}}
' File2

Answer 3

You can do this with a while loop

while read arg < FILE1; do echo -n -e "$arg\t"; grep "$arg" FILE2 | wc -l; done

This will read FILE1 and for each line the for loop will store the string into the variable $arg.

It will then echo the $arg (-n means don't insert a line return (\n) at the end, -e means execute excaped characters).

Then it will display the number of occurrences it found $arg in FILE2.