Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I do have the following situation:

File1 looks like:

A  
B  
C  

File2 looks like:

chr1 \t 1234523 \t A  
chr3 \t 1234231 \t A  
chr6 \t 121233 \t A  
chr1 \t 1126685 \t B  
chr1 \t 15834523 \t B  
chr4 \t 12345647 \t C  
chr12 \t 1456523 \t C  

I would like to get the output:

A \t 3 
B \t 2  
C \t 2  

I know that I could do it like

grep A File2 | wc -l

However, than I would need to do it for each single line in File1 (which are 700).

How can I automate that?

(Sorry, I am a total beginner)

share|improve this question
add comment

3 Answers

Assuming that \ts are actually tabs, and that the occurrences are on the same column, and that A only matches A, not AA. Let a be the file with A,B,C and b the file where you want to count matches on (the second one you provided).

  • First, you need to get only the possible matches from b, ignoring everything else. This is the third column of b, so we can use cut that is, well, intended to cut parts of a file

    cut -f 3 b

  • Then, you need to turn this into a list of occurences and their counts: you can sort and use uniq to count those, on the output of cut

    sort | uniq -c

  • Finally, you did this for all values in b, but you only want those from a. You can use join which joins two different files on common fields (in this case, the first and only field of a (it seems to do that by default) and the second field (2) of b, which is the second file (-2)

    join -2 2 a result-from-b

You can chain this in several different ways, a possible way is using named pipes from bash's process substitution:

join -2 2 a <(cut -f 3 b | sort | uniq -c)

This should at least be better than individual greps, as you only process b thrice (remove other columns, sort, and uniq) and then I suppose the join will only read each file once, as it requires the inputs to be sorted. Of course this relies on the assumptions I made (and you also have to sort a, but that's just <(sort a) instead of a if it was not sorted before.

share|improve this answer
    
Thanks so much! That worked perfectly well! And also thanks for the detailed explanation! –  user30012 Jan 10 '13 at 15:25
    
@user30012 if this answer solved your problem, please remember to mark it as accepted and/or upvote it instead of posting a thank you comment. That is the way thanks are expressed on SE sites. –  terdon Feb 11 '13 at 18:13
add comment

It looks from your sample input that you want to count each distinct value in the last field of tab-separated records. Here's an awk snippet that does this.

awk -F '\t' '
     {++a[$NF]}
     END {for (x in a) {print x "\t" a[x]}}
' File2
share|improve this answer
add comment

You can do this with a while loop

while read arg < FILE1; do echo -n -e "$arg\t"; grep "$arg" FILE2 | wc -l; done

This will read FILE1 and for each line the for loop will store the string into the variable $arg.

It will then echo the $arg (-n means don't insert a line return (\n) at the end, -e means execute excaped characters).

Then it will display the number of occurrences it found $arg in FILE2.

share|improve this answer
    
No need to pipe to wc -l and count lines; grep can already count occurrences with the -c option. –  laebshade Jan 10 '13 at 4:17
    
@laebshade you're right, however, I used wc -l in case the search string matched more than once per line in FILE2. –  h3rrmiller Jan 10 '13 at 13:10
    
That would skew the results. If you have more than one instance per line, suddenly your counted results aren't accurate. –  laebshade Jan 11 '13 at 22:58
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.