2

I am using grep at the end of a set of piped commands to filter some rows out of a file. Afterwards I will want to do some arithmetic based on the number of remaining rows.

e.g. 

chrN="chr1|chr2|chr3|chr4|chr5|chr6|chr7"
otherCommands | grep -Ew $chrN  > $ChIP".bed"
count_pos=`wc -l $ChIP"bed" | awk '{print 1000000/$1}'`

It strikes me as very wasteful after piping together lots of command to then write it to a file and then read through it again just to collect one number. .. but I don't know how to do that.

So my question is how do I save the number of rows that passed the grep filter to the variable count_pos and save the filtered rows to the file too?

Improve this question

2 Answers 2

Reset to default
6

Do everything with awk. Initialize counter, count and print (grep+count) with pattern matching and output count at the end:

awk 'BEGIN { c=0} pattern{ c++; print $0 } END{ print 1000000/c }'

You don't have to print if you don't want to. Or, you could use print $0 > output_file inside awk to print grepped results in a file, and count on stdout.

Improve this answer
4
  • thx, that is a good option but I think the above is exactly what I was looking for.
    – Stephen Henderson
    Mar 14, 2014 at 16:37
  • 2
    This replaces grep, tee, wc and awk in a single command.
    – orion
    Mar 14, 2014 at 16:40
  • indeed you maybe right, it maybe better, except for my innate fear of awk ;) I will test it though!
    – Stephen Henderson
    Mar 14, 2014 at 16:43
  • Then why use awk to do arithmetics? :) I used to fear awk too. But it's just a very logical and powerful tool - it's like perl, except it's easier to read and works better in a pipe.
    – orion
    Mar 14, 2014 at 16:53
5

You can use a tee to write to both a file and stdout:

chrN="chr1|chr2|chr3|chr4|chr5|chr6|chr7"
count_pos=$(otherCommands |
  grep -Ew "$chrN" |
  tee "$ChIP.bed" |
  wc -l |
  awk '{print 1000000/$1}')
Improve this answer
3
  • OK, thats a great answer.. if I could just push it a little further?.. say I wanted not to write to the file but instead save the count in count_pos then use that as a variable in the next command of my now very long pipe?
    – Stephen Henderson
    Mar 14, 2014 at 16:41
  • @Stephen, Not sure what you mean. This will save the count in count_pos ($() is usually a better option than backticks).
    – Graeme
    Mar 14, 2014 at 16:46
  • No forget what I said.. I have to pass through all the data to count it .. Doh! So I might as well stop the pipe there. Apols and thx again..
    – Stephen Henderson
    Mar 14, 2014 at 16:49

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .