filter and count with grep in a single pass?

Question

I am using grep at the end of a set of piped commands to filter some rows out of a file. Afterwards I will want to do some arithmetic based on the number of remaining rows.

e.g.

chrN="chr1|chr2|chr3|chr4|chr5|chr6|chr7"
otherCommands | grep -Ew $chrN  > $ChIP".bed"
count_pos=`wc -l $ChIP"bed" | awk '{print 1000000/$1}'`

It strikes me as very wasteful after piping together lots of command to then write it to a file and then read through it again just to collect one number. .. but I don't know how to do that.

So my question is how do I save the number of rows that passed the grep filter to the variable count_pos and save the filtered rows to the file too?

Answer 1

Do everything with awk. Initialize counter, count and print (grep+count) with pattern matching and output count at the end:

awk 'BEGIN { c=0} pattern{ c++; print $0 } END{ print 1000000/c }'

You don't have to print if you don't want to. Or, you could use print $0 > output_file inside awk to print grepped results in a file, and count on stdout.

Answer 2

You can use a tee to write to both a file and stdout:

chrN="chr1|chr2|chr3|chr4|chr5|chr6|chr7"
count_pos=$(otherCommands |
  grep -Ew "$chrN" |
  tee "$ChIP.bed" |
  wc -l |
  awk '{print 1000000/$1}')