31

I'm trying to count the number of lines of output a certain program produces. The problem is, the program takes a long time to run, and I want to display the output to the user. Is there a way to count the number of lines the last command outputted?

I could do program | wc -l but that wouldn't show the output to the user. So as far as I know, I have to do program; program | wc -l - but the program takes at least a minute to run, so I don't want to have to do it more than once just to show a line count at the bottom.

EDIT:

  • Is there a way of showing the output as it happens (line by line) and then returning a count at the end?
  • How about: have the program keep track of its own output and just read that value from the variable (e.g. STDOUT_WRITE_COUNT), or log it to a file/API, at the end of the program. WDYT? – mecampbellsoup Dec 27 '16 at 2:11
40

You can use tee to split the output stream sending one copy to wc and the other copy to STDOUT like normal.

program | tee >(wc -l)

The >(cmd) is bash syntax which means run cmd and replace the >(cmd) bit with the path to (a named pipe connected to) that program's STDIN.

  • 2
    >(cmd) is ksh syntax also recognised by zsh and bash and is only using named pipes on systems that don't have /dev/fd/n. – Stéphane Chazelas Apr 17 '13 at 22:26
  • @StephaneChazelas Yes, most shells support it, but it's not in POSIX, so it can't be relied upon to be everywhere. – Patrick Apr 17 '13 at 22:28
  • Yes, I was just pointing out that process substitution was not a bash invention as the wording in your answer could let one believe. – Stéphane Chazelas Apr 17 '13 at 23:01
  • 1
    @TheLibbster Depends how you define efficient. This method does involve spawning 2 additional processes, where as sed and awk are just one. But tee and wc are both extremely small (much smaller than sed and awk). – Patrick Apr 18 '13 at 0:09
  • 1
    @TheLibbster yes, according to some simple tests I just did, it's actually about twice as fast as both the sed and awk methods. (I ddd 100mb of /dev/urandom to a file and then ran that file through each method several times) – Patrick Apr 19 '13 at 0:32
10

One option is to use awk, which can do the counting and print to stdout.

program | awk '{ print } END { print NR }'

In awk, NR is the current line number. You can accomplish the same with perl:

program | perl -pe 'END {print "$.\n"}'

Or sed:

program | sed -n 'p;$='
  • Is there a way of showing the output as it happens (line by line) and then returning a count at the end? – Libbux Apr 17 '13 at 23:10
6

You can clone stdout on stderr.

program | tee /dev/stderr | wc -l

That way, program's stdout is piped to tee to be written to stderr, which is printed on the console. tee also writes the data piped to it to its stdout, which is piped to wc.

2

my favorite option:

program | grep "" -c
  • OP may have asked something else, but I came here looking for just getting a count of number of lines output and didn't care about showing the actual output, and this does the job. Thanks! – Nikhil VJ Feb 24 at 16:04
0
tail -f /var/log/squid/access.log | ( c=0; pl() { echo $c; c=0; }; trap pl SIGHUP; while read a; do (( c=c+1 )); done ) & ( trap 'kill $! ; exit' SIGINT; trap '' SIGHUP; while true; do kill -HUP $! ; sleep 1; done)
0

This might be late. But I would just address your follow up question on how to catch the counted number in a variable.

This is what you want YOUR_VAR=$(PROGRAM | tee /dev/stderr | wc -l).

We take advantage of tee generating two streams here and direct one to /dev/stderr, which would appear on you screen, and the other to wc -l, which would report the number of lines.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.