Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I know of course that

cat logfile.txt | wc -l
120

will tell me the number of lines in a file.

Whereas

tail -f logfile.txt

will show me the new lines that another program writes to logfile.txt.

Is it possible to combine both so that I get a continuous updating line count of logfile.txt with standard text utilities?

I do know about

watch wc -l logfile.txt

but I do not want to re-count the whole file each time, that seems to be a waste. One would need an appended-only count every second or so and probably an \r instead of an \n at the end of line.

share|improve this question
1  
Is your file that big that recounting everything is a problem? In terms of waste: piping cat output to wc is also a big waste!! –  Bernhard Feb 19 '13 at 12:51
    
Yes, it potentially is very big. –  towi Feb 19 '13 at 12:52

4 Answers 4

up vote 16 down vote accepted

Maybe:

tail -n +0 -f file | awk '{printf "\r%lu", NR}'

Beware that it would output a number for every line of input (though overriding the previous value if sent to a terminal).

Or you can implement the tail -f by hand in shell:

n=0
while :; do 
  n=$(($n + $(wc -l)))
  printf '\r%s' "$n"
  sleep 1
done < file

(note that it runs up to one wc and one sleep command per second which not all shells have built in. With ksh93 while sleep is builtin, to get a built in wc (at least on Debian), you need to add /opt/ast/bin at the front of $PATH (regardless of whether that directory exists or not) or use command /opt/ast/bin/wc (don't ask...)).

You could use pv, as in:

tail -n +0 -f file | pv -bl > /dev/null

But beware that it adds k, M... suffixes when the number is over 1000.

share|improve this answer
    
Whow to your tail | awk solution. Know your options: -n +0 would not have occurred to me in this combination. –  towi Feb 19 '13 at 13:07
    
whoo! pv -- another useful new tool. thanks a bunch. –  towi Feb 20 '13 at 13:52
    
With grep you can add a filter to your stream: tail -n +0 -f <my.log> | grep --line-buffered <mystring> | awk '{printf "\r%lu", NR}' –  tombolinux Feb 11 at 19:58
1  
@tombolinux, awk is a superset of grep. tail -n +0 -f file | awk '/mystring/ {printf "\r%lu", ++n}' –  Stéphane Chazelas Feb 11 at 20:00

Try to count it with pure bash without wc:

a=0 ; tail -f file | while read -r line ; do ((a++)) ; echo $a ; done

or even like this to rewrite previous value:

a=0 ; tail -f file | while read -r line ; do ((a++)) ; echo -ne "\r$a" ; done
share|improve this answer

I don' believe there is anything like that. But it should be easy to whip up something along the lines of:

#!/usr/bin/perl

$for_a_while = 1;

$oldcount = -1;
$count = 0;
open($fh, "<", $ARGV[0]);

for (;;) {
  for ($curpos = tell($fh); <$fh>; $curpos = tell($fh)) {
    $count++;
  }
  if($count != $oldcount) {
    print "$count\n";
    $oldcount = $count;
  }
  sleep($for_a_while);
  seek($fh, $curpos, 0);
}

(General idea cribbed from perlfunc(1))

share|improve this answer
1  
The number would increment every time you do a printf foo >> file. You'd need to count the newline characters (like wc -l does in the shell solution I suggested), not the records returned by <$fh>. I don't think you need to use tell or seek at all. –  Stéphane Chazelas Feb 19 '13 at 15:56
    
The <$fh> reads a line by default, not records. The Perl manpage cited tells to do it this way for the sake of a possibly uncooperative environment (could depend on the filesystem, I guess NFS or other network-mounted filesystems might require a bit of prodding). –  vonbrand Feb 19 '13 at 16:10
    
Try it for yourself, upon reaching end of file, <$fh> will return a record even if it's not terminated by a newline character. So if perl is sitting at the end of the file, and someone later does a printf foo >> file, then <$fh> will return foo (not a line since it's not terminated by a newline character), and $count will be incremented even though no extra line has been added to the file. –  Stéphane Chazelas Feb 19 '13 at 16:16
    
OP was supposed to monitor logfiles written a line at a time? –  vonbrand Feb 19 '13 at 16:27
    
No, which is why your solution may not work. For instance, if the applications writing to the file buffers its output, then at any given time, the last line is likely not to be terminated so will be counted twice. –  Stéphane Chazelas Feb 19 '13 at 17:22

Continuing the awk-based solution: you might not need to see the counter ticking for each line in your log; it that's the case, you can have it like this (number would change for each 10 lines):

tail -n +0 logfile.txt | \
    awk 'a+=1{}a%10==0{printf "\r%lu", a}END{printf "\r%lu", a}'
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.