43

I want to do some simple computation of the number of lines per minute added to a log file. I also want to store the count for each second.

What I need is the output of the following command as a list which will be updated every second:

watch -n1 'wc -l my.log'

How can I output the 'update' of the 'watch' command as a list?

6 Answers 6

51

You can use the -t switch to watch which causes it not to print header. However, that will still clear the screen so you might be better off with a simple shell loop:

while sleep 1; do
    wc -l my.log
done

One of the advantages is, that you can easily add other commands (e.g. date) and/or pipe the output through sed to reformat it. By the way, if you swap sleep 1 with wc in the loop, it will automatically terminate on errors.

9
  • However note that it will not do it exactly every second (with zsh or ksh93, you could adjust the sleeping time so as to account for the drift incurred by running the commands in the loop, though) Nov 19, 2012 at 11:51
  • 2
    @StephaneChazelas neither will it with wait - just try watch -n 1 "sleep 5".
    – peterph
    Nov 19, 2012 at 12:26
  • Indeed (I've checked both procps and busybox implementations). I thought that was the only thing watch was useful for, but it doesn't give you even that so your solution is as good as a watch based one, but neither answers the "rate at which a log file grows" question with great accuracy. Nov 19, 2012 at 13:22
  • Well, if lines per minute are the ultimate goal, then sampling it once every 10 seconds is more than enough imho - so the overhead is not that terrible (unless the file grows really big, of course). And actually from within the loop, one can print timing information (even both before and after the command finishes if necessary), and then the accuracy can get better (orders of magnitude) even for bigger files.
    – peterph
    Nov 19, 2012 at 13:28
  • 2
    @peterph watch has a -p option that will do that right, if at all possible (obviously, you can't do a command that takes 5 seconds every 1 second, if you're not allowed multiple simultaneous). I know, I wrote it :-P
    – derobert
    Jun 19, 2014 at 17:46
14

An old question, but I just found a very easy answer:

watch -n1 'wc -l my.log | tee -a statistics.log'

This will execute your wc each second, add its output to the statistics.log file, and also show it on the screen.
So, you'll end up with a file filled with numbers, representing the successive number of lines of my.log.

2
  • 1
    Please note that this command is watch "($MYCMD | tee -a $MYLOG)", not watch "($MYCMD)" | tee -a $MYLOG. If you got this wrong like I did, the output is going to be very very confusing. Another thing to note here is that this command does not add the timestamp of each command's execution by default, so the answer with the loop might still work better for you.
    – ffledgling
    Aug 19, 2019 at 15:34
  • Yes, quotes were meaningful in my answer, as well as they are in computer science ;)
    – Orabîg
    Oct 8, 2022 at 22:12
5

You could create a script that does it for you. I called mine keep (like, keep doing it) and put it on the bin path.

This is my script:

#!/bin/bash
echo "Repeating command $* every second"
while sleep 1; do
    "$@"
done
1
  • 2
    You should use "$@" to execute the command (arguments) instead of unquoted $*. This way you'll have the shell keep quoted arguments, etc. as the user would expect them. FTFY. Mar 3, 2016 at 22:56
3

How about

tail -f file.log | pv -rl > /dev/null
3

Try the following:

watch -n1 'wc -l my.log >> statistics.log'
1

I came across this question when I was trying to get better/logged output from du -sh $data_path. I used the "while command, do sleep" pattern found here, but used some complex AWK to give the output I wanted.

while du -sh $data_path; do sleep 1; done | awk '
$1 != size {
    size=$1;
    path=$2;
    time=systime();
    seconds=time-prevtime;
    if(seconds < 1000000000){
        seconds=seconds" seconds"
    }else{
        seconds=""
    }
    print size, path, strftime("%m/%d/%Y@%H:%M:%S", time), seconds; 
    prevtime=time
}'

I actually did this as a oneliner, which is why there are semicolons. But to make it readable, I broke it out. The output looks like:

502G /var/lib/cassandra/dump/ 05/22/2018@04:46:17
503G /var/lib/cassandra/dump/ 05/22/2018@04:46:59 42 seconds
504G /var/lib/cassandra/dump/ 05/22/2018@04:47:57 58 seconds
505G /var/lib/cassandra/dump/ 05/22/2018@04:48:55 58 seconds
506G /var/lib/cassandra/dump/ 05/22/2018@04:49:53 58 seconds
507G /var/lib/cassandra/dump/ 05/22/2018@04:50:50 57 seconds
508G /var/lib/cassandra/dump/ 05/22/2018@04:51:46 56 seconds
509G /var/lib/cassandra/dump/ 05/22/2018@04:52:44 58 seconds
510G /var/lib/cassandra/dump/ 05/22/2018@04:53:41 57 seconds

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .