14

When doing a tail -f error.log, how to insert programmatically a line break after nothing has been appened to the file for 3 seconds ?

(obviously, once one line break has been added, no other line break should be added until other line(s) of text is added to the log file)

For instance, these lines are appened to error.log :

foo
bar
boo [[wait 4 seconds]]
2far
2foo
2bar
2boo [[wait 40 seconds]]
2far

This would be the output in the console :

foo
bar
boo

2far
2foo
2bar
2boo

2far
Improve this question
  • You could probably adapt my function in askubuntu.com/a/993821/158442, or use ts to add timestamping to the output and process the timestamps – muru Feb 22 '18 at 13:30
  • 1
    Worth mentioning that if you're doing it interactively, you can just hit the enter key a bunch of times. :) – Wildcard Feb 22 '18 at 23:06
13

You could always implement the tail -f (well here, unless you uncomment the seek(), more like tail -n +1 -f as we're dumping the whole file) by hand with perl for instance:

perl -e '
  $| = 1;
  # seek STDIN, 0, 2; # uncomment if you want to skip the text that is
                      # already there. Or if using the ksh93 shell, add
                      # a <((EOF)) after < your-file
  while (1) {
    if ($_ = <STDIN>) {
      print; $t = 0
    } else {
      print "\n"            if $t == 3;
      # and a line of "-"s after 10 seconds:
      print "-" x 72 . "\n" if $t == 10;
      sleep 1;
      $t++;
    }
  }' < your-file

Or let tail -f do the tailing and use perl to insert the newlines if there's no input for 3 seconds:

tail -f file | perl -pe 'BEGIN{$SIG{ALRM} = sub {print "\n"}} alarm 3'

Those assume that the output itself is not slowed down (like when the output goes to a pipe that is not actively read).

Improve this answer
  • It took me a long time to figure out why the second one actually works :) – hobbs Feb 23 '18 at 7:35
  • I've tried the first one, and it printed ALL the files before hand, so it isn't optimal. The second one works like a charm. I have added the "tail -n 0 -f $1 |" option (-n 0) to avoid displaying the old lines of files. – Cedric Feb 27 '18 at 10:12
  • Small question : How could I amend the second solution to display an additional line of dashes (-------) after 10 seconds ? (I have tried multiple ways, but cannot make anything work) – Cedric Feb 27 '18 at 10:18
  • 1
    @Cedric, see edit for your first point. Your second requirement would be easier with the first approach. – Stéphane Chazelas Feb 27 '18 at 10:19
8

bash + date solution:

while IFS= read -r line; do        
    prev=$t         # get previous timestamp value
    t=$(date +%s)   # get current timestamp value
    [[ ! -z "$prev" ]] && [[ "$((t-prev))" -ge 3 ]] && echo ""
    echo "$line"    # print current line
done < <(tail -f error.log)
Improve this answer
  • In Bash, you can use $SECONDS to count time intervals. I think it's the number of seconds since the shell started, not that it matters when taking a difference. – ilkkachu Feb 22 '18 at 14:13
  • @ilkkachu, or read -t or $TMOUT. $SECONDS is broken in bash and mksh. time bash -c 'while ((SECONDS < 3)); do :; done' will last in between 2 and 3 seconds. Better to use zsh or ksh93 instead here (with typeset -F SECONDS) – Stéphane Chazelas Feb 22 '18 at 14:16
  • @StéphaneChazelas, I don't think it's any different than using date +%s. Both give the time in full seconds, which does have the effect that the interval from, say, 1.9 to 4.0 looks like 3 full seconds, even though it's really 2.1. It's hard to work around that if all you can't access the fractional seconds. But yes, they should probably sleep here instead of busylooping, and then read -t could just as well be used. Even if you do sleep manually, time bash -c 'while [[ $SECONDS -lt 3 ]]; do sleep 1; done' works just fine. – ilkkachu Feb 22 '18 at 14:31
  • 1
    ksh93 and zsh are OK with that (zsh used not to). Even with integer $SECONDS, setting SECONDS=0 ensures that $SECONDS will reach 1 in exactly 1 second. That's not the case with bash as it uses time() to track $SECONDS instead of gettimeofday(). I reported bugs to mksh, zsh and bash some time ago, only zsh was fixed. (good point about the issue being the same with date +%s). Note that it's not a busyloop here, as we're reading from the output of tail -f over a pipe. – Stéphane Chazelas Feb 22 '18 at 14:48
  • +1 and Bash has a "shortcut" using the built-in printf to emulate date without external tools or command substitution: printf -v t '%(%s)T' -1. – David Foerster Feb 22 '18 at 20:43
6

Python solution (with dynamic time gap argument):

tailing_by_time.py script:

import time, sys

t_gap = int(sys.argv[1])    # time gap argument
ts = 0
while True:
    line = sys.stdin.readline().strip()    # get/read current line from stdin
    curr_ts = time.time()                  # get current timestamp
    if ts and curr_ts - ts >= t_gap:
        print("")                          # print empty line/newline
    ts = curr_ts
    if line:
        print(line)                        # print current line if it's not empty

Usage:

tail -f error.log | python tailing_by_time.py 3
Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.