When doing a tail -f error.log, how to insert programmatically a line break after nothing has been appened to the file for 3 seconds ?
(obviously, once one line break has been added, no other line break should be added until other line(s) of text is added to the log file)
For instance, these lines are appened to error.log :
foo
bar
boo [[wait 4 seconds]]
2far
2foo
2bar
2boo [[wait 40 seconds]]
2far
This would be the output in the console :
foo
bar
boo
2far
2foo
2bar
2boo
2far
You could always implement the tail -f (well here, unless you uncomment the seek(), more like tail -n +1 -f as we're dumping the whole file) by hand with perl for instance:
perl -e '
$| = 1;
# seek STDIN, 0, 2; # uncomment if you want to skip the text that is
# already there. Or if using the ksh93 shell, add
# a <((EOF)) after < your-file
while (1) {
if ($_ = <STDIN>) {
print; $t = 0
} else {
print "\n" if $t == 3;
# and a line of "-"s after 10 seconds:
print "-" x 72 . "\n" if $t == 10;
sleep 1;
$t++;
}
}' < your-file
Or let tail -f do the tailing and use perl to insert the newlines if there's no input for 3 seconds:
tail -f file | perl -pe 'BEGIN{$SIG{ALRM} = sub {print "\n"}} alarm 3'
Those assume that the output itself is not slowed down (like when the output goes to a pipe that is not actively read).
bash + date solution:
while IFS= read -r line; do
prev=$t # get previous timestamp value
t=$(date +%s) # get current timestamp value
[[ ! -z "$prev" ]] && [[ "$((t-prev))" -ge 3 ]] && echo ""
echo "$line" # print current line
done < <(tail -f error.log)
$SECONDS to count time intervals. I think it's the number of seconds since the shell started, not that it matters when taking a difference.
– ilkkachu
Feb 22 '18 at 14:13
read -t or $TMOUT. $SECONDS is broken in bash and mksh. time bash -c 'while ((SECONDS < 3)); do :; done' will last in between 2 and 3 seconds. Better to use zsh or ksh93 instead here (with typeset -F SECONDS)
– Stéphane Chazelas
Feb 22 '18 at 14:16
date +%s. Both give the time in full seconds, which does have the effect that the interval from, say, 1.9 to 4.0 looks like 3 full seconds, even though it's really 2.1. It's hard to work around that if all you can't access the fractional seconds. But yes, they should probably sleep here instead of busylooping, and then read -t could just as well be used. Even if you do sleep manually, time bash -c 'while [[ $SECONDS -lt 3 ]]; do sleep 1; done' works just fine.
– ilkkachu
Feb 22 '18 at 14:31
SECONDS=0 ensures that $SECONDS will reach 1 in exactly 1 second. That's not the case with bash as it uses time() to track $SECONDS instead of gettimeofday(). I reported bugs to mksh, zsh and bash some time ago, only zsh was fixed. (good point about the issue being the same with date +%s). Note that it's not a busyloop here, as we're reading from the output of tail -f over a pipe.
– Stéphane Chazelas
Feb 22 '18 at 14:48
printf to emulate date without external tools or command substitution: printf -v t '%(%s)T' -1.
– David Foerster
Feb 22 '18 at 20:43
Python solution (with dynamic time gap argument):
tailing_by_time.py script:
import time, sys
t_gap = int(sys.argv[1]) # time gap argument
ts = 0
while True:
line = sys.stdin.readline().strip() # get/read current line from stdin
curr_ts = time.time() # get current timestamp
if ts and curr_ts - ts >= t_gap:
print("") # print empty line/newline
ts = curr_ts
if line:
print(line) # print current line if it's not empty
Usage:
tail -f error.log | python tailing_by_time.py 3
tsto add timestamping to the output and process the timestamps – muru Feb 22 '18 at 13:30