I have a output log file which will be continuously appended and I am planning to have a script (runs for every 5 min) to retrieve the last 5 min data from of the output log.

Sample data from the output log file:

2015-10-29 09:19:39,630 INFO line1 of log  
2015-10-29 09:21:39,630 INFO line2 of log
2015-10-29 09:22:39,630 INFO line3 of log
2015-10-29 09:23:39,630 INFO line4 of log
2015-10-29 09:24:39,630 INFO line5 of log
2015-10-29 09:25:39,630 INFO line6 of log

If i am running my script at 09:25 it should capture last 5 lines and send to a separate output file.


This is all you need:

five_min=$( date -d "5 minutes ago" +"%F %T" )
while IFS= read -r line; do
    [[ "$five_min" < "$line" ]] && echo "$line"
done < output.log

So, it's a huge file. Then use tac to output the file from the bottom first:

five_min=$( date -d "5 minutes ago" +"%F %T" )
tac output.log | while IFS= read -r line; do
    [[ "$five_min" < "$line" ]] && echo "$line"
done | tac

The while command, just like other languages, executes it's body while the condition is true. In POSIX shells, the "condition" of the if and while commands is itself a command. The "truth" of the "condition" is that the command exits with a zero exit status. It is very important to realize that the words [ and [[ are actually commands, not just syntax (from a bash shell type help [[ and help [)

In this answer, the condition is IFS= read -r line which is the canonical way to read a line from a file, and keep all whitespace and backslash sequences intact. read will return with a non-zero exit status if there's nothing more to read from it's input stream. In my first response, I redirected the contents of the log file to the standard input of the while command, and read will read that data.

|improve this answer|||||
  • Thanks for your response..! The output log file is so huge so i just want to read the file from bottom and also is there any way we can achieve it through AWK command and also i am new to these scripting so can you help me in understanding the While command – rv krishna Dec 6 '15 at 20:22
#To get previous 5 minutes logs from now(IST)
d1=$(date –date="-5 min" +'%Y-%m-%d %H:%M:%S.%3N')
d2=$(date +'%Y-%m-%d %H:%M:%S.%3N') #Added missing closing parenthesis

awk -v d1="$d1" -v d2="$d2" '$0 > d1 && $0 < d2 || $0 ~ d2' /var/log/<file_name >  > /var/log/$<new_file_name>

More at : http://www.writeulearn.com/linux-concepts

|improve this answer|||||

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.