1

I have a logfile with 2 distinct events (among others) that I need to capture.

Each event generates a separate, dedicated line in the logfile with this format:

timestamp - PID - process - event-type - event-details

I don't care much about anything but the event-details column of the file, and the data I'm expecting to receive there, looks like this:

Example 1: { "values":{ "SPEED":"7.0" } }

Example 2: { "values":{ "CADENCE":"41" } }

I've been trying to write a shell script that would only read the last line of the logfile every time, and depending on the contents of the event-details column, redirect the resulting SPEED or CADENCE data to a specific text file (when I say resulting SPEED/CADENCE data I mean the "integer" after the SPEED":" expression for example).

So far I was able to redirect the results to two different files, but:

  1. I have to "tail" the logfile twice in order for the script to work and...
  2. ...as a result of that, I have the feeling that the second file is not being updated at the same rate as the first one...as if, for some reason, I was missing some of the CADENCE events due to the order in which the script was written.

I tried using the sleep function, and also tried to "tail" more than one line at a time to try to mitigate the lack of CADENCE update with no luck. I just keep missing CADENCE events from time to time.

A note on the logfile behavior: Looking at the log, there are 3 events that appear most of the time, and they are always logged in the same order of appearance (CADENCE, SPEED and OTHER), and from time to time there is a 4rd event. I just wanted to clarify that the missing CADENCE events have nothing to do with that "4rd" event appearance.

This is a summarized version of the script that I have currently running:

#!/bin/bash
while :
do
   tail -1 logfile.txt | grep -oP '(?<=SPEED":")[0-9]+' > spd.txt
   tail -1 logfile.txt | grep -oP '(?<=CADENCE":")[0-9]+' > cad.txt
done

=======UPDATE:=======

This is the complete log line and ouput expected:

Example of line 1:

Input (from logfile.txt):

03-16 21:05:28.641 2797-2842/process:Service D/WEBSOCKET: receiving: { "values":{ "Speed MPH":"3.1", "Speed KPH":"4.9", "Miles":"0.551", "Kilometers":"0.886" } }

Output (sent to spd.txt):

4.9

Example of line 2:

Input (from logfile.txt):

03-16 21:05:29.309 2797-2842/process:Service D/WEBSOCKET: receiving: { "values":{ "RPM":"27" } }

Output: (sent to cad.txt):

27

0

No need of while loop here...

tail -f logfile.txt | awk '/SPEED/{print >"spd.txt"}/CADENCE/{print >"cad.txt"}'
  • Hi Kamaraj, thanks for the help! I just tried with the proposed solution but it is writing the complete log line to the text file. I was wondering if there is a way to only print the resulting Speed/Cadence values. – danbee Mar 30 '17 at 14:26
  • please provide the input file contents and the expected output – Kamaraj Mar 31 '17 at 6:37
  • Sure! I updated the post with the information. Thanks a lot! – danbee Mar 31 '17 at 14:04
0

Got it Kamaraj, thanks for the help! it was key to find the right answer:

This is the script that what worked for me:

tail -1 logfile.txt | awk -F"\":\"" '{for (c=1;c<=NF;c++) {if ($c ~ /Speed KPH/) {print $(c+1)+0 > "spd.txt"} {if ($c ~ /RPM/) {print $(c+1)+0 > "cad.txt"}}}}'

Had to go back to tail -1 because tail -f was affected by buffering and it would also leave a trail in the txt file with all the values captured. I was only expecting a single line with the most current result in the output text files.

Thanks and best regards!

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.