0

I wish to get the last line of a log file that matches a test string and output to another log file.

I need to test for a number of different text strings and depending on the output send to a different log file.

I am using this:

tail -f -n 1 input.log | grep string1 >>output1.log

Then I want to repeat the test like this:

tail -f -n 1 input.log | grep string2 >>output2.log
tail -f -n 1 input.log | grep string3 >>output3.log

How to I achieve this in a single efficient bash script?

  • 1
    This does what you want, right? Then why don't you put them into a bash script? It's just three lines. And I wouldn't know of any way to do it more efficiently. – pfnuesel Feb 4 '18 at 15:28
  • tail -f -n 1 starts following the log file from the (current) last line. also, awk would be better suited to this than grep. – cas Feb 4 '18 at 15:28
5

using awk rather than grep:

awk '/string1/ { print >> "output1.log" ; fflush() }
     /string2/ { print >> "output2.log" ; fflush() }
     /string3/ { print >> "output3.log" ; fflush() }' input.log

This outputs all matching lines to their respective output.log files. That's because I can't make sense of your conflicting tail -f and tail -n 1 requirements. If you really do want it to start following input.log from the current last line, then pipe tail -f -n 1 into the awk script and get rid of the input.log at the end of the line. e.g.

tail -f -n 1 input.log | awk '...same awk script as above...'

You could also do it with tee and grep and process substitution (but it will be noticeably slower):

tee >(grep --line-buffered string1 >> output1.log) \
    >(grep --line-buffered string2 >> output2.log) \
    >(grep --line-buffered string3 >> output3.log) < input.log

or

tail -f -n 1 input.log | tee .....

NOTE: The fflush() in the awk solution, and the --line-buffered option in the tee solution are only needed if you're piping the output of tail -f (or some other never-ending process) into awk or tee.

Without them, the output files will only be written when the output buffers (of awk or grep) are full - and if the job is aborted while there is unwritten output in the buffers (e.g. by pressing Ctrl-C) then any output still in the buffers will be lost.

With them, both solutions run much slower (because they flush output on every write) - but that's unlikely to be significant except with very large input files.

BTW, this isn't an issue when the input ends - in that case, both awk and grep flush their output buffers automatically before exiting.

Another alternative would be to run the pipe to awk (or tee) in a sub-shell that trapped and ignored the Interrupt signal from Ctrl-C. e.g.

tail -f input.log | ( 
  trap '' INT
  awk '/string1/ { print >> "output1.log" }
       /string2/ { print >> "output2.log" }
       /string3/ { print >> "output3.log" }'
  )

The tail -f is affected (killed) by pressing Ctrl-C, but the sub-shell running awk ignores it. awk flushes its output buffers and exits when the tail is killed because its input has finished.

See Trap Ctrl-C in awk script for an another example/explanation.


If you don't want it to follow the log file, then don't use tail's -f option.

Just use tail -n 1 | ..... or see Guy's answer which prints the last match for all three strings into their respective output files.

  • I have tried the tail -f option and piped into awk as suggested above, however the first time the script is run it creates a null file with no content. The input file has the matching string that I am using in the awk command, however this is not sent to the output file. If I use tail -n 1 then it works. I want to constantly monitor the input file, however tail -f does not seem to work. I would welcome any ideas on how to resolve this. Many thanks – Colin Davis Feb 10 '18 at 14:10
  • that's caused by awk's output buffering. it doesn't actually write anything to the output file(s) until there's enough output to fill the output buffer. If you kill the awk process before then, then nothing will get written. I'll update the answer to prevent that. – cas Feb 10 '18 at 22:51
1

Editing the solution from cas, to focus on your line.

I wish to get the last line of a log file that matches a test string and output to another log file.

awk '
    # every time theres a match, save that line into a variable
    /string1/ { str1 = $0 }
    /string2/ { str2 = $0 }
    /string3/ { str3 = $0 }
    END {
      # and only print to file after reading the whole log.
      print str1 >> "output1.log"
      print str2 >> "output2.log"
      print str3 >> "output3.log" 
}' input.log
0

input file

praveen
ajay
abhi
praveen
ajay
abhi
kiran
tomato

Our requirement is to search the different text string the write the output to different text file

I have used for loop and sed command for the same

I Tried to search for content praveen, ajay, abhi and saves the output in contentsearch.txt

Script

for i in praveen ajay abhi; do sed -n "/$i/p" input.txt > $i.txt; done

Below will be the Output

praveen.txt  will contains line where praveen is present
ajay.txt will contains line where ajay is present
abhi.txt will contains where abhi is present

==============================================================================

If You want to search for last line of log file which matches test string and output to other file Then use below command

sed -n '/Contentsearch/p' input.txt | tail -1 > output.txt

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.