Multiple Grep Searches

Question

I wish to get the last line of a log file that matches a test string and output to another log file.

I need to test for a number of different text strings and depending on the output send to a different log file.

I am using this:

tail -f -n 1 input.log | grep string1 >>output1.log

Then I want to repeat the test like this:

tail -f -n 1 input.log | grep string2 >>output2.log
tail -f -n 1 input.log | grep string3 >>output3.log

How to I achieve this in a single efficient bash script?

Answer 1

using awk rather than grep:

awk '/string1/ { print >> "output1.log" ; fflush() }
     /string2/ { print >> "output2.log" ; fflush() }
     /string3/ { print >> "output3.log" ; fflush() }' input.log

This outputs all matching lines to their respective output.log files. That's because I can't make sense of your conflicting tail -f and tail -n 1 requirements. If you really do want it to start following input.log from the current last line, then pipe tail -f -n 1 into the awk script and get rid of the input.log at the end of the line. e.g.

tail -f -n 1 input.log | awk '...same awk script as above...'

You could also do it with tee and grep and process substitution (but it will be noticeably slower):

tee >(grep --line-buffered string1 >> output1.log) \
    >(grep --line-buffered string2 >> output2.log) \
    >(grep --line-buffered string3 >> output3.log) < input.log

or

tail -f -n 1 input.log | tee .....

---

NOTE: The fflush() in the awk solution, and the --line-buffered option in the tee solution are only needed if you're piping the output of tail -f (or some other never-ending process) into awk or tee.

Without them, the output files will only be written when the output buffers (of awk or grep) are full - and if the job is aborted while there is unwritten output in the buffers (e.g. by pressing Ctrl-C) then any output still in the buffers will be lost.

With them, both solutions run much slower (because they flush output on every write) - but that's unlikely to be significant except with very large input files.

BTW, this isn't an issue when the input ends - in that case, both awk and grep flush their output buffers automatically before exiting.

Another alternative would be to run the pipe to awk (or tee) in a sub-shell that trapped and ignored the Interrupt signal from Ctrl-C. e.g.

tail -f input.log | ( 
  trap '' INT
  awk '/string1/ { print >> "output1.log" }
       /string2/ { print >> "output2.log" }
       /string3/ { print >> "output3.log" }'
  )

The tail -f is affected (killed) by pressing Ctrl-C, but the sub-shell running awk ignores it. awk flushes its output buffers and exits when the tail is killed because its input has finished.

See Trap Ctrl-C in awk script for an another example/explanation.

---

If you don't want it to follow the log file, then don't use tail's -f option.

Just use tail -n 1 | ..... or see Guy's answer which prints the last match for all three strings into their respective output files.

Answer 2

Editing the solution from cas, to focus on your line.

I wish to get the last line of a log file that matches a test string and output to another log file.

awk '
    # every time theres a match, save that line into a variable
    /string1/ { str1 = $0 }
    /string2/ { str2 = $0 }
    /string3/ { str3 = $0 }
    END {
      # and only print to file after reading the whole log.
      print str1 >> "output1.log"
      print str2 >> "output2.log"
      print str3 >> "output3.log" 
}' input.log

Answer 3

input file

praveen
ajay
abhi
praveen
ajay
abhi
kiran
tomato

Our requirement is to search the different text string the write the output to different text file

I have used for loop and sed command for the same

I Tried to search for content praveen, ajay, abhi and saves the output in contentsearch.txt

Script

for i in praveen ajay abhi; do sed -n "/$i/p" input.txt > $i.txt; done

Below will be the Output

praveen.txt  will contains line where praveen is present
ajay.txt will contains line where ajay is present
abhi.txt will contains where abhi is present

==============================================================================

If You want to search for last line of log file which matches test string and output to other file Then use below command

sed -n '/Contentsearch/p' input.txt | tail -1 > output.txt