8

I'm watching a log and want to detect once a program has had 3 failed attempts:

tail -f file.log | grep --line-buffered program\ failed\ string

If the line count from grep hits 3 I want to return an error.

How can I do this?

4
  • Use a tool that can count (awk, perl). Nov 23, 2017 at 18:07
  • 1
    Can't you just add -m3 and then return the error when grep exits? Nov 23, 2017 at 18:31
  • @steeldriver I'm trying something like tail -f file.log | grep --line-buffered -m 3 string && echo done now but not working. Nov 23, 2017 at 18:36
  • @steeldriver - using grep with -m3 doesn't change the fact that grep will exit with status 0 if there's at least one match. The only thing that grep -m3 does is that it exits after the 3rd match instead of processing the whole file like simple grep or exiting after the 1st match like grep -q. One cannot know the no. of matches based on the exit status (unless the exit code is 1 which means there was no match) Nov 23, 2017 at 18:49

2 Answers 2

6

awk is a great tool for scanning streams.

I assumed it should display all lines to see the log until it exit, contrary to your example with grep which display only error lines.

tail -f file.log | awk '
    BEGIN {
        count = 0
    }

    {
        print($0)

        if ($0 ~ /program failed/) {
            count++
            if (count == 3) {
                exit(1)
            }
        }
    }
'

You can move the awk code to tail.awk and call tail -f file.log | awk -f tail.awk if you prefer.

Equivalently, in a more compact form:

tail -f file.log | awk '1; /program failed/ && ++count == 3 { exit 1 }'
2
  • thanks, stops printing like grep -m 3 but the snippet hangs forever instead of returning stdout or stderr. Nov 23, 2017 at 18:44
  • @PhilipKirkbride - works fine here, make sure you have the closing quote... Nov 23, 2017 at 18:52
2

Just in case someone might prefer a Python alternative:

#!/usr/bin/env python2
# -*- encoding: ascii -*-
"""tail.py"""

import sys
import argparse
import time
import re

# Create a command-line parser
parser = argparse.ArgumentParser()

parser.add_argument(
    "-f", "--input-file",
    help="File to search through.",
    dest="input_file", type=str,
)
parser.add_argument(
    "-p", "--pattern",
    help="Regular expression to match.",
    default=r'.*',
    dest="pattern", type=str,
)
parser.add_argument(
    "-m", "--match-limit",
    help="Number of matches before exiting.",
    default=float(1),
    dest="match_limit", type=int,
)
parser.add_argument(
    "-q", "--quiet",
    help="Don't print matched lines.",
    default=False,
    dest="quiet", type=bool,
)

# Define the file-watching function
def watch_for_matches(file_handle, pattern, match_limit, quiet):

    # Count the number of matched lines
    matches_found = 0

    # Get the next line
    line = file_handle.readline()

    # Check to see if the limit has been reached
    while(matches_found < match_limit):

        # Match the line against the given regular expression
        if(line and re.search(pattern, line)):

            # Optionally print the matched line
            if not quiet:
                sys.stdout.write(line)

            # Increment the match counter
            matches_found += 1

        # Optionally wait for a moment 
        time.sleep(0.25)

        # Get the next line of input
        line = file_handle.readline()

    # If the match limit is reached, exit with an error
    sys.exit(1)

# Parse the command-line arguments
args = parser.parse_args()

# Execute the function
if args.input_file:
    with open(args.input_file, 'r') as file_handle:
        watch_for_matches(file_handle, args.pattern, args.match_limit, args.quiet)

# If no file is given, use standard input instead
else:
    watch_for_matches(sys.stdin, args.pattern, args.match_limit, args.quiet)

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .