6

I'm watching a log and want to detect once a program has had 3 failed attempts:

tail -f file.log | grep --line-buffered program\ failed\ string

If the line count from grep hits 3 I want to return an error.

How can I do this?

  • Use a tool that can count (awk, perl). – don_crissti Nov 23 '17 at 18:07
  • 1
    Can't you just add -m3 and then return the error when grep exits? – steeldriver Nov 23 '17 at 18:31
  • @steeldriver I'm trying something like tail -f file.log | grep --line-buffered -m 3 string && echo done now but not working. – Philip Kirkbride Nov 23 '17 at 18:36
  • @steeldriver - using grep with -m3 doesn't change the fact that grep will exit with status 0 if there's at least one match. The only thing that grep -m3 does is that it exits after the 3rd match instead of processing the whole file like simple grep or exiting after the 1st match like grep -q. One cannot know the no. of matches based on the exit status (unless the exit code is 1 which means there was no match) – don_crissti Nov 23 '17 at 18:49
5

awk is a great tool for scanning streams.

I assumed it should display all lines to see the log until it exit, contrary to your example with grep which display only error lines.

tail -f file.log | awk '
    BEGIN {
        count = 0
    }

    {
        print($0)

        if ($0 ~ /program failed/) {
            count++
            if (count == 3) {
                exit(1)
            }
        }
    }
'

You can move the awk code to tail.awk and call tail -f file.log | awk -f tail.awk if you prefer.

Equivalently, in a more compact form:

tail -f file.log | awk '1; /program failed/ && ++count == 3 { exit 1 }'
  • thanks, stops printing like grep -m 3 but the snippet hangs forever instead of returning stdout or stderr. – Philip Kirkbride Nov 23 '17 at 18:44
  • @PhilipKirkbride - works fine here, make sure you have the closing quote... – don_crissti Nov 23 '17 at 18:52
2

Just in case someone might prefer a Python alternative:

#!/usr/bin/env python2
# -*- encoding: ascii -*-
"""tail.py"""

import sys
import argparse
import time
import re

# Create a command-line parser
parser = argparse.ArgumentParser()

parser.add_argument(
    "-f", "--input-file",
    help="File to search through.",
    dest="input_file", type=str,
)
parser.add_argument(
    "-p", "--pattern",
    help="Regular expression to match.",
    default=r'.*',
    dest="pattern", type=str,
)
parser.add_argument(
    "-m", "--match-limit",
    help="Number of matches before exiting.",
    default=float(1),
    dest="match_limit", type=int,
)
parser.add_argument(
    "-q", "--quiet",
    help="Don't print matched lines.",
    default=False,
    dest="quiet", type=bool,
)

# Define the file-watching function
def watch_for_matches(file_handle, pattern, match_limit, quiet):

    # Count the number of matched lines
    matches_found = 0

    # Get the next line
    line = file_handle.readline()

    # Check to see if the limit has been reached
    while(matches_found < match_limit):

        # Match the line against the given regular expression
        if(line and re.search(pattern, line)):

            # Optionally print the matched line
            if not quiet:
                sys.stdout.write(line)

            # Increment the match counter
            matches_found += 1

        # Optionally wait for a moment 
        time.sleep(0.25)

        # Get the next line of input
        line = file_handle.readline()

    # If the match limit is reached, exit with an error
    sys.exit(1)

# Parse the command-line arguments
args = parser.parse_args()

# Execute the function
if args.input_file:
    with open(args.input_file, 'r') as file_handle:
        watch_for_matches(file_handle, args.pattern, args.match_limit, args.quiet)

# If no file is given, use standard input instead
else:
    watch_for_matches(sys.stdin, args.pattern, args.match_limit, args.quiet)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.