7

Consider the following toy example:

this is a line 
this line contains FOO 
this line is not blank

This line also contains FOO

Some random text

This line contains FOO too
Not blank 
Also not blank

More random text 
FOO!
Yet more random text
FOO!

So, I want the results of a grep for FOO, but with the extra wrinkle that lines following the matching lines should be included, as long as they are not blank, and they do not themselves contain FOO. So the matches would look as follows, with the different matches separated:

MATCH 1

this line contains FOO 
this line is not blank

MATCH 2

This line also contains FOO

MATCH 3

This line contains FOO too 
Not blank 
Also not blank

MATCH 4

FOO!
Yet more random text

MATCH 5

FOO!

Bonus points (metaphorically speaking) for a simple single line script that can be run on the command line.

ADDENDUM: Adding a running count of the match number would be quite handy, if it is not too hard.

|improve this question
  • 2
    do you want all the Matches in one output or you want them to be ran separately ? – Vivek Kanadiya Mar 20 '19 at 19:16
  • I guess one output is fine, as long as they are clearly separated, say by a blank line. – Faheem Mitha Mar 20 '19 at 19:18
  • might not possible in one output but I can try to see in each separate output! – Vivek Kanadiya Mar 20 '19 at 19:22
  • For matches 4 & 5, the simple logic could say "I've matched (the 4th) "FOO", so I should continue printing in this matching block until I see a blank line, and so the final "FOO" is included in this block", but i see you've separated it -- because it's a separate "FOO" match. Do you require the 5th "FOO" to be printed separately? – Jeff Schaller♦ Mar 20 '19 at 19:28
  • @JeffSchaller Yes, please separate different FOO matches. – Faheem Mitha Mar 20 '19 at 19:29
9

Using awk rather than grep:

awk '/FOO/ { if (matching) printf("\n"); matching = 1 }
     /^$/  { if (matching) printf("\n"); matching = 0 }
     matching' file

A version that enumerates the matches:

awk 'function flush_print_maybe() {
         if (matching) printf("Match %d\n%s\n\n", ++n, buf)
         buf = ""
     }
     /FOO/ { flush_print_maybe(); matching = 1 }
     /^$/  { flush_print_maybe(); matching = 0 }
     matching { buf = (buf == "" ? $0 : buf ORS $0) }
     END   { flush_print_maybe() }' file

Both awk programs uses a very simple "state machine" to determine if it's currently matching or not matching. A match of the pattern FOO will cause it to enter the matching state, and a match of the pattern ^$ (an empty line) will cause it to enter the non-matching state.

Output of empty lines between matching sets of data happens at state transitions from matching (either into matching or into non-matching).

The first program prints any line when in the matching state.

The second program collects lines in a buf variable when in a matching state. It flushes (empties) this after possibly printing it (depending on the state), together with a Match N label at state transitions (when the first program would output an empty line).

Output of this last program on the sample data:

Match 1
this line contains FOO
this line is not blank

Match 2
This line also contains FOO

Match 3
This line contains FOO too
Not blank
Also not blank

Match 4
FOO!
Yet more random text

Match 5
FOO!
|improve this answer
  • Nitpicking, these output an extra blank line if there’s a non-matching section at the end of the input ;-). – Stephen Kitt Mar 20 '19 at 22:04
  • @StephenKitt Yes. It will. Each result block will be terminated by an empty line. – Kusalananda♦ Mar 20 '19 at 22:07
  • Only if there’s an empty line following the result block (so the example text in the question doesn’t cause an empty line to be printed at the end). – Stephen Kitt Mar 20 '19 at 22:09
  • @StephenKitt It's almost midnight here, I'll get back to it later. :-) – Kusalananda♦ Mar 20 '19 at 22:10
6
sed -ne '/FOO/{x;P;x};/FOO/,/^$/p' testfile

Each block of non-empty lines in the output is a single chunk of matched data from the input. The number of newlines varies.

This

  1. suppresses output (-n); then
  2. prints a blank line before every occurrence of "FOO" (/FOO/{x;P;x} - uses the empty hold space);
  3. selects ranges of lines starting from ones containing FOO (/FOO/) and ending on empty lines (/^$/); and finally
  4. prints those lines (p).
this line contains FOO
this line is not blank


This line also contains FOO


This line contains FOO too
Not blank
Also not blank


FOO!
Yet more random text

FOO!
|improve this answer
5

I don’t think this is doable with grep, but it is with AWK:

#! /usr/bin/awk -f

/FOO/ {
  matched = 1
  if (notfirst) print ""
  notfirst = 1
}

/^$/ {
  matched = 0
}

matched

With a count of matches:

#! /usr/bin/awk -f

/FOO/ {
  matched = 1
  if (matches) print ""
  printf "Match %d\n", ++matches
}

/^$/ {
  matched = 0
}

matched

In both cases, the first two blocks determine whether the current record should be copied to the output. When the current record matches “FOO”, the first block sets matched to 1, outputs a blank record if necessary (to separate the forthcoming output from the previous match); in the second variant, it also increments the matches counter and outputs a header. When the current record is empty, the second block sets matched to 0. The lonely matched condition prints the current record if matched is 1.

|improve this answer
2

I've included a pcregrep solution and a python solution.

Multiline Grep Solution

If you have pcregrep installed, you could use a multiline pattern, such as ^.*FOO.*$\n?(^.*\S.*$\n?)*, e.g.:

pcregrep -M '^.*FOO.*$\n?(^.*\S.*$\n?)*' test.txt

The subexpression ^.*FOO.*$\n? will match any line containing the string FOO and the subexpression (^.*\S.*$\n?)* will match any number of subsequent lines that containing a non-whitespace character.

Python Solution

Here is a Python script that should do what you want:

#!/usr/bin/env python3
# -*- encoding: utf8 -*-
"""grep_follow.py

Search a text file for a pattern,
and output that pattern and the
non-empty lines which immediately follow it.
"""

import re
import sys

# Get the search pattern and the input file as command-line arguments
pattern = sys.argv[1]
input_file = sys.argv[2]

# Set a flag to determine whether or not to output the current line
print_flag = False

with open(input_file, "r") as _input_file:

    # Iterate over the lines of the input file
    for line in _input_file:

        # Remove trailing whitespace
        line = line.rstrip()

        # If the line is empty, stop producing output
        if not line.strip():
            print_flag = False

        # If the line matches the search pattern, start producing output
        elif re.search(pattern, line):
            print_flag = True

        # If the print flag is set then output the line
        if print_flag:
            print(line)

You would run it like this:

$ python grep_follow.py FOO test.txt

this line contains FOO
this line is not blank
This line also contains FOO
This line contains FOO too
Not blank
Also not blank
FOO!
Yet more random text
FOO!
|improve this answer
  • 1
    a more intuitive regex in the eyes of this beholder is '(?s)^\N*FOO.*?(?=\Z|\n\n|\N*FOO\N*)', but +1 – iruvar Mar 21 '19 at 3:23
1
awk '/FOO/{print "===match " ++i "==="} /FOO/,/^$/' file

===match 1===
this line contains FOO
this line is not blank

===match 2===
This line also contains FOO

===match 3===
This line contains FOO too
Not blank
Also not blank

===match 4===
FOO!
Yet more random text
===match 5===
FOO!

A similar variant where FOO could be easily changed to something else:

awk -vpat=FOO '$0~pat{print "===match " ++i "==="} $0~pat,/^$/' file

Omitting the terminating empty line from the default print is left as an exercise to the reader ;-)

|improve this answer
1

One way can be using Perl's range operator ...:

$ perl -lne '
     print s|^(?=.*FOO)|$/."MATCH#".++$c.$/|re if /FOO/ ... /^$/;
 ' input.txt

MATCH#1
this line contains FOO 
this line is not blank


MATCH#2
This line also contains FOO


MATCH#3
This line contains FOO too
Not blank 
Also not blank


MATCH#4
FOO!
Yet more random text

MATCH#5
FOO!

In case the extra blank lines are an issue, you can use the below code, where make use of the range operator in scalar context in order to have a peek into the state machine values and perform actions based on them.

Note :

$/ = RS = "\n"

$, = OFS = "" 

$ perl -lne '
    print
      s{^(?=.*FOO)}
       {
         ($,,$/)[$r>1] . 
         "MATCH#" . ++$count . $/
       }rex 
          if $r = /FOO/ ... /^$/;
' input.txt

Here's another method to tackle the pbm, this time using the paragraph mode of Perl. We read in records a para at a time and split them on newlines into fields.

Then we make sure to handle only those paras that happen to have the string foo in them. Finally, we print the fields with the constraint on foo-containing fields being treated specially. Even here we deal with the first field differently than the rest. 

$ perl -aln -F'\n' -00 -e '
     next unless /FOO/;
     shift @F until $F[0] =~ /FOO/;
     push @F, $, if ! eof;
     print "MATCH#", ++$k, $\, shift @F;
     print for map { /FOO/ and $_ = "\nMATCH#" . ++$k . "\n$_";$_ } @F;
' input.txt
|improve this answer
0

Here is a one-liner that may not exactly match the criteria, but comes close enough for most purposes.

gawk '/FOO/{p=1} {if(p) print} /^$/{p=0}' infile.txt

First rule sets p=1 if it sees FOO. Second rule prints the current line if p is not zero. Third rule sets p=0 if the line (already printed) is blank. Swap second and third rules if you do not want the blank line terminator to be output.

This is a variant which numbers and separates the matches if there are multiple FOO instances not separated by blank lines

gawk '/FOO/{p=1; n++; print "\n" n} /^$/{p=0} //{if (p) print $0 }' infile.txt
|improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.