Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

It seems I am misusing grep/egrep.

I was trying to search for strings in multiple line and could not find a match while I know that what I'm looking for should match. Originally I thought that my regexes were wrong but I eventually read that these tools operate per line (also my regexes were so trivial it could not be the issue).

So which tool would one use to search patterns across multiple lines?

share|improve this question
    
possible duplicate of Multiline pattern match using sed, awk or grep –  Ciro Santilli Sep 15 at 21:21
    
@CiroSantilli - I do not think that this Q and the one you linked to are duplicates. The other Q is asking how you'd do multi-line pattern match (i.e. what tool should/can I use to do this) while this one is asking how to do this with grep. They are tightly related but not dups, IMO. –  slm Sep 16 at 1:38
    
@sim those cases are hard to decide: I can see your point. I think this particular case is better as a duplicate because the user said "grep" suggesting the verb "to grep", and top answers, including accepted, don't use grep. –  Ciro Santilli Sep 16 at 6:32

4 Answers 4

up vote 5 down vote accepted

Here's a sed one that will give you grep-like behavior across multiple lines:

sed -n '/foo/{:start /bar/!{N;b start};/your_regex/p}' your_file

How it works

  • -n suppresses the default behavior of printing every line
  • /foo/{} instructs it to match foo and do what comes inside the squigglies to the matching lines. Replace foo with the starting part of the pattern.
  • :start is a branching label to help us keep looping until we find the end to our regex.
  • /bar/!{} will execute what's in the squigglies to the lines that don't match bar. Replace bar with the ending part of the pattern.
  • N appends the next line to the active buffer (sed calls this the pattern space)
  • b start will unconditionally branch to the start label we created earlier so as to keep appending the next line as long as the pattern space doesn't contain bar.
  • /your_regex/p prints the pattern space if it matches your_regex. You should replace your_regex by the whole expression you want to match across multiple lines.
share|improve this answer

Here's a simpler approach using Perl:

perl -e '$f=join("",<>); print $& if $f=~/foo\nbar.*\n/m' file

or (since JosephR took the sed route, I'll shamelessly steal his suggestion)

perl -n000e 'print $& while /^foo.*\nbar.*\n/mg' file

Explanation

$f=join("",<>); : this reads the entire file and saves it's contents (newlines and all) into the variable $f. We then attempt to match foo\nbar.*\n, and print it if it matches (the special variable $& holds the last match found). The ///m is needed to make the regular expression match across newlines.

The -0 sets the input record separator. Setting this to 00 activates 'paragraph mode' where Perl will use consecutive newlines (\n\n) as the record separator. In cases where there are no consecutive newlines, the entire file is read (slurped) at once.

Warning:

Do not do this for large files, it will load the entire file into memory and that may be a problem.

share|improve this answer

One way to do this is with Perl. e.g. here's the contents of a file named foo:

foo line 1
bar line 2
foo
foo
foo line 5
foo
bar line 6

Now, here's some Perl which will match against any line that begins with foo followed by any line that begins with bar:

cat foo | perl -e 'while(<>){$all .= $_}
  while($all =~ /^(foo[^\n]*\nbar[^\n]*\n)/m) {
  print $1; $all =~ s/^(foo[^\n]*\nbar[^\n]*\n)//m;
}'

The Perl, broken down:

  • while(<>){$all .= $_} This loads the entire standard input in to the variable $all
  • while($all =~ While the variable all has the regular expression...
  • /^(foo[^\n]*\nbar[^\n]*\n)/m The regex: foo at the beginning of the line, followed by any number of non-newline chars, followed by a newline, followed immediately by "bar", and the rest of the line with bar in it. /m at the end of the regex means "match across multiple lines"
  • print $1 Print the part of the regex that was in parenthesis (in this case, the entire regular expression)
  • s/^(foo[^\n]*\nbar[^\n]*\n)//m Erase the first match for the regex, so we can match multiple cases of the regex in the file in question

And the output:

foo line 1
bar line 2
foo
bar line 6
share|improve this answer
3  
Just dropped by to say your Perl can be shortened to the more idiomatic: perl -n0777E 'say $& while /^foo.*\nbar.*\n/mg' foo –  Joseph R. Feb 2 at 13:10

I generally use a tool called pcregrep which can be installed in most of the linux flavour using yum or apt.

For eg.

Suppose if you have a file named testfile with content

abc blah
blah blah
def blah
blah blah

You can run the following command:

$ pcregrep -M  'abc.*(\n|.)*def' testfile

to do pattern matching across multiple lines.

Moreover, you can do the same with sed as well.

$ sed -e '/abc/,/def/!d' testfile
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.