4

Is there a way to grep a log and find text between the log entry delimeters? Our log file separates the line entry with characters "-------" So when I search the text word I want all the lines before and after the delimeters in the log.

Sample log

------------------------------------------------------------------------

    r132279 | USERID | 2014-04-30 12:59:09 -0700 (Wed, 30 Apr 2014) | 3 lines
    Removed unused "Calculated Fields" column entry.
    Jira ID: JIRA-977

------------------------------------------------------------------------

In the above i would Grep the word Fields but want all the lines between the "----" lines

6

If you know how big the record is, then you can output additional lines of context before (-B) and after (-A) the matching line e.g.

grep -A2 -B2 'Fields' sample.log

or for context both before and after the match line

grep -C3 'Fields' sample.log

As far as I know, the only way to do a true multiline match (rather than a single line match plus context) in GNU grep is by using the PCRE regex mode (-P) with the -z flag to prevent breaking on newlines. For example, you could try

grep -zPo '(\n-+\n)\K(.|\n)+?Fields(.|\n)+?(?=\n-+\n)'

which does a non-greedy match of the string Fields surrounded by any characters OR newlines, provided it is bookended by the newline-hyphens-newline delimiters. An equivalent expression in pcregrep is

pcregrep -Mo '(\n-+\n)\K(.|\n)+?Fields(.|\n)+?(?=\n-+\n)'


Another option for this kind of record-structured data is awk: in particular, GNU awk allows a regular expression to be used for the internal record separator RS e.g.

$ gawk -vRS='\n-+\n' '/Fields/ {print}' sample.log

r132279 | USERID | 2014-04-30 12:59:09 -0700 (Wed, 30 Apr 2014) | 3 lines

Removed unused "Calculated Fields" column entry.

Jira ID: JIRA-977
  • gawk worked well . But when there is no DESCRIPTION field it outputs all lines. Thanks – user3609755 May 12 '14 at 19:27
3

A Perl solution similar to the gawk one in steeldriver's answer in case gawk isn't available:

perl -ne 'BEGIN{$/= "-"x72 . "\n"} chomp and print if /Fields/' log_file

Replace 72 by the actual number of dashes in your delimiter.

  • Nice! I must make an effort to learn some perl - I was trying to figure out a way to do a multiline match with grep -zP but couldn't – steeldriver May 12 '14 at 19:46
  • @steeldriver I have seen people around here refer to a tool called pcregrep that is capable of doing multiline greps. – Joseph R. May 12 '14 at 19:48
  • Thanks, I have added tentative grep -zP and pcregrep solutions - feel free to improve them since I only know just about enough perl to be dangerous! – steeldriver May 12 '14 at 23:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.