3

I have a logfile that I am trying to 'grep' data out of via a bash script. The data I am specifically after is all lines between two timestamps (including the top timestamp) that have the pattern "ERR-" and include an empty line after each entry for readability.

example logfile:

Tue May 24 21:22:12 2022
ERR-0045 Lock detected in /tmp/file.lck
Tue May 24 21:44:12 2022
Errors in file /tmp/filename01.trc:
ERR-0001: Error detected. /tmp/filename.log
Tue May 24 21:47:25 2022
im some output
Tue May 24 21:47:25 2022
im some output too
im some output aswell
Tue May 24 21:48:03 2022
Errors in file /tmp/filename09.trc:
ERR-0100: error
ERR-0050: failure of sorts.
ERR-0052: line 3421
Tue May 24 21:49:07 2022
Completed process xyz

So my desired output would look like :

Tue May 24 21:22:12 2022
ERR-0045 Lock detected in /tmp/file.lck
    
Tue May 24 21:44:12 2022
Errors in file /tmp/filename01.trc:
ERR-0001: Error detected. /tmp/filename.log
        
Tue May 24 21:48:03 2022
Errors in file /tmp/filename09.trc:
ERR-0100: error
ERR-0050: failure of sorts.
ERR-0052: line 3421

I have tried using combinations of sed/awk/cat without much success. Where I am having trouble is:

  1. It's not always two lines before ERR- which contains the timestamp
  2. There can be multiple ERR-'s in one timestamp block
  3. The date will obviously change so i don't want to hard code that in, although the format won't change.

Thanks in advance

0

2 Answers 2

3

Script tst.awk:

function print_r() { if (e) print r; r = ""; e = 0 }

/^([[:alpha:]]{3} ){2}[[:digit:]]{1,2} [[:digit:]]{2}(:[[:digit:]]{2}){2} [[:digit:]]{4}$/ {
    print_r()
}
/^ERR-/{ e = 1 }
{ r = r ORS $0 }
END{ print_r() }

Usage and output:

$ awk -f tst.awk file

Tue May 24 21:22:12 2022
ERR-0045 Lock detected in /tmp/file.lck

Tue May 24 21:44:12 2022
Errors in file /tmp/filename01.trc:
ERR-0001: Error detected. /tmp/filename.log

Tue May 24 21:48:03 2022
Errors in file /tmp/filename09.trc:
ERR-0100: error
ERR-0050: failure of sorts.
ERR-0052: line 3421

You can use your own expressions for the timestamp or the error. Here is a whole line matching of the presented format (no date validation). And "ERR-" at the beginning of a line.


Notes:

  • We define a function to print a record (r) if error found (e). Also to reset both variables after printing. For awk, variables that have not been initialised with a value are evaluated to the empty string or zero.

  • When the regular expression for the date is matched, we call this function. To finish about previous log record and to start holding the new record. Because the record is a multiline log, we don't know yet if we have to print it or not.

  • When the error pattern is matching, we set e.

  • For every row, we append the row to the existing record, separated by the ORS, the output record separator, default newline. Also, the empty line between output lines is placed at the beginning of r, r is always the empty string when we are here for a new timestamp.

  • At the END we call the function again, because the last record is still held.

2
  • This is exactly what i wanted and it works perfectly with my "real data". I certainly don't understand the code - would you mind giving a pseudo code explanation of what it does? If not that is okay too - thank you so much. May 26, 2022 at 3:41
  • I have updated with some notes, welcome.
    – thanasisp
    May 26, 2022 at 7:35
0

Using Raku (formerly known as Perl_6)

~$ raku -e 'my @a=slurp; @a.=split(/ <?after ^^ ERR \V* > \n <?before ^^ Tue > /); \
            put $_.subst(/^ <(.+)> ^^ Tue/) ~ "\n" if /^^ ERR / for @a;'   file

The problem seems to be that you have some timestamped records with ERR lines and some without, and you'd like to discard the latter.

Above, the file is slurped into @a array. Then @a is split on \n between records <?after ^^ ERR \V* > ending with a line that starts with ERR followed by zero-or-more \V non-vertical whitespace characters, and <?before ^^ Tue > before a line starting with Tue.

In the second statement output(…) is refined. The @a elements are individually (and automatically) represented by $_ in the preamble to the for clause. Each element is if /^^ ERR/ conditionally-tested to make sure they contain a line starting with ERR: if so, the $_ element is $_.subst(/^ <(.+)> ^^ Tue/) substituted removing all leading characters before an internal start-of-line Tue. Essentially here, .subst without a replacement means to delete the match between <()> capture markers. To separate individual records, $_.subst(…) ~ "\n" elements are ~ tilde concatenated with a record-separating "\n" newline.

Sample Input:

Tue May 24 21:22:12 2022
ERR-0045 Lock detected in /tmp/file.lck
Tue May 24 21:44:12 2022
Errors in file /tmp/filename01.trc:
ERR-0001: Error detected. /tmp/filename.log
Tue May 24 21:47:25 2022
im some output
Tue May 24 21:47:25 2022
im some output too
im some output aswell
Tue May 24 21:48:03 2022
Errors in file /tmp/filename09.trc:
ERR-0100: error
ERR-0050: failure of sorts.
ERR-0052: line 3421
Tue May 24 21:49:07 2022
Completed process xyz

Sample Output:

Tue May 24 21:22:12 2022
ERR-0045 Lock detected in /tmp/file.lck

Tue May 24 21:44:12 2022
Errors in file /tmp/filename01.trc:
ERR-0001: Error detected. /tmp/filename.log

Tue May 24 21:48:03 2022
Errors in file /tmp/filename09.trc:
ERR-0100: error
ERR-0050: failure of sorts.
ERR-0052: line 3421

Certainly, the above is a bare bones implementation (it's meant to be--for readability). To work on dates starting on other days of the week, substitute Tues above with the following:

 [ Mon || Tue || Wed || Thu || Fri || Sat || Sun ] 

(It should be obvious from the code above how to add months to your regex, should you run into problems with specificity).

https://raku.org

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .