1

So, I have the following log sample:

2015-03-30 10:19:44,200 DEBUG [Default : 1] [Application:ABC BIW: IDA:6d87de8e-1276-4496-b49d-dd4cd375cbe4 Info:P23 Code:ABCD] A log message 1
2015-03-30 10:19:45,250 DEBUG [Default : 1] [Application:ABC BIW: IDA:6d87de8e-1276-4496-b49d-dd4cd375cbe4 Info:P23 Code:ABCD] A log message 2
2015-03-30 10:19:45,700 DEBUG [Default : 1] [Application:ABC BIW: IDA:6d87de8e-1276-4496-b49d-dd4cd375cbe4 Info:P23 Code:ABCD] log message of exception occurred in 
abc.cdef.class.sdk.SampleException: Message exception..
<stacktrace lines goes here>
2015-03-30 10:19:46,100 DEBUG [Default : 1] [Application:ABC BIW: IDA:6d87de8e-1276-4496-b49d-dd4cd375cbe4 Info:P23 Code:ABCD] A log message 3
2015-03-30 10:19:46,120 DEBUG [Default : 1] [Application:ABC BIW: IDA:6d87de8e-1276-4496-b49d-dd4cd375cbe4 Info:P23 Code:ABCD] A log message 4
2015-03-30 10:19:46,120 DEBUG [Default : 1] [Application:ABC BIW: IDA:6d800e80-5276-4006-a44d-dd4cea7bcbe1 Info:P13 Code:AB4D] A log message 6

Each request that comes in the server as a unique IDA. I want to grep all lines of log that have a certain IDA and put in another logfile with all the occurrences found. I’m using grep because it also append the file and line number ( grep –n ) where those patterns were found (the serach goes on 2 ou 3 files). But still, I also want that the stacktrace exceptions to be included, which is not happening because those lines don’t have the IDA that I am searching.

Furthermore that stacktrace exception may also happen as part of the last entry for a specific IDA, like this:

2015-03-30 10:19:44,200 DEBUG [Default : 1] [Application:ABC BIW: IDA:6d87de8e-1276-4496-b49d-dd4cd375cbe4 Info:P23 Code:ABCD] A log message 1
2015-03-30 10:19:45,200 DEBUG [Default : 1] [Application:ABC BIW: IDA:6d87de8e-1276-4496-b49d-dd4cd375cbe4 Info:P23 Code:BECD] A log message 2
2015-03-30 10:19:44,200 DEBUG [Default : 1] [Application:ABC BIW: IDA:6d87de8e-1276-4496-b49d-dd4cd375cbe4 Info:P23 Code:DFCD] A log message 3
2015-03-30 10:19:45,200 DEBUG [Default : 1] [Application:ABC BIW: IDA:6d87de8e-1276-4496-b49d-dd4cd375cbe4 Info:P23 Code:BBCD] A log message 4
2015-03-30 10:19:45,200 DEBUG [Default : 1] [Application:ABC BIW: IDA:6d87de8e-1276-4496-b49d-dd4cd375cbe4 Info:P23 Code:ABCC] log message of exception occurred in 
abc.cdef.class.sdk.SampleException: Message exception..
<stacktrace lines goes here>
2015-03-30 10:19:45,200 DEBUG [Default : 1] [Application:ABC BIW: IDA:ed8ede83-1116-4697-a69d-ee6cd575cbe7 Info:P23 Code:ABDD] A log message 5

Any suggestions to accomplish this?

P.S.1- I've been searching around, but I've mostle found searches between tags, which in my case is not good enough.

P.S.2-My solution os currently just a one liner grep, but i am open to other solutions, such as perl, awk, sed, python or any other "bash command way to go".

Improve this question

3 Answers 3

Reset to default
3

Perl solution:

perl -ne '$id = "6d87de8e-1276-4496-b49d-dd4cd375cbe4";
          print if $match = (/IDA:$id/ .. /IDA:(?!$id)/)
                and $match !~ /E0$/
         ' *.log

Explanation:

  • /regex1/ .. /regex2/ returns true for lines between the matches.
  • IDA:(?!$id) meand IDA: not followed by $id.
  • the last line in a range is denoted by the E0 suffix which the condition checks, so the last line (the one with a different IDA) is not printed.

To include file names and line numbers, use

print "$ARGV:$.:$_"

and add the following before the closing quote:

; $. = 0 if eof
Improve this answer
7
  • This is printing duplicates, i.e. lines matched preceeded by filename where it was found and another entry without the filename preceeded.
    – Sousa Gaspar
    Mar 30, 2015 at 22:31
  • @jik: You don't show any such lines in the samples.
    – choroba
    Mar 30, 2015 at 22:33
  • I mean you solution is giving me duplicates. Is not my sample that has those.
    – Sousa Gaspar
    Mar 30, 2015 at 22:36
  • @jik: When I run it on your samples, I don't get any duplicates. Maybe replace the *.log with the actual mask that doesn't include output of your previous grep command?
    – choroba
    Mar 30, 2015 at 22:38
  • Yeah, you're right. Sorry, my bad. But still, how do I get here the file name where it was found and the line number?
    – Sousa Gaspar
    Mar 30, 2015 at 22:46
1

This prints the respective data (each preceded with the line number) to files named as the IDA:

awk 'match($0,/IDA:[^ ]+/) { print NR, $0 > substr($0,RSTART+4,RLENGTH-4) }' logfile
Improve this answer
1
  • Exceptions are not considered here. Still is a nice awk work.
    – Sousa Gaspar
    Mar 30, 2015 at 22:51
1

Sed solution:

sed -n ':2;/IDA:6d87de8/{p;/exception/{:1;n;/IDA:/!{p;b1};b2}}'

The script suppres output (-n option), until meet appropriate IDA (/IDA:6d87de8/) which will be printed (p). Then pattern(line) will be checked for "exeption" presence and, if so, starts to operate with next line (n). If the next line do not consist IDA: the line will be printed and script goes to next line(loop started at :1 point). If the line cosist IDA: script starts from the begining(loop at :2 point)

If you'd like to enumerate lines - output lines by nl or grep -n ".*" and pipe it to sed script.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .