Extract part of the logs to other file

Question

1) We have a process.log file in which we have lot of text data and in between we have some XML data published.
2) There of thousands of different XML published in the logs along with other text data.
3) Now i need to select only the XML files which are published after Outgoing XML: value
4) Also the XML file which must be selected and copied to a new file should be the one which matches the value in the ALERTID tag.
5) The ALERTID value will be provided in the script input. So in our case mGMjhgHgffHhhFdH1u4 will be provided in the input and we need to select the full XML file published for this alertid. Starting tag is from <xml version..> and ending tag is </Alert>
5) So i need to select the relevant Outgoing XML file in a new file based on a particular ALERTID so it can be replayed in different environments.

Format of the log file is like below:

Info Jan 11 17:30:26.12122 The process is not responding to heartbeats
Debug Jan 11 17:30:26.12123  Incoming XML :<xml version "1.0" encoding ="UTF-8"?>
<Alert trigger = "true" >
<Alerttype>orderReject</Alerttype>
<AlertID>ghghfsjUtYuu78T1</AlertID>
<Order>uusingas</Order>
<Quantity>1254</Quanity>
</Alert> (CreateInitEventHandler. C:356)
Debug Jan 11 17:30:26.12199 The process is going down with warnings
Debug Jan 11 17:30:26.148199 Outgoing XML: <xml version "1.0" encoding ="UTF-8"?>
<Alert trigger = "true" >
<Alerttype>orderheld</Alerttype>
<AlertID>mGMjhgHgffHhhFdH1u4</AlertID>
<Order>uwiofhdf</Order>
<Quantity>7651</Quanity>
</Alert>(CreateEventHandler. C:723)
Debug Jan 11 17:30:26.13214 The process has restarted and thread opened
Debug Jan 11 17:30:26.13215 The heartbeat is recieved from alertlistener process

Now the requirement is to take AlertID in the input, scan the process log and extract the matching outgoing XML in a separate file.

Using awk i am able to extract all the outgoing xml files but not sure how to extract the one related to a particular AlertID.

Also i cannot install/use any new XML parser as per the company policy. This needs to be achieved using shell/perl/awk/sed

Eg:

awk '/Outgoing/{p=1; s=$0} P & & /<\/Alert>/ {print $0 FS s; s="" ;p=0}p' 1.log>2.log

Answer 1

Supposing your ID is given in a variable called ALERTID:

sed -e '/Outgoing XML/!d;:a' -e '$d;N;s/.*\(<xml version.*<\/Alert>\).*/\1/;Ta' -e "/$ALERTID/!d" yourfile.log

Explanation:

• /Outgoing XML/!d;:a delete stuff until the Outgoing XML line and start a loop then
• $d to delete an unfinished record at the end of the file
• N;s/.*\(<xml version.*<\/Alert>\).*/\1/;Ta appends lines until the </Alert> tag is found and removes everything before and after the desired block "/$ALERTID/!ddeletes blocks without the$ALERTID`

Maybe better to read:

sed '/Outgoing XML/!d;:a
     $d;N
     s/.*\(<xml version.*<\/Alert>\).*/\1/;Ta
     /'$ALERTID'/!d' yourfile.log

Answer 2

create a shell script, getalert.sh, with following contents:

awk '
/^Debug .* Outgoing XML/{
  sub(/^.* Outgoing XML: /,"")
  H=$0
  LC=0
  next
}
/<\/Alert>/ {
  sub(/Alert>.*$/,"Alert>")
  if (LC>0) {print}
  LC=0
  next
}

/<AlertID>'$1'<\/AlertID>/{
  print H
  print
  LC=1
  next
}

/<AlertID>.*<\/AlertID>/{
  H=""
  LC=0
  next
}

{ if (LC > 0) {
    print
  } else {
    H = H $0
  }
}' $2

run it as

getalert.sh mGMjhgHgffHhhFdH1u4 process.log