3

I have a log file with many instances of statements as shown below:

14 Aug 19 16:30:11.506: <DATA>  |POS|IDLE|1|01131844090|5950|$hostIp|$size  |$data
20 Aug 19 16:30:12.439: <DATA>  |POS|IDLE|1|01131844090|5950|$hostIp|$size  |$data
21 ###############################################################################

I want to replace IDLE just preceding the #########.. to say, END and have other cases of IDLE retained.

How do I achieve it?

Edit: Maybe something is different with my actual file. Answers to this question haven't met the requirement. Detailed extract and my requirement is shown below:

  22 Aug 19 16:47:33.159: <DATA>  |POS|RINGING|1|1126710938|5950|$hostIp|$size  |$data
    23 Aug 19 16:47:33.453: <DATA>  |POS|INIT|1|1126710938|5950|$hostIp|$size  |$data
    24 Aug 19 16:47:33.484: <DATA>  |POS|TRAINING|1|1126710938|5950|$hostIp|$size  |$data
    25 Aug 19 16:48:05.824: <DATA>  |POS|IDLE|1|1126710938|5950|$hostIp|$size  |$data
    26 Aug 19 16:48:05.916: <DATA>  |POS|INIT|1|1126710938|5950|$hostIp|$size  |$data
    27 Aug 19 16:48:05.947: <DATA>  |POS|IDLE|1|1126710938|5950|$hostIp|$size  |$data
    28 Aug 19 16:48:23.792: <DATA>  |POS|IDLE|1|1126710938|5950|$hostIp|$size  |$data
    29 Aug 19 16:48:23.853: <DATA>  |POS|INIT|1|1126710938|5950|$hostIp|$size  |$data
    30 Aug 19 16:48:23.884: <DATA>  |POS|IDLE|1|1126710938|5950|$hostIp|$size  |$data
    31 ##############################################################################

After operating on the above instance, the result should be: (IDLE preceding #### line is changed to END.Note there are many instances of such events. My actual need is to find time diff between RINGING and IDLE - aka END)

22 Aug 19 16:47:33.159: <DATA>  |POS|RINGING|1|1126710938|5950|$hostIp|$size  |$data
23 Aug 19 16:47:33.453: <DATA>  |POS|INIT|1|1126710938|5950|$hostIp|$size  |$data
24 Aug 19 16:47:33.484: <DATA>  |POS|TRAINING|1|1126710938|5950|$hostIp|$size  |$data
25 Aug 19 16:48:05.824: <DATA>  |POS|IDLE|1|1126710938|5950|$hostIp|$size  |$data
26 Aug 19 16:48:05.916: <DATA>  |POS|INIT|1|1126710938|5950|$hostIp|$size  |$data
27 Aug 19 16:48:05.947: <DATA>  |POS|IDLE|1|1126710938|5950|$hostIp|$size  |$data
28 Aug 19 16:48:23.792: <DATA>  |POS|IDLE|1|1126710938|5950|$hostIp|$size  |$data
29 Aug 19 16:48:23.853: <DATA>  |POS|INIT|1|1126710938|5950|$hostIp|$size  |$data
30 Aug 19 16:48:23.884: <DATA>  |POS|END|1|1126710938|5950|$hostIp|$size  |$data
31 ##############################################################################
2

Update: The ex command way will work only on the first matching line. The perl example will replace for all lines with the pattern, but will only give the output and not edit the origitnal file, so you will have to redirect the output to a new file. YOUR-LOGFILE should be replaced by the location of the file you are trying to modify.

perl -0pe 's/IDLE(.*\n.*###)/END\1/g' YOUR-LOGFILE > NEWFILE

This will replace the line just above the line containing the pattern and then exit.

ex -s +'/######/-1 s/IDLE/END/ | x' YOUR-LOGFILE

  • thanks. Yes. ex worked on the first instance. However, I see the perl script didn't meet the requirement either. – Prabhu Sep 8 '14 at 11:12
  • @Prabhu now you can use the perl script. Earlier there were some characters missing due to the SE editor's features. I have updated my answer now. – beginer Sep 8 '14 at 12:16
  • Sure. I will do. – Prabhu Sep 8 '14 at 12:36
2

Try:

$ awk '!/^#+$/ {print l; l=$0; next} {gsub("IDLE","END",l); print l; l=$0} END {print $0}' < log.txt

which gives:

Aug 19 16:30:11.506: <DATA>  |POS|IDLE|1|01131844090|5950|$hostIp|$size  |$data
Aug 19 16:30:12.439: <DATA>  |POS|END|1|01131844090|5950|$hostIp|$size  |$data
###############################################################################
  • ,thanks. With my large set not all IDLE in line preceding the line containing #####.. are replaced. But I see the line - ####.. getting duplicated above all occurrences of #####... – Prabhu Sep 8 '14 at 11:10
  • @garethTheRed: en.wikipedia.org/wiki/Cat_(Unix)#Useless_use_of_cat – cuonglm Sep 8 '14 at 11:13
  • @Gnouc - Guilty as charged! I've edited. – garethTheRed Sep 8 '14 at 11:16
  • Where did your log file come from? Is it correctly formatted as per your example? I've expanded your example by duplicating lines and it still works. Also, what about the newline characters? Is this from a Windows machine? – garethTheRed Sep 8 '14 at 11:34
  • I have edited the questions adding more details. I copied the contents from my VI editor. – Prabhu Sep 8 '14 at 11:42
1
sed '$!N;/###$/s/IDLE/END/;P;D
' <<\DATA
22 Aug 19 16:47:33.159: <DATA>  |POS|RINGING|1|1126710938|5950|$hostIp|$size  |$data
23 Aug 19 16:47:33.453: <DATA>  |POS|INIT|1|1126710938|5950|$hostIp|$size  |$data
24 Aug 19 16:47:33.484: <DATA>  |POS|TRAINING|1|1126710938|5950|$hostIp|$size  |$data
25 Aug 19 16:48:05.824: <DATA>  |POS|IDLE|1|1126710938|5950|$hostIp|$size  |$data
26 Aug 19 16:48:05.916: <DATA>  |POS|INIT|1|1126710938|5950|$hostIp|$size  |$data
27 Aug 19 16:48:05.947: <DATA>  |POS|IDLE|1|1126710938|5950|$hostIp|$size  |$data
28 Aug 19 16:48:23.792: <DATA>  |POS|IDLE|1|1126710938|5950|$hostIp|$size  |$data
29 Aug 19 16:48:23.853: <DATA>  |POS|INIT|1|1126710938|5950|$hostIp|$size  |$data
30 Aug 19 16:48:23.884: <DATA>  |POS|IDLE|1|1126710938|5950|$hostIp|$size  |$data
31 ##############################################################################
DATA

That expands sed's normal one-line buffer to a two-line buffer. On every line but the last it appends the next input line to pattern space. If the two-line buffer ends with the string ### it replaces the first occurring IDLE string with END. It then Prints up to the first occurring newline in the buffer and Deletes same before starting over with what remains. It's an extremely simple, POSIX-portable, and very fast solution to your problem.

OUTPUT

22 Aug 19 16:47:33.159: <DATA>  |POS|RINGING|1|1126710938|5950|$hostIp|$size  |$data
23 Aug 19 16:47:33.453: <DATA>  |POS|INIT|1|1126710938|5950|$hostIp|$size  |$data
24 Aug 19 16:47:33.484: <DATA>  |POS|TRAINING|1|1126710938|5950|$hostIp|$size  |$data
25 Aug 19 16:48:05.824: <DATA>  |POS|IDLE|1|1126710938|5950|$hostIp|$size  |$data
26 Aug 19 16:48:05.916: <DATA>  |POS|INIT|1|1126710938|5950|$hostIp|$size  |$data
27 Aug 19 16:48:05.947: <DATA>  |POS|IDLE|1|1126710938|5950|$hostIp|$size  |$data
28 Aug 19 16:48:23.792: <DATA>  |POS|IDLE|1|1126710938|5950|$hostIp|$size  |$data
29 Aug 19 16:48:23.853: <DATA>  |POS|INIT|1|1126710938|5950|$hostIp|$size  |$data
30 Aug 19 16:48:23.884: <DATA>  |POS|END|1|1126710938|5950|$hostIp|$size  |$data
31 ##############################################################################

To edit a file in-place portably you can do:

 sed '$!N;/###$/s/IDLE/END/;P;D
 ' <<FILE >file
 $(cat file)
 FILE

Or else, with GNU sed at least you might do:

 sed -i '$!N;/###$/s/IDLE/END/;P;D' file

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.