Replacing a pattern in line preceding another pattern of interest

Question

I have a log file with many instances of statements as shown below:

14 Aug 19 16:30:11.506: <DATA>  |POS|IDLE|1|01131844090|5950|$hostIp|$size  |$data
20 Aug 19 16:30:12.439: <DATA>  |POS|IDLE|1|01131844090|5950|$hostIp|$size  |$data
21 ###############################################################################

I want to replace IDLE just preceding the #########.. to say, END and have other cases of IDLE retained.

How do I achieve it?

Edit: Maybe something is different with my actual file. Answers to this question haven't met the requirement. Detailed extract and my requirement is shown below:

  22 Aug 19 16:47:33.159: <DATA>  |POS|RINGING|1|1126710938|5950|$hostIp|$size  |$data
    23 Aug 19 16:47:33.453: <DATA>  |POS|INIT|1|1126710938|5950|$hostIp|$size  |$data
    24 Aug 19 16:47:33.484: <DATA>  |POS|TRAINING|1|1126710938|5950|$hostIp|$size  |$data
    25 Aug 19 16:48:05.824: <DATA>  |POS|IDLE|1|1126710938|5950|$hostIp|$size  |$data
    26 Aug 19 16:48:05.916: <DATA>  |POS|INIT|1|1126710938|5950|$hostIp|$size  |$data
    27 Aug 19 16:48:05.947: <DATA>  |POS|IDLE|1|1126710938|5950|$hostIp|$size  |$data
    28 Aug 19 16:48:23.792: <DATA>  |POS|IDLE|1|1126710938|5950|$hostIp|$size  |$data
    29 Aug 19 16:48:23.853: <DATA>  |POS|INIT|1|1126710938|5950|$hostIp|$size  |$data
    30 Aug 19 16:48:23.884: <DATA>  |POS|IDLE|1|1126710938|5950|$hostIp|$size  |$data
    31 ##############################################################################

After operating on the above instance, the result should be: (IDLE preceding #### line is changed to END.Note there are many instances of such events. My actual need is to find time diff between RINGING and IDLE - aka END)

22 Aug 19 16:47:33.159: <DATA>  |POS|RINGING|1|1126710938|5950|$hostIp|$size  |$data
23 Aug 19 16:47:33.453: <DATA>  |POS|INIT|1|1126710938|5950|$hostIp|$size  |$data
24 Aug 19 16:47:33.484: <DATA>  |POS|TRAINING|1|1126710938|5950|$hostIp|$size  |$data
25 Aug 19 16:48:05.824: <DATA>  |POS|IDLE|1|1126710938|5950|$hostIp|$size  |$data
26 Aug 19 16:48:05.916: <DATA>  |POS|INIT|1|1126710938|5950|$hostIp|$size  |$data
27 Aug 19 16:48:05.947: <DATA>  |POS|IDLE|1|1126710938|5950|$hostIp|$size  |$data
28 Aug 19 16:48:23.792: <DATA>  |POS|IDLE|1|1126710938|5950|$hostIp|$size  |$data
29 Aug 19 16:48:23.853: <DATA>  |POS|INIT|1|1126710938|5950|$hostIp|$size  |$data
30 Aug 19 16:48:23.884: <DATA>  |POS|END|1|1126710938|5950|$hostIp|$size  |$data
31 ##############################################################################

Answer 1

Update: The ex command way will work only on the first matching line. The perl example will replace for all lines with the pattern, but will only give the output and not edit the origitnal file, so you will have to redirect the output to a new file. YOUR-LOGFILE should be replaced by the location of the file you are trying to modify.

perl -0pe 's/IDLE(.*\n.*###)/END\1/g' YOUR-LOGFILE > NEWFILE

This will replace the line just above the line containing the pattern and then exit.

ex -s +'/######/-1 s/IDLE/END/ | x' YOUR-LOGFILE

Answer 2

Try:

$ awk '!/^#+$/ {print l; l=$0; next} {gsub("IDLE","END",l); print l; l=$0} END {print $0}' < log.txt

which gives:

Aug 19 16:30:11.506: <DATA>  |POS|IDLE|1|01131844090|5950|$hostIp|$size  |$data
Aug 19 16:30:12.439: <DATA>  |POS|END|1|01131844090|5950|$hostIp|$size  |$data
###############################################################################

Answer 3

sed '$!N;/###$/s/IDLE/END/;P;D
' <<\DATA
22 Aug 19 16:47:33.159: <DATA>  |POS|RINGING|1|1126710938|5950|$hostIp|$size  |$data
23 Aug 19 16:47:33.453: <DATA>  |POS|INIT|1|1126710938|5950|$hostIp|$size  |$data
24 Aug 19 16:47:33.484: <DATA>  |POS|TRAINING|1|1126710938|5950|$hostIp|$size  |$data
25 Aug 19 16:48:05.824: <DATA>  |POS|IDLE|1|1126710938|5950|$hostIp|$size  |$data
26 Aug 19 16:48:05.916: <DATA>  |POS|INIT|1|1126710938|5950|$hostIp|$size  |$data
27 Aug 19 16:48:05.947: <DATA>  |POS|IDLE|1|1126710938|5950|$hostIp|$size  |$data
28 Aug 19 16:48:23.792: <DATA>  |POS|IDLE|1|1126710938|5950|$hostIp|$size  |$data
29 Aug 19 16:48:23.853: <DATA>  |POS|INIT|1|1126710938|5950|$hostIp|$size  |$data
30 Aug 19 16:48:23.884: <DATA>  |POS|IDLE|1|1126710938|5950|$hostIp|$size  |$data
31 ##############################################################################
DATA

That expands sed's normal one-line buffer to a two-line buffer. On every line but the last it appends the next input line to pattern space. If the two-line buffer ends with the string ### it replaces the first occurring IDLE string with END. It then Prints up to the first occurring newline in the buffer and Deletes same before starting over with what remains. It's an extremely simple, POSIX-portable, and very fast solution to your problem.

OUTPUT

22 Aug 19 16:47:33.159: <DATA>  |POS|RINGING|1|1126710938|5950|$hostIp|$size  |$data
23 Aug 19 16:47:33.453: <DATA>  |POS|INIT|1|1126710938|5950|$hostIp|$size  |$data
24 Aug 19 16:47:33.484: <DATA>  |POS|TRAINING|1|1126710938|5950|$hostIp|$size  |$data
25 Aug 19 16:48:05.824: <DATA>  |POS|IDLE|1|1126710938|5950|$hostIp|$size  |$data
26 Aug 19 16:48:05.916: <DATA>  |POS|INIT|1|1126710938|5950|$hostIp|$size  |$data
27 Aug 19 16:48:05.947: <DATA>  |POS|IDLE|1|1126710938|5950|$hostIp|$size  |$data
28 Aug 19 16:48:23.792: <DATA>  |POS|IDLE|1|1126710938|5950|$hostIp|$size  |$data
29 Aug 19 16:48:23.853: <DATA>  |POS|INIT|1|1126710938|5950|$hostIp|$size  |$data
30 Aug 19 16:48:23.884: <DATA>  |POS|END|1|1126710938|5950|$hostIp|$size  |$data
31 ##############################################################################

To edit a file in-place portably you can do:

 sed '$!N;/###$/s/IDLE/END/;P;D
 ' <<FILE >file
 $(cat file)
 FILE

Or else, with GNU sed at least you might do:

 sed -i '$!N;/###$/s/IDLE/END/;P;D' file