Replacing a pattern in line preceding another pattern of interest

Question

I have a log file with many instances of statements as shown below:

14 Aug 19 16:30:11.506: <DATA>  |POS|IDLE|1|01131844090|5950|$hostIp|$size  |$data
20 Aug 19 16:30:12.439: <DATA>  |POS|IDLE|1|01131844090|5950|$hostIp|$size  |$data
21 ###############################################################################

I want to replace IDLE just preceding the #########.. to say, END and have other cases of IDLE retained.

How do I achieve it?

Edit: Maybe something is different with my actual file. Answers to this question haven't met the requirement. Detailed extract and my requirement is shown below:

  22 Aug 19 16:47:33.159: <DATA>  |POS|RINGING|1|1126710938|5950|$hostIp|$size  |$data
    23 Aug 19 16:47:33.453: <DATA>  |POS|INIT|1|1126710938|5950|$hostIp|$size  |$data
    24 Aug 19 16:47:33.484: <DATA>  |POS|TRAINING|1|1126710938|5950|$hostIp|$size  |$data
    25 Aug 19 16:48:05.824: <DATA>  |POS|IDLE|1|1126710938|5950|$hostIp|$size  |$data
    26 Aug 19 16:48:05.916: <DATA>  |POS|INIT|1|1126710938|5950|$hostIp|$size  |$data
    27 Aug 19 16:48:05.947: <DATA>  |POS|IDLE|1|1126710938|5950|$hostIp|$size  |$data
    28 Aug 19 16:48:23.792: <DATA>  |POS|IDLE|1|1126710938|5950|$hostIp|$size  |$data
    29 Aug 19 16:48:23.853: <DATA>  |POS|INIT|1|1126710938|5950|$hostIp|$size  |$data
    30 Aug 19 16:48:23.884: <DATA>  |POS|IDLE|1|1126710938|5950|$hostIp|$size  |$data
    31 ##############################################################################

After operating on the above instance, the result should be: (IDLE preceding #### line is changed to END.Note there are many instances of such events. My actual need is to find time diff between RINGING and IDLE - aka END)

22 Aug 19 16:47:33.159: <DATA>  |POS|RINGING|1|1126710938|5950|$hostIp|$size  |$data
23 Aug 19 16:47:33.453: <DATA>  |POS|INIT|1|1126710938|5950|$hostIp|$size  |$data
24 Aug 19 16:47:33.484: <DATA>  |POS|TRAINING|1|1126710938|5950|$hostIp|$size  |$data
25 Aug 19 16:48:05.824: <DATA>  |POS|IDLE|1|1126710938|5950|$hostIp|$size  |$data
26 Aug 19 16:48:05.916: <DATA>  |POS|INIT|1|1126710938|5950|$hostIp|$size  |$data
27 Aug 19 16:48:05.947: <DATA>  |POS|IDLE|1|1126710938|5950|$hostIp|$size  |$data
28 Aug 19 16:48:23.792: <DATA>  |POS|IDLE|1|1126710938|5950|$hostIp|$size  |$data
29 Aug 19 16:48:23.853: <DATA>  |POS|INIT|1|1126710938|5950|$hostIp|$size  |$data
30 Aug 19 16:48:23.884: <DATA>  |POS|END|1|1126710938|5950|$hostIp|$size  |$data
31 ##############################################################################

beginer · Accepted Answer · 2014-09-08 12:15:11Z

2

Update: The ex command way will work only on the first matching line. The perl example will replace for all lines with the pattern, but will only give the output and not edit the origitnal file, so you will have to redirect the output to a new file. YOUR-LOGFILE should be replaced by the location of the file you are trying to modify.

perl -0pe 's/IDLE(.*\n.*###)/END\1/g' YOUR-LOGFILE > NEWFILE

This will replace the line just above the line containing the pattern and then exit.

ex -s +'/######/-1 s/IDLE/END/ | x' YOUR-LOGFILE

edited Sep 8, 2014 at 12:15

answered Sep 8, 2014 at 10:42

beginer

2,66816 silver badges18 bronze badges

thanks. Yes. ex worked on the first instance. However, I see the perl script didn't meet the requirement either.
– Prabhu
Sep 8, 2014 at 11:12
@Prabhu now you can use the perl script. Earlier there were some characters missing due to the SE editor's features. I have updated my answer now.
– beginer
Sep 8, 2014 at 12:16
Sure. I will do.
– Prabhu
Sep 8, 2014 at 12:36

Add a comment |

garethTheRed · Accepted Answer · 2014-09-08 13:44:09Z

2

Try:

$ awk '!/^#+$/ {print l; l=$0; next} {gsub("IDLE","END",l); print l; l=$0} END {print $0}' < log.txt

which gives:

Aug 19 16:30:11.506: <DATA>  |POS|IDLE|1|01131844090|5950|$hostIp|$size  |$data
Aug 19 16:30:12.439: <DATA>  |POS|END|1|01131844090|5950|$hostIp|$size  |$data
###############################################################################

edited Sep 8, 2014 at 13:44

answered Sep 8, 2014 at 10:56

garethTheRed

33.5k4 gold badges92 silver badges101 bronze badges

,thanks. With my large set not all IDLE in line preceding the line containing #####.. are replaced. But I see the line - ####.. getting duplicated above all occurrences of #####...
– Prabhu
Sep 8, 2014 at 11:10
@garethTheRed: en.wikipedia.org/wiki/Cat_(Unix)#Useless_use_of_cat
– cuonglm
Sep 8, 2014 at 11:13
@Gnouc - Guilty as charged! I've edited.
– garethTheRed
Sep 8, 2014 at 11:16
Where did your log file come from? Is it correctly formatted as per your example? I've expanded your example by duplicating lines and it still works. Also, what about the newline characters? Is this from a Windows machine?
– garethTheRed
Sep 8, 2014 at 11:34
I have edited the questions adding more details. I copied the contents from my VI editor.
– Prabhu
Sep 8, 2014 at 11:42

| Show 6 more comments

Community · Accepted Answer · 2020-06-11 12:04:56Z

sed '$!N;/###$/s/IDLE/END/;P;D
' <<\DATA
22 Aug 19 16:47:33.159: <DATA>  |POS|RINGING|1|1126710938|5950|$hostIp|$size  |$data
23 Aug 19 16:47:33.453: <DATA>  |POS|INIT|1|1126710938|5950|$hostIp|$size  |$data
24 Aug 19 16:47:33.484: <DATA>  |POS|TRAINING|1|1126710938|5950|$hostIp|$size  |$data
25 Aug 19 16:48:05.824: <DATA>  |POS|IDLE|1|1126710938|5950|$hostIp|$size  |$data
26 Aug 19 16:48:05.916: <DATA>  |POS|INIT|1|1126710938|5950|$hostIp|$size  |$data
27 Aug 19 16:48:05.947: <DATA>  |POS|IDLE|1|1126710938|5950|$hostIp|$size  |$data
28 Aug 19 16:48:23.792: <DATA>  |POS|IDLE|1|1126710938|5950|$hostIp|$size  |$data
29 Aug 19 16:48:23.853: <DATA>  |POS|INIT|1|1126710938|5950|$hostIp|$size  |$data
30 Aug 19 16:48:23.884: <DATA>  |POS|IDLE|1|1126710938|5950|$hostIp|$size  |$data
31 ##############################################################################
DATA

That expands sed's normal one-line buffer to a two-line buffer. On every line but the last it appends the next input line to pattern space. If the two-line buffer ends with the string ### it replaces the first occurring IDLE string with END. It then Prints up to the first occurring newline in the buffer and Deletes same before starting over with what remains. It's an extremely simple, POSIX-portable, and very fast solution to your problem.

OUTPUT

22 Aug 19 16:47:33.159: <DATA>  |POS|RINGING|1|1126710938|5950|$hostIp|$size  |$data
23 Aug 19 16:47:33.453: <DATA>  |POS|INIT|1|1126710938|5950|$hostIp|$size  |$data
24 Aug 19 16:47:33.484: <DATA>  |POS|TRAINING|1|1126710938|5950|$hostIp|$size  |$data
25 Aug 19 16:48:05.824: <DATA>  |POS|IDLE|1|1126710938|5950|$hostIp|$size  |$data
26 Aug 19 16:48:05.916: <DATA>  |POS|INIT|1|1126710938|5950|$hostIp|$size  |$data
27 Aug 19 16:48:05.947: <DATA>  |POS|IDLE|1|1126710938|5950|$hostIp|$size  |$data
28 Aug 19 16:48:23.792: <DATA>  |POS|IDLE|1|1126710938|5950|$hostIp|$size  |$data
29 Aug 19 16:48:23.853: <DATA>  |POS|INIT|1|1126710938|5950|$hostIp|$size  |$data
30 Aug 19 16:48:23.884: <DATA>  |POS|END|1|1126710938|5950|$hostIp|$size  |$data
31 ##############################################################################

To edit a file in-place portably you can do:

 sed '$!N;/###$/s/IDLE/END/;P;D
 ' <<FILE >file
 $(cat file)
 FILE

Or else, with GNU sed at least you might do:

 sed -i '$!N;/###$/s/IDLE/END/;P;D' file

Stack Exchange Network

Replacing a pattern in line preceding another pattern of interest

3 Answers 3

OUTPUT

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged
text-processing
regular-expression
.

Hot Network Questions

Replacing a pattern in line preceding another pattern of interest

3 Answers 3

OUTPUT

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged text-processingregular-expression.

Related

Hot Network Questions

Not the answer you're looking for? Browse other questions tagged
text-processing
regular-expression
.