14

I have to replace a large block of a text (shell script code) in a file with another block of text.

I am impressed with the How can I use sed to replace a multi-line string? answered by antak and Multi-line replace answered by Bruce Ediger

But I have some trouble in using them.

  1. antak already mentioned in his answer that streaming entire file (1h;2,$H;$!d;g;) to the buffer is not advisable for large files, because it overload memory.

  2. I know sed can be used with the block feature to preserve the text outside the block unchanged. I want to use this feature. But if I use,

    sed -i '/marker1/,/marker2/s/.*/new text (code)/' filename
    

it will insert new text (code) repeatedly for each stream. Hence I have to make the visual block as one stream, using something similar to what suggested by antak earlier, but for the block (not for entire file).

  1. As mentioned by Bruce Ediger append feature of ex which begin with a end with .(dot) can be tried, but my new text (code) contain lines begin with dot, which may be considered as the dot of append syntax. How can I use it in this situation?

  2. ex's dd 'number of lines' may delete multiple lines, but if I have a block between /marker1/ and /marker2/ with the number of lines not fixed (varying) is to be replaced with new text (code), how to do it ?

6
  • I find your use of the term "visual block" unusual and confusing. Do you mean simply a block (group) of consecutive, complete lines? Aug 16, 2016 at 6:55
  • Yes, I am editing the question to avoid confusion.
    – gibies
    Aug 16, 2016 at 7:04
  • What is the source for the replacement text ? Is it in a file or is it "only in the programmers mind" ? Aug 16, 2016 at 10:48
  • 1
    This is for situational patching of a shell script. Hence the source for the replacement text is the programmer's mind (Or a master script on which sed or ex command works).
    – gibies
    Aug 16, 2016 at 11:50
  • Don Crissti I would have used r the read file command, to copy the text from another file, if source for the replacement text is in a file.
    – gibies
    Aug 16, 2016 at 12:01

3 Answers 3

16

I suggest using the change command (which is essentially a delete coupled with an append, though the append is only applied for the last line in the range which is exactly what you want here):

sed -i '/marker1/,/marker2/c\
New text 1\
New text 2' filename

Here using GNU sed's syntax for in-place editing (-i). That c command is otherwise standard and portable. GNU sed supports:

sed '/marker1/,/marker2/cNew text 1\
New text 2' filename

as a non-standard extension.

Newline and backslash characters must be escaped (with backslash) in the replacement text.

3
  • The change command works fine for both sed and ex alike. Thank you very much.
    – gibies
    Aug 16, 2016 at 8:03
  • Is there any way to avoid "\" and use here document for sed as in the case of ex syntax suggested by Bruce Ediger in Multi-line replace.
    – gibies
    Aug 16, 2016 at 13:40
  • You wouldn't need the backslashes if you use the r command discussed in other answers to that question.  I'm not sure how you would use a here document with sed. Aug 16, 2016 at 23:24
9

Using GNU sed

To replace lines starting a line matching 3 and continuing to a line matching 5 with New Code:

$ seq 8 | sed '/3/,/5/{/5/ s/.*/New Code/; t; d}'
1
2
New Code
6
7
8

For lines in the range /3/,/5/, we test to see if the line matches 5 (meaning that this is the last line in the group) and, if so, we do the substitution to insert New Code. If the substitution was made, then the t command tells sed to jump to the end of the commands (in which case New Code is printed). If not, the d command tells sed to delete the line.

All other lines are printed normally.

To change the file in place, the -i option can be used:

sed -i.bak '/3/,/5/{/5/ s/.*/New Code/; t; d}' file

Using awk

To do the same using awk:

$ seq 8 | awk '/3/,/5/{if (!f)print "New Code"; f=1; next}; 1'
1
2
New Code
6
7
8

The awk command treats lines in the range /3/,/5/ specially. For lines in that range, we test to see if f is zero (that is, if !f is true) and, if so, we print New Code and set f to 1 and then we skip the rest of the commands and jump to the next line.

For lines outside the range /3/,/5/, no jump is done and the 1 causes the line to be printed. In more detail, 1 is a condition. Because 1 is not zero, the condition evaluates to true. Because no action is associated with the condition, the default action is performed which is to print the line. Thus 1 is shorthand for print-the-line.

To change a file in place, the -i inplace option can be used with GNU awk 4.1 or above:

awk -i inplace '/3/,/5/{if (!f)print "New Code"; f=1; next} 1' file
4
  • cool. I am not good at awk, could you please explain bit your awk expression
    – Rahul
    Aug 16, 2016 at 6:49
  • 1
    @Rahul Sure. I have added an explanation of the awk code.
    – John1024
    Aug 16, 2016 at 6:57
  • 1
    Thanks John This is exactly what I was looking for.
    – gibies
    Aug 16, 2016 at 7:44
  • Much simple to remove block then add new one to last line: sed '/3/,/4/d;/5/cNew Code' awk '/5/{print "New Code"}/3/,/5/{next}1'
    – Costas
    Aug 16, 2016 at 10:03
2

Based on the above discussion I come up with a solution using ex

seq 15 > test1.txt
ex test1.txt << EOEX
/^7/,/^9/c
abcd
123
.xyz
hfr4
.
w!
q
EOEX
cat test1.txt

The above gives

1
2
3
4
5
6
abcd
123
.xyz
hfr4
10
11
12
13
14
15

Thanks g-man to make me aware of change command and give clarification about dot. Both sed and ex share almost similar syntax for the change command (also for delete command, append command, etc).

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .