6

I have a file which contains some text and - on a single line - a marker which indicates where new content should be added

foo
bar
%SUBSTITUTE%
foo

The line line substitute should be replaced by a new multiline string text="some text" (note that I do not know the string it may for instance be the result of reading a file text=cat "file"``). The result of the replacement should read

foo
bar
some text
%SUBSTITUTE%
foo

I had a working version based on perl which stopped working (apparently due to perl version change). Now I am trying standard utilities like tr and sed to replace the line. I am having some trouble with it, because the string to be pasted may contain arbitrary characters including backslashes etc. I do not want to escape these before pasting.

Is there a safe way to do it which works with standard tools? The other questions I find for the problem are particular solutions where the text to be pasted is known.

6
  • You might be able to use the venerable (and somewhat scary) m4. The use case you describe is very similar to its intended functionality. Nov 15, 2014 at 12:48
  • Do you have an example?
    – highsciguy
    Nov 15, 2014 at 12:52
  • I'm unclear on your example. Shouldn't your marker %SUBSTITUTE% be gone in the second piece of text? It's still there. You also have The line line substitute. iIs that intentional? Nov 15, 2014 at 13:02
  • It should stay (which complicates the problem). The percent signs are to make it a tex comment. It indicates where the next chunk of text shall be inserted.
    – highsciguy
    Nov 15, 2014 at 13:06
  • use this command: sed '/%SUBSTITUTE%/i\some text\nnew line' file Nov 15, 2014 at 13:19

4 Answers 4

6

If you have the GNU version of sed, you should be able to use the r command to read and insert new content from a file, then delete the marker line e.g.

sed '/%SUBSTITUTE%/{
r path/to/newcontent
d
}' file

If you want to retain the %SUBSTITUTE% marker after the insertion, that's tricky because whatever you do, the GNU r extension queues the file contents until the end of the current pattern cycle (retaining it before would just be a matter of removing the d command). Probably the simplest way is to append it to the newcontent file: you could do that on the fly like

sed '/%SUBSTITUTE%/{
r /dev/stdin
d
}' file < <(sed '$a %SUBSTITUTE%' path/to/newcontent)

Taking a completely different approach, you could split the first file on %SUBSTITUTE% and then cat in the new content

csplit -s file '/%SUBSTITUTE%/'
cat xx00 newcontent xx01

You could also do a bash read loop over the lines of the first file, and cat the new content file when you match the marker string - however I've had my knuckles rapped for suggesting read loops for text processing on this forum. Unfortunately neither provides an in-place solution.

4
  • It would better be GNU independent but I will try it. I would however need to re-insert the %SUBSTITUTE% because it should stay in the file to indicate where the next substitution is to be made.
    – highsciguy
    Nov 15, 2014 at 12:50
  • sed -e "/%SUBSTITUTE%/i$(sed 's/$/\\/' path/to/newcontent)" -e 's/\\$//' file works too. The main trick is to blackslash newlines in newcontent than get it off. Note: If you have \n, \t, etc. its will substituted by its real meaning: «newline», «tab», etc.
    – Costas
    Nov 15, 2014 at 15:58
  • 1
    Additionally in my sed (GNU sed) 4.2.2 I found undocumented command e, so if supported you can do sed -e '/%SUBSTITUTE%/e cat /path/to/newcontent' file
    – Costas
    Nov 15, 2014 at 16:13
  • 1
    You don't need GNU sed for r - but neither thr GNU nor any other version will insert arbitrary text as a replacement to marker - ronly works on the end of the line cycle. @Costas suggestion, however, would work. But Costas - it is documented. Do info sed.
    – mikeserv
    Feb 19, 2015 at 2:38
3

Here's another way using ed.
Insert the whole content of FILE before the marker (i.e. before the line containing %SUBSTITUTE%):

ed -s originalfile <<< $'/%SUBSTITUTE%/- r FILE\nw\nq'

where:
/%SUBSTITUTE%/ : sets address at first line matching %SUBSTITUTE%
- or -1 : offsets address one line before
r FILE : Reads FILE to after the addressed line.
w : writes to originalfile (replace with ,p to just print the content instead of writing)
q : quits editor

Replacing FILE with !echo "$TEXT" will insert the content of $TEXT before the marker:

export TEXT
ed -s originalfile <<'IN'
/%SUBSTITUTE%/-1 r !echo "$TEXT"
w
q
IN
2

I found the following solution based on awk:

text=`cat "filepaste"`
export text;
<"$file" awk '
    BEGIN {REPLACE=ENVIRON["text"] "\n%SUBSTITUTE%" }
    {gsub(/^%SUBSTITUTE%$/, REPLACE); print}
'

Here "filepaste" contains the content to substitute for %SUBSTITUTE%. An advantage is that this string may be acted upon using different shell tools without the need to save it back to a file. Reading the awk variable REPLACE from the environment variable avoids the expansion of escaped characters in text.

0

Since you had a Perl solution, here's another. I am using rep.txt to store the replacement:

$ perl -pe '$re=`cat rep.txt`; chomp($re); s/%SUBSTITUTE%\n/$re/' file.txt
foo
bar
multi
  line
string
foo

That reads each line of the target file (file.txt), then applies the script given by -e to it and prints (-p).

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .