I use shell scripts to setup different types of VMs. Often these scripts include multiline variables that need to be inserted into config files at certain positions using sed.

If I create them like this, everything is fine:

VAR="Config Line1\nConfig Line2\nConfig Line 3"
sed -i "/MatchingPattern/ a $VAR" somefile

This doesn't make the script very readable though, especially since the text blocks can be quite long.

If I write them like so:

VAR="Config Line1
Config Line2
Config Line 3"
sed -i "/MatchingPattern/ a $VAR" somefile

I get an error when running the script: sed: -e expression #1, char 31: unknown command:C'`

Is there a way to use sed with variables declared like that?

  • 1
    I find it easier to use awk, wrapped in a bash script, for this kinds of task. I use the Bash part to provide usage info (-h or --help), as well as create a temporary working directory that is automagically deleted when bash exits (i.e, an EXIT trap). That way, if there is an error or no required matches found, I can choose not to overwrite the original file. – Nominal Animal Dec 15 '16 at 18:38

The standard syntax for the a command is:

sed -e 'a\
first line\
  second line\
last line'

BSD (at least FreeBSD and OS/X) sed strips the leading blanks, and needs the -e to work around a bug. GNU sed allows moving the first line to right after the a\ as long as it's non-empty.

So, you'd need to preprocess the input:

VAR='content with
multiple lines
  some with lead blanks
or even backslash\'

preprocessed_VAR=$(printf '%s\n' "$VAR" |
  sed 's/\\/&&/g;s/^[[:blank:]]/\\&/;s/$/\\/')

sed -i -e "/MatchingPattern/a\\
${preprocessed_VAR%?}" somefile

(replace -i with -i '' on FreeBSD or OS/X).

On GNU/Linux and with the GNU shell (bash) or zsh, you can do instead:

sed -i '/MatchingPattern/r /dev/stdin' <<< "$VAR"

That works because bash and zsh implement here-strings with a deleted temporary files and /dev/stdin on Linux is implemented as a symlink to the file (which means it can be opened several time, and each time, it's open at the beginning).

Here you could also use GNU awk instead:

(export VAR
gawk -i inplace '{print}; /MatchingPattern/{print ENVIRON["VAR"]}' somefile)

Or perl (where that -i comes from):

(export VAR
perl -pi -e '$_ .= "$ENV{VAR}\n" if /MatchingPattern/' somefile)

The error is due to the sed append command a needing a backslash after the actual a:

With GNU sed:

$ sed "/pattern/a\\$VAR" file.in >file.out

BSD sed, is, as far as I know, only able to insert a single line into a file with the a command, unless the newlines are properly escaped.

Closely related answer for how to solve this with BSD sed: http://unix.stackexchange.com/a/60322

  • 1
    They can insert multiple line. That's what it's for. But you need a backslash at the end of all but the last line in all implementations. Otherwise, the second line would be treated as the next command in the sed script. – Stéphane Chazelas Sep 7 '17 at 14:16
  • @StéphaneChazelas Thanks, but that's what I believe I pointed out. "unless the newlines are properly escaped" – Kusalananda Sep 7 '17 at 14:27
  • Yes, but that's the same in GNU sed or any sed. – Stéphane Chazelas Sep 7 '17 at 14:31
  • @StéphaneChazelas The difference is that GNU sed interprets the \n in the string. – Kusalananda Sep 7 '17 at 14:32
  • Ah OK, though that's another (non-standard and non compliant as POSIX requires the \n to be inserted as n) way to "escape the newlines" – Stéphane Chazelas Sep 7 '17 at 14:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.