5

Thanks to this link I know how to pass a variable that contains slashes as a pattern to sed:

sed "s~$var~replace~g" $file. Juste use a single-byte character in place of /.

Thanks to this other link I also know how to replace just the first occurrence of a pattern in a file (not in a line):

sed "0,/$var/s/$var/replacement/" filename or sed 0,/$var/{s/$var/replacement/} filename

But if I do: sed '0,~$var~s~$var~replacement~' filename (or anything else that begins with 0, then no slash), I've got an error: unknown command: '0'.

How could I combine the two? Maybe by using awk or perl or ... ?

  • 2
    You need to escape custom delmiters for non-replacements. i.e \~$var~s~$var~replacement~ – 123 Jul 20 '16 at 14:27
9

While:

sed "0,\~$var~s~$var~replacement~"

Can be used to change the regex delimiter, embedding variable expansions inside sed (or any other interpreter) code is a very unwise thing to do in the general case.

First, here, the delimiter is not the only character that needs to be escaped. All the regular expression operators need to as well.

But more importantly, and especially with GNU sed, that's a command injection vulnerability. If the content of $var is not under your control, it's just as bad as passing arbitrary data to eval.

Try for instance:

$ var='^~s/.*/uname/e;#'
$ echo | sed "0,\~$var~s~$var~replacement~"
Linux

The uname command was run, thankfully a harmless one... this time.

Non-GNU sed implementations can't run arbitrary commands, but can overwrite any file (with the w command), which is virtually as bad.

A more correct way is to escape the problematic characters in $var first:

NL='
'
case $var in
  (*"$NL"*)
    echo >&2 "Sorry, can't handle variables with newline characters"
    exit 1
esac
escaped_var=$(printf '%s\n' "$var" | sed 's:[][\/.^$*]:\\&:g')
# and then:
sed "0,/$escaped_var/s/$escaped_var/replacement/" < file

Another approach is to use perl:

var=$var perl -pe 's/\Q$ENV{var}\E/replacement/g && $n++ unless $n' < file

Note that we're not expanding the content of $var inside the code passed to perl (which would be another command injection vulnerability), but are letting perl expand its content as part of its regexp processing (and within \Q...\E which means regexp operators are not treated specially).

If $var contains newline characters, that may only match if there's only one at the end. Alternatively, one may pass the -0777 option so the input be processed as a single record instead of line-by-line.

1

Similar to Stephane's perl solution, awk can avoid the issue of protecting a 'dangerous' interpolated value with an alternate approach:

export pattern="anypattern" # or whatever shell quoting is needed
awk 'NR==1,$0~ENVIRON["pattern"] {sub(ENVIRON["pattern"],"replacement")} 1' input
# or gsub if you want multiple matches on the first line with any match

and if the data is (possibly) large and the pattern (possibly) costly, awk can also avoid unnecessary matching once one match has been found with a slightly different idiom:

awk '!x&&$0~ENVIRON["pattern"] {sub(ENVIRON["pattern"],"replacement");x=1} 1' input
# ditto

Reminder: awk (or sed) treats & in the replacement value specially; if you need an actual & prefix with \ (which usually must first be shell-quoted; here it is in '') and similarly double an actual \

  • Or simply (for your second case) awk '!x && sub(ENVIRON["pattern"],"replacement") {x=1};1' – Stéphane Chazelas Jul 22 '16 at 7:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.