I have a file A that contains pairs of strings, one per line:

\old1 \new1
\old2 \new2

I would like to iterate over file A, and for each line perform the replacement (e.g. "\old1 -> \new1") globally in some file B. I had no trouble getting it to work without backslashes using sed or perl -pi -e using something like the following:

while read -r line
 set -- $line
 sed -i -e s/$1/$2/g target
done < replacements

However, I can't figure out how to make either sed or perl treat the backslashes verbatim in the replacement strings. Is there a clean solution for this?

5 Answers 5


You'll need to escape all characters that are special in regexps, not just backslashes but also [.*^$ and the s delimiter (for sed). In Perl, use the quotemeta function.

A further issue with your attempt is that when you run set -- $line, the shell performs its own expansion: it performs globbing in addition to word splitting, so if your line contains a* b* and there are files called a1 and a2 in the current directory then you'll be replacing a1 with a2. You need to turn off globbing with set -f in this approach.

Here's a solution that mangles the replacement list directly into a list of sed arguments. It assumes that there is no space character in the source and replacement texts, but anything other than a space and a newline should be treated correctly. The first replacement adds a \ before the characters that need protecting, and the second replacement turns each line from foo bar into -e s/foo/bar/g. Warning, untested.

set -f
sed_args=$(<replacement sed -e 's~[/.*[\\^$]~\\&~g' \
                            -e 's~^\([^ ]*\)  *\([^ ]*\).*~-e s/\1/\2/g~')
sed -i $sed_args target

In Perl, you'll have fewer issues with quoting if you just let Perl read the replacement file directly. Again, untested.

perl -i -pe 'BEGIN {
   open R, "<replacement" or die;
   while (<R>) {
       ($from, $to, @ignored) = split / +/;
       $s{$from} = $to;
   close R;
   $regexp = join("|", map {quotemeta} keys %s);
  • Perl snippet works great; thanks for the tip about quotemeta. I'm still a little surprised that running verbatim string replacements from a list didn't have some simple canned solution, but I'm happy with the Perl code. Commented Apr 17, 2011 at 0:37
  • There's a pending edit suggestion about adding chomp Commented Apr 17, 2011 at 5:37
  • @Michael: Suggested edits leave a notification on every page, so I go through them anyway (the rare times when you don't reach them first). Commented Apr 17, 2011 at 12:01

For simple cases, there are simple solutions, so if you happen to have clean, plain, core words, without .?+*{}()[]\/ and maybe more fancy sed-stuff, you can transfer the list of pairs to a sed-command-file with sed:

sed -re 's,(^\\| \\|$),/,g;s/^/s/;s/$/g/' pairs.txt > pairs.sed
sed -f pairs.sed input > output

This is an attempt to escape the backslash using parameter expansion with pattern substitution.

$ set -- \\foo \\bar
$ echo $1
$ echo ${1/\\/\\\\}
$ echo "This is \foo to me"
This is \foo to me
$ echo "This is \foo to me" | sed s/${1/\\/\\\\}/${2/\\/\\\\}/
This is \bar to me

This makes the same assumption about spaces as @Gilles answer does, but it eschews the while...read loop. It first backslash escapes any occurrence of any of sed's BRE metacharacters, then prints its current line number, then globally substitutes every pair of not-space characters it can find into a working sed substitution statement. Next in the pipeline a second sed transforms the first sed's output into something like:


...so the third sed can read its script on stdin and operate directly on file2 without any mucking around with a shell loop.

<file1 \
sed 's|[]\*/^$.[]|\\&|g;=
     s|\([^ ]*\) \([^ ]*\)|s/\1/\2/g;|g' |
sed 'N;s/\(\n\)\(.*\)/{\1\2\1}\1/'       |
sed -f - file2

You might need to pre-process your list of substitutions to escape anything like the slashes that will have special meanings when put in a regex. First escape them, then use them to iterate over.

Depending on what function you are using to do the replace, sometimes there are flags you can add to treat strings literally. If you show off your partial solution maybe we can suggest just the right way to finish it.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .