Replacing text from a list of replacements. Added complication: backslashes

Question

I have a file A that contains pairs of strings, one per line:

\old1 \new1
\old2 \new2
.....

I would like to iterate over file A, and for each line perform the replacement (e.g. "\old1 -> \new1") globally in some file B. I had no trouble getting it to work without backslashes using sed or perl -pi -e using something like the following:

while read -r line
do
 set -- $line
 sed -i -e s/$1/$2/g target
done < replacements

However, I can't figure out how to make either sed or perl treat the backslashes verbatim in the replacement strings. Is there a clean solution for this?

Answer 1

You'll need to escape all characters that are special in regexps, not just backslashes but also [.*^$ and the s delimiter (for sed). In Perl, use the quotemeta function.

A further issue with your attempt is that when you run set -- $line, the shell performs its own expansion: it performs globbing in addition to word splitting, so if your line contains a* b* and there are files called a1 and a2 in the current directory then you'll be replacing a1 with a2. You need to turn off globbing with set -f in this approach.

Here's a solution that mangles the replacement list directly into a list of sed arguments. It assumes that there is no space character in the source and replacement texts, but anything other than a space and a newline should be treated correctly. The first replacement adds a \ before the characters that need protecting, and the second replacement turns each line from foo bar into -e s/foo/bar/g. Warning, untested.

set -f
sed_args=$(<replacement sed -e 's~[/.*[\\^$]~\\&~g' \
                            -e 's~^\([^ ]*\)  *\([^ ]*\).*~-e s/\1/\2/g~')
sed -i $sed_args target

In Perl, you'll have fewer issues with quoting if you just let Perl read the replacement file directly. Again, untested.

perl -i -pe 'BEGIN {
   open R, "<replacement" or die;
   while (<R>) {
       chomp;
       ($from, $to, @ignored) = split / +/;
       $s{$from} = $to;
   }
   close R;
   $regexp = join("|", map {quotemeta} keys %s);
}
s/($regexp)/$s{$1}/ego'

Answer 2

This is an attempt to escape the backslash using parameter expansion with pattern substitution.

$ set -- \\foo \\bar
$ echo $1
\foo
$ echo ${1/\\/\\\\}
\\foo
$ echo "This is \foo to me"
This is \foo to me
$ echo "This is \foo to me" | sed s/${1/\\/\\\\}/${2/\\/\\\\}/
This is \bar to me
$ 

Answer 3

For simple cases, there are simple solutions, so if you happen to have clean, plain, core words, without .?+*{}()[]\/ and maybe more fancy sed-stuff, you can transfer the list of pairs to a sed-command-file with sed:

sed -re 's,(^\\| \\|$),/,g;s/^/s/;s/$/g/' pairs.txt > pairs.sed
sed -f pairs.sed input > output

Answer 4

You might need to pre-process your list of substitutions to escape anything like the slashes that will have special meanings when put in a regex. First escape them, then use them to iterate over.

Depending on what function you are using to do the replace, sometimes there are flags you can add to treat strings literally. If you show off your partial solution maybe we can suggest just the right way to finish it.

Answer 5

This makes the same assumption about spaces as @Gilles answer does, but it eschews the while...read loop. It first backslash escapes any occurrence of any of sed's BRE metacharacters, then prints its current line number, then globally substitutes every pair of not-space characters it can find into a working sed substitution statement. Next in the pipeline a second sed transforms the first sed's output into something like:

[linenum]{
s/\1/\2/g;s/\1/\2/g;
}

...so the third sed can read its script on stdin and operate directly on file2 without any mucking around with a shell loop.

<file1 \
sed 's|[]\*/^$.[]|\\&|g;=
     s|\([^ ]*\) \([^ ]*\)|s/\1/\2/g;|g' |
sed 'N;s/\(\n\)\(.*\)/{\1\2\1}\1/'       |
sed -f - file2