Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have a file A that contains pairs of strings, one per line:

\old1 \new1
\old2 \new2
.....

I would like to iterate over file A, and for each line perform the replacement (e.g. "\old1 -> \new1") globally in some file B. I had no trouble getting it to work without backslashes using sed or perl -pi -e using something like the following:

while read -r line
do
 set -- $line
 sed -i -e s/$1/$2/g target
done < replacements

However, I can't figure out how to make either sed or perl treat the backslashes verbatim in the replacement strings. Is there a clean solution for this?

share|improve this question

4 Answers 4

up vote 2 down vote accepted

You'll need to escape all characters that are special in regexps, not just backslashes but also [.*^$ and the s delimiter (for sed). In Perl, use the quotemeta function.

A further issue with your attempt is that when you run set -- $line, the shell performs its own expansion: it performs globbing in addition to word splitting, so if your line contains a* b* and there are files called a1 and a2 in the current directory then you'll be replacing a1 with a2. You need to turn off globbing with set -f in this approach.

Here's a solution that mangles the replacement list directly into a list of sed arguments. It assumes that there is no space character in the source and replacement texts, but anything other than a space and a newline should be treated correctly. The first replacement adds a \ before the characters that need protecting, and the second replacement turns each line from foo bar into -e s/foo/bar/g. Warning, untested.

set -f
sed_args=$(<replacement sed -e 's~[/.*[\\^$]~\\&~g' \
                            -e 's~^\([^ ]*\)  *\([^ ]*\).*~-e s/\1/\2/g~')
sed -i $sed_args target

In Perl, you'll have fewer issues with quoting if you just let Perl read the replacement file directly. Again, untested.

perl -i -pe 'BEGIN {
   open R, "<replacement" or die;
   while (<R>) {
       chomp;
       ($from, $to, @ignored) = split / +/;
       $s{$from} = $to;
   }
   close R;
}
$regexp = join("|", map {quotemeta} keys %s);
s/($regexp)/$s{$1}/ego'
share|improve this answer
    
Perl snippet works great; thanks for the tip about quotemeta. I'm still a little surprised that running verbatim string replacements from a list didn't have some simple canned solution, but I'm happy with the Perl code. –  Leo Alekseyev Apr 17 '11 at 0:37
    
There's a pending edit suggestion about adding chomp –  Michael Mrozek Apr 17 '11 at 5:37
    
@Michael: Suggested edits leave a notification on every page, so I go through them anyway (the rare times when you don't reach them first). –  Gilles Apr 17 '11 at 12:01

This is an attempt to escape the backslash using parameter expansion with pattern substitution.

$ set -- \\foo \\bar
$ echo $1
\foo
$ echo ${1/\\/\\\\}
\\foo
$ echo "This is \foo to me"
This is \foo to me
$ echo "This is \foo to me" | sed s/${1/\\/\\\\}/${2/\\/\\\\}/
This is \bar to me
$ 
share|improve this answer

For simple cases, there are simple solutions, so if you happen to have clean, plain, core words, without .?+*{}()[]\/ and maybe more fancy sed-stuff, you can transfer the list of pairs to a sed-command-file with sed:

sed -re 's,(^\\| \\|$),/,g;s/^/s/;s/$/g/' pairs.txt > pairs.sed
sed -f pairs.sed input > output
share|improve this answer

You might need to pre-process your list of substitutions to escape anything like the slashes that will have special meanings when put in a regex. First escape them, then use them to iterate over.

Depending on what function you are using to do the replace, sometimes there are flags you can add to treat strings literally. If you show off your partial solution maybe we can suggest just the right way to finish it.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.