How to search and replace strings matching a replacement list for multiple files

Question

I have a file, say map.txt, containing a list of search strings and corresponding replacements:

search -> replacement
bigBone -> bb
fishMarket -> fm
dogCollar -> dc
...

I need to perform search and replacement of all strings matching the above for all files recursively in a folder except symbolic links. I know how to do it one at a time like this:

$ find /some/folder -type f -exec sed -i 's/old_text/new_text/g' {} \;

How can I perform this on a massive scale using the above mapping? I have read this question, but do not quite get it.

Answer 1

Answer

If you are not concerned about speed (this is a one time task), then maybe you could try this:

cat map.txt | while read line; do
neww=${line##* };
oldw=${line%% *};
find /some/folder -type f -exec sed -i "s/$oldw/$neww/g" {} \;
done

Not optimal, I know... :-P

PS: check in a test folder to see if it works!

Explanation

Basically:

1. Cat file map.txt.
2. Read each line and get the word to be replaced $oldw and the replacement $neww.
3. For each pair, execute the find command you were already using (notice the double quotes this time in order to allow variable substitution).

About parameter expansion

In order to set the variables $oldw and $neww we have to get the first and last word of each line. For doing so, we are using parameter expansion (pure Bash implementation), although we could have used other ways to get the first and last word of the string (i.e. cut or awk).

• ${line##* } : from variable line, remove largest prefix (double #) pattern, where pattern is any characters (*) followed by a space (). So we get the last word in line.
• ${line%% *} : from variable line, remove largest suffix (double %) pattern, where pattern is a space () followed by any characters (*). So we get the first word in line.

Words were separated by a space in this case, but we could have used any separator.

Answer 2

Sometimes i need to search and replace terms in configuration files.

I wrote a script available on github sandr - search and replace which permits to create/use a map and perform replacement in files.

An example of use :

$ cat file
Voyez ce jeu exquis wallon, de graphie en kit mais bref. Portez ce vieux whisky au juge blond qui fume sur son île intérieure, à côté de l'alcôve ovoïde, où les bûches se consument dans l'âtre, ce qui lui permet de penser à la cænogenèse de l'être dont il est question dans la cause ambiguë entendue à Moÿ, dans un capharnaüm qui, pense-t-il, diminue çà et là la qualité de son œuvre. Prouvez, beau j
$ cat map.txt 
wallon => WALLON
se => SE
penser => PENSER
beau => BEAU
$ cat file | ./sandr -a map.txt 
Voyez ce jeu exquis WALLON, de graphie en kit mais bref. Portez ce vieux whisky au juge blond qui fume sur son île intérieure, à côté de l'alcôve ovoïde, où les bûches SE consument dans l'âtre, ce qui lui permet de PENSER à la cænogenèSE de l'être dont il est question dans la cauSE ambiguë entendue à Moÿ, dans un capharnaüm qui, penSE-t-il, diminue çà et là la qualité de son œuvre. Prouvez, BEAU j