I am using bash and trying to employ sed command for matching a string and replace it with another between two files.

Objective: To replace all strings[sp_*] by matching them from another file containing [sp_* Var_Names]. Please be noted that: 1. The order is synchronized but not consecutive in both files, so cannot use paste. 2. Functionality is similar to primeKey operations in mysql.

File 1

|     +--sp_O18983_
|  +--| (52)
|  |  |  +--sp_Q9BDS6_
|  |  +--| (26)
|  |     |  +--sp_O19024_
|  |     +--| (29)
|  |        +--sp_Q9XT45_



Purpose: To sed -ie 's/O00574/CXCR6_HUMAN/g' File1

inline Bash script:

cat File2 | while read id; do upID=`echo $id | cut -d " " -f1`; upName=`echo $id | cut -d " " -f2`; sed -ie 's/sp_$upID/$upName/g' File1; done



cat File2 | while read id;
    upID=`echo $id | cut -d " " -f1`
    upName=`echo $id | cut -d " " -f2`

    sed -ie 's/sp_$upID/$upName/g' File1

Problem: The sed command does not work in the loop. No change is observed in File1, at all. If I echo the sed command from the script and then run it in the terminal it works as expected. I cannot figure out what could be the problem.

Thank you for your valuable comments and solution.

  • 3
    You are using ' ' quotes in your sed statement which prevents variable expansion. Use " " quotes instead. Also you can make it simpler with "while read upID upName" and not need the two cut statements. Jun 2, 2016 at 15:38
  • Dear Stephen, Thanks for the error and added information. It would be very useful in solving many other cases. Thanks :) Somehow, I was saturated to see the single quotes. ;P Jun 2, 2016 at 16:06
  • For the "while upId upName" the upName overwrites the upId everytime. Jun 2, 2016 at 16:08

1 Answer 1


generate a sed script from your index file (File2) instead of a loop then run that script against your File1.. It will be MUCH faster :).

 awk '{ print "s/sp_"$1"/"$2"/g"}' File2.txt > tranform.sed

then do:

 sed -i -f transform.sed File1.txt 

so your entire script could be:

awk '{ print "s/sp_"$1"/"$2"/g"}' File2.txt > transform.sed
sed -f transform.sed File1.txt

## if you want to remove your transformation file
rm transform.sed
  • 1
    Nice, but if this is scripted you shouldn't use a fixed temp file name. Since the Original Poster specified bash, I'd opt for process substitution instead: sed -i -f <(awk '{print "s/sp_"$1"/"$2"/g"}' File2.txt) File1.txt
    – Wildcard
    Jun 2, 2016 at 23:36
  • So, you use the powerful awk to generate an input file for the less powerful sed, instead of using awk directly? Jun 3, 2016 at 7:42
  • The real question is does it work? :) There is always ten different ways to do something. awk is neither better or worse than sed. They are tools to get something done. Can you do it is awk in two easy to read lines?
    – Rob
    Jun 3, 2016 at 10:45
  • Thanks everyone for the response. As pointed out in the comments by +Stephen Harris, the problem was with the single quoting 's///g'. With double quotes it works perfectly fine. If I try the awk + sed it takes longer than the doing while directly on the file and sed -i. Jun 3, 2016 at 12:03

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .