66

I want to do:

cat update_via_sed.sh | sed 's/old_name/new_name/' > new_update_via_sed.sh

in my program.

But I want to use variables, e.g.

old_run='old_name_952'
new_run='old_name_953'

I have tried using them but the substitution doesn't happen (no error). I have tried:

cat update_via_sed.sh | sed 's/old_run/new_run/'
cat update_via_sed.sh | sed 's/$old_run/$new_run/'
cat update_via_sed.sh | sed 's/${old_run}/${new_run}/'
1

9 Answers 9

54

You could do:

sed "s/$old_run/$new_run/" < infile > outfile

But beware that $old_run would be taken as a regular expression and so any special characters that the variable contains, such as / or . would have to be escaped. Similarly, in $new_run, the & and \ characters would need to be treated specially and you would have to escape the / and newline characters in it.

If the content of $old_run or $new_run is not under your control, then it's critical to perform that escaping, or otherwise that code amounts to a code injection vulnerability.

8
  • 3
    I was using single quote in sed params, switched to double quotes and it worked file. Awesome.
    – Aakash
    Jul 28, 2016 at 8:30
  • it isn't that sed alone will not use regex but sed -E would ?
    – Kiwy
    Apr 24, 2018 at 10:04
  • @Kiwy, no. -e is to specify a sed expression, so you can pass more than one (as in sed -e exp1 -e exp2) or combinations of files and expressions (sed -f file -e exp). You may be confusing with -E which with some implementations, tells sed to use EREs instead of BREs. Apr 24, 2018 at 10:06
  • I mistyped, but OK, that explain why i have so many issues using sed without -E I only master the extended regex not the regular one. Thanks for the explanaiton.
    – Kiwy
    Apr 24, 2018 at 10:12
  • @Kiwy, see also the linked Q&A for more options supported by some sed implementations for yet more regular expression syntaxes. Apr 24, 2018 at 10:18
33

This worked:

cat update_via_sed.sh | sed 's/'"$old_run"'/'"$new_run"'/'

As I want to 'reuse' the same file I actually use this for anyone wishing a similar approach:

cat update_via_sed.sh | sed 's/'"$old_run"'/'"$new_run"'/' > new_update; mv -f new_update update_via_sed.sh

The above created a new file then deletes the current file than rename the new file to be the current file.

2
  • 5
    You don't need cat, mv, or extra '' unless you're going to have special characters in them. So it's sed -i s/"$old"/"$new"/ update_via_sed.sh. However be aware that this does not work for any value of old and new. e.g. it must not contain / etc., as $old is still a search and $new a replacement pattern where some characters have special meanings. Mar 25, 2013 at 16:25
  • 2
    sed -i "s/$old/$new/" is ok too (with the above precautions). sed usually considers the separator to be the first character after the s command - i.e. if your variables contain slashes / but not lets say at (@), you can use "s@$old@$new@".
    – peterph
    Mar 25, 2013 at 17:23
23

You can use variables in sed as long as it's in a double quote (not a single quote):

sed "s/$var/r_str/g" file_name >new_file

If you have a forward slash (/) in the variable then use different separator like below

sed "s|$var|r_str|g" file_name >new_file
3
  • 1
    +1 for mentioning needing to use double quotes. That was my problem. Nov 16, 2016 at 17:01
  • 1
    Exactly what I was looking for, including the use of | as in my case, the variables contains a path so it has / in it
    – yorch
    Nov 17, 2016 at 21:51
  • For some reasons the pipe | worked for me on MacOS 10.14, but other seperators like @ or # not. I had forward slashes in my env var too.
    – davidenke
    Oct 22, 2018 at 12:30
21

'in-place' sed (usng the -i flag) was the answer. Thanks to peterph.

sed -i "s@$old@$new@" file
2
  • 2
    You have the quotes in the wrong place. quotes must be around variables, not s@ (which is not special to the shell in any way). Best is to put everything inside quotes like sed -i "s@$old@$new@" file. Note that -i is not a standard sed option. Mar 26, 2013 at 14:20
  • how can I escape $ in this command. Like, I am going to change $xxx as a whole to the value of a variable $JAVA_HOME. I tried sed -i "s@\$xxx@$JAVA_HOME@" file, but error says no previous regular expression
    – hakunami
    Nov 6, 2014 at 8:07
3

man bash gives this about single quoting

Enclosing characters in single quotes preserves the literal value of each character within the quotes. A single quote may not occur between single quotes, even when preceded by a backslash.

Whatever you type on the command line, bash interprets it and then it sends the result to the program it is supposed to be sent to.In this case, if you use sed 's/$old_run/$new_run/', bash first sees the sed, it recognises it as an executable present in $PATH variable. The sed executable requires an input. Bash looks for the input and finds 's/$old_run/$new_run/'. Single quotes say bash not to interpret the content in them and pass them as they are. So, bash then passes them to sed. Sed gives an error because $ can occur only at the end of line.

Instead if we use double quotes, i.e., "s/$old_run/$new_run/", then bash sees this and interprets $old_run as a variable name and makes a substitution (this phase is called variable expansion). This is indeed what we required.

But, you have to be careful using double quotes because, they are interpreted first by bash and then given to sed. So, some symbols like ` must be escaped before using them.

2
$ echo $d1 | sed -i 's/zkserver1/'${d1}'/g' deva_file
sed: -e expression #1, char 26: unterminated `s' command 

In $d1 I am storing values

I cannot replace the value from the variable, so I tried the following command it is working:

echo $d1 | sed -i 's/zkserver1/'"${d1}"'/g' deva_file

missing double quotation mark in "${d1}", that was the error.

1

At risk of being flippant - have you considered perl.

Which - amongst other things - is quite good at doing 'sed style' operations, but also a more fully featured programming thingummy.

It looks like in your example, what you're actually trying to do is increment a number.

So how about:

#!/usr/bin/env perl
use strict;
use warnings 'all'; 

while ( <DATA> ) {
    s/old_name_(\d+)/"old_name_".($1+1)/e;
    print;
}

__DATA__
old_name_932

or as a one liner - sed style:

perl -pe 's/old_name_(\d+)/"old_name_".($1+1)/e'
-3

break the string

bad => linux see a string $old_run :

sed 's/$old_run/$new_run/'

good => linux see a variable $old_run:

sed 's/'$old_run'/'$new_run'/'
1
  • 2
    And if $old_run or $new_run contains spaces?
    – Kusalananda
    Jun 22, 2018 at 7:20
-4

You can also use Perl:

perl -pi -e ""s/$val1/$val2/g"" file
1
  • 1
    Consider variables that may have spaces in their values. The empty strings around the Perl expression ("") won't do anything.
    – Kusalananda
    Jun 22, 2018 at 7:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.