60

I want to do:

cat update_via_sed.sh | sed 's/old_name/new_name/' > new_update_via_sed.sh

in my program.

But I want to use variables, e.g.

old_run='old_name_952'
new_run='old_name_953'

I have tried using them but the substitution doesn't happen (no error). I have tried:

cat update_via_sed.sh | sed 's/old_run/new_run/'
cat update_via_sed.sh | sed 's/$old_run/$new_run/'
cat update_via_sed.sh | sed 's/${old_run}/${new_run}/'

11 Answers 11

43

You could do:

sed "s/$old_run/$new_run/" < infile > outfile

But beware that $old_run would be taken as a regular expression and so any special characters that the variable contains, such as / or . would have to be escaped. Similarly, in $new_run, the & and \ characters would need to be treated specially and you would have to escape the / and newline characters in it.

If the content of $old_run or $new_run is not under your control, then it's critical to perform that escaping, or otherwise that code amounts to a code injection vulnerability.

  • 3
    I was using single quote in sed params, switched to double quotes and it worked file. Awesome. – Aakash Jul 28 '16 at 8:30
  • it isn't that sed alone will not use regex but sed -E would ? – Kiwy Apr 24 '18 at 10:04
  • @Kiwy, no. -e is to specify a sed expression, so you can pass more than one (as in sed -e exp1 -e exp2) or combinations of files and expressions (sed -f file -e exp). You may be confusing with -E which with some implementations, tells sed to use EREs instead of BREs. – Stéphane Chazelas Apr 24 '18 at 10:06
  • I mistyped, but OK, that explain why i have so many issues using sed without -E I only master the extended regex not the regular one. Thanks for the explanaiton. – Kiwy Apr 24 '18 at 10:12
  • @Kiwy, see also the linked Q&A for more options supported by some sed implementations for yet more regular expression syntaxes. – Stéphane Chazelas Apr 24 '18 at 10:18
31

This worked:

cat update_via_sed.sh | sed 's/'"$old_run"'/'"$new_run"'/'

As I want to 'reuse' the same file I actually use this for anyone wishing a similar approach:

cat update_via_sed.sh | sed 's/'"$old_run"'/'"$new_run"'/' > new_update; mv -f new_update update_via_sed.sh

The above created a new file then deletes the current file than rename the new file to be the current file.

  • 4
    You don't need cat, mv, or extra '' unless you're going to have special characters in them. So it's sed -i s/"$old"/"$new"/ update_via_sed.sh. However be aware that this does not work for any value of old and new. e.g. it must not contain / etc., as $old is still a search and $new a replacement pattern where some characters have special meanings. – frostschutz Mar 25 '13 at 16:25
  • 1
    sed -i "s/$old/$new/" is ok too (with the above precautions). sed usually considers the separator to be the first character after the s command - i.e. if your variables contain slashes / but not lets say at (@), you can use "s@$old@$new@". – peterph Mar 25 '13 at 17:23
22

You can use variables in sed as long as it's in a double quote (not a single quote):

sed "s/$var/r_str/g" file_name >new_file

If you have a forward slash (/) in the variable then use different separator like below

sed "s|$var|r_str|g" file_name >new_file
  • 1
    +1 for mentioning needing to use double quotes. That was my problem. – speckledcarp Nov 16 '16 at 17:01
  • 1
    Exactly what I was looking for, including the use of | as in my case, the variables contains a path so it has / in it – yorch Nov 17 '16 at 21:51
  • For some reasons the pipe | worked for me on MacOS 10.14, but other seperators like @ or # not. I had forward slashes in my env var too. – davidenke Oct 22 '18 at 12:30
21

'in-place' sed (usng the -i flag) was the answer. Thanks to peterph.

sed -i "s@$old@$new@" file
  • 2
    You have the quotes in the wrong place. quotes must be around variables, not s@ (which is not special to the shell in any way). Best is to put everything inside quotes like sed -i "s@$old@$new@" file. Note that -i is not a standard sed option. – Stéphane Chazelas Mar 26 '13 at 14:20
  • how can I escape $ in this command. Like, I am going to change $xxx as a whole to the value of a variable $JAVA_HOME. I tried sed -i "s@\$xxx@$JAVA_HOME@" file, but error says no previous regular expression – hakunami Nov 6 '14 at 8:07
3

man bash gives this about single quoting

Enclosing characters in single quotes preserves the literal value of each character within the quotes. A single quote may not occur between single quotes, even when preceded by a backslash.

Whatever you type on the command line, bash interprets it and then it sends the result to the program it is supposed to be sent to.In this case, if you use sed 's/$old_run/$new_run/', bash first sees the sed, it recognises it as an executable present in $PATH variable. The sed executable requires an input. Bash looks for the input and finds 's/$old_run/$new_run/'. Single quotes say bash not to interpret the content in them and pass them as they are. So, bash then passes them to sed. Sed gives an error because $ can occur only at the end of line.

Instead if we use double quotes, i.e., "s/$old_run/$new_run/", then bash sees this and interprets $old_run as a variable name and makes a substitution (this phase is called variable expansion). This is indeed what we required.

But, you have to be careful using double quotes because, they are interpreted first by bash and then given to sed. So, some symbols like ` must be escaped before using them.

1

At risk of being flippant - have you considered perl.

Which - amongst other things - is quite good at doing 'sed style' operations, but also a more fully featured programming thingummy.

It looks like in your example, what you're actually trying to do is increment a number.

So how about:

#!/usr/bin/env perl
use strict;
use warnings 'all'; 

while ( <DATA> ) {
    s/old_name_(\d+)/"old_name_".($1+1)/e;
    print;
}

__DATA__
old_name_932

or as a one liner - sed style:

perl -pe 's/old_name_(\d+)/"old_name_".($1+1)/e'
1
$ echo $d1 | sed -i 's/zkserver1/'${d1}'/g' deva_file
sed: -e expression #1, char 26: unterminated `s' command 

In $d1 I am storing values

I cannot replace the value from the variable, so I tried the following command it is working

echo $d1 | sed -i 's/zkserver1/'"${d1}"'/g' deva_file

missing double quotation mark in "${d1}", that was the error

-2

Assumed $old_run and $new_run are already set:

cat update_via_sed.sh | eval $(print "sed 's/$old_run/$new_run/g'")

This shows the change on screen. You can redirect output to file if needed.

-2

For using a variable in sed , use double quotes "" , to enclose the variable , in the pattern that you are using. For eg: a="Hi" If you want to append variable 'a' to a pattern, you can use the below: sed -n "1,/pattern"${a}"pattern/p" filename

  • 2
    The expansion of $a is unquoted here, which means that any spaces in its value will invoke word splitting in the shell. – Kusalananda Jun 22 '18 at 7:47
-2

You can also use Perl:

perl -pi -e ""s/$val1/$val2/g"" file
  • Consider variables that may have spaces in their values. The empty strings around the Perl expression ("") won't do anything. – Kusalananda Jun 22 '18 at 7:25
-2

break the string

bad => linux see a string $old_run :

sed 's/$old_run/$new_run/'

good => linux see a variable $old_run:

sed 's/'$old_run'/'$new_run'/'
  • 1
    And if $old_run or $new_run contains spaces? – Kusalananda Jun 22 '18 at 7:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.