In using sed, I often create rather complicated and intricate regexes that I need to match twice in a file. Is there a way for me to save this regex and just reference it twice?

Maybe something that looks like this?

sed ' complicated_regex=/^(([a-f0-9]{32})+([a-zA-Z0-9=]{{$i}})?)+$/
' my_file

Update: An answer has presented the solution of using a bash variable. This doesn't work. Given a test.txt.

foo bar
bar foo

And the script


VALUE='foo \([a-z]\+\)'

sed 's/"${VALUE}"/foo happy \1/' test.txt

This should produce the output

foo happy bar
bar foo

But instead I get the error

sed: -e expression #1, char 24: invalid reference \1 on `s' command's RHS
  • If you use Perl, you can reference previous parts of the expression within the expression: perl -pe 's/^(([a-f0-9]{32})+([a-zA-Z0-9=]{{$i}})?)+:\1$/simple_output/' my_file – glenn jackman Sep 21 '11 at 20:10
  • 1
    When you find something too complicated for sed, don't hesitate to go for awk or perl. – Gilles 'SO- stop being evil' Sep 21 '11 at 23:32
  • 2
    @Cory: Your example works fine, if you get its syntax right. You have the "$variable" still enclosed in 'single-quotes' which hides it from the shell expansion you expect... it should be sed 's/'"${VALUE}"'/foo happy \1/' test.txt – Peter.O Sep 22 '11 at 4:32

You can use shell variables:

sed s/^"$complicated_regex":"$complicated_regex"\$/'simple-output'/ my_file

I'm not sure about what you meant by $i, but you might need to put it outside single quotes:

  • Of course this works only if the sed command is invoked from the shell, but there is a similar solution with almost all programming languages. (And I don't think it's possible to use variables within sed.) – Stéphane Gimenez Sep 21 '11 at 18:59
  • Hrm. Trying this, backreferences appear to be broken. s/$complicated_regex/\1/ gives an error saying that is an invalid reference. – Cory Klein Sep 21 '11 at 19:30
  • Ah, maybe my fault, I'm used to zsh variable substitutions. See updated answer. – Stéphane Gimenez Sep 21 '11 at 19:37
  • You'll have to remove the anchors from the variable and put them in the sed script: sed "s/^${complicated_regex}:${complicated_regex}\$/simple-output/" my_file – glenn jackman Sep 21 '11 at 20:09
  • Duh! Yes, I forgot to check that I was provided a valid regex concatenation :-) – Stéphane Gimenez Sep 21 '11 at 20:13

The easiest way to drop in a shell variable value to sed and not worry about how your backslash-escaping will need to change for the rest of your sed script, is to stuff everything into single quotes except the variable, and put that in double quotes.

All of the following code examples assume: VALUE='foo \([a-z]\+\)'

The following broken code fails because the variable VALUE isn't expanded:

sed 's/"${VALUE}"/foo happy \1/' test.txt

The following broken code fails because the backslash on \1 gets eaten by the shell (because it's in double quotes rather than single quotes) before sed ever sees it:

sed "s/${VALUE}/foo happy \1/" test.txt

The following code works as expected:

sed 's/'"${VALUE}"'/foo happy \1/' test.txt

Tho following code also works:

sed "s/${VALUE}/foo happy \\1/" test.txt

So does the following:

sed s/"${VALUE}"/foo\ happy\ \\1/ test.txt

But why get complicated? Single quotes around a sed script make everything much clearer, especially for the non-shell-scripting-gurus reading your code. My favored way is, again, to drop out of single quotes to double quotes just for the variable expansion and jump right back to single quotes:

sed 's/'"${VALUE}"'/foo happy \1/' test.txt

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.