Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

In using sed, I often create rather complicated and intricate regexes that I need to match twice in a file. Is there a way for me to save this regex and just reference it twice?

Maybe something that looks like this?

sed ' complicated_regex=/^(([a-f0-9]{32})+([a-zA-Z0-9=]{{$i}})?)+$/
s/complicated_regex:complicated_regex/simple-output/
' my_file

Update: An answer has presented the solution of using a bash variable. This doesn't work. Given a test.txt.

#test.txt
foo bar
bar foo

And the script

#!/bin/bash

VALUE='foo \([a-z]\+\)'

sed 's/"${VALUE}"/foo happy \1/' test.txt

This should produce the output

foo happy bar
bar foo

But instead I get the error

sed: -e expression #1, char 24: invalid reference \1 on `s' command's RHS
share|improve this question
    
If you use Perl, you can reference previous parts of the expression within the expression: perl -pe 's/^(([a-f0-9]{32})+([a-zA-Z0-9=]{{$i}})?)+:\1$/simple_output/' my_file – glenn jackman Sep 21 '11 at 20:10
1  
When you find something too complicated for sed, don't hesitate to go for awk or perl. – Gilles Sep 21 '11 at 23:32
2  
@Cory: Your example works fine, if you get its syntax right. You have the "$variable" still enclosed in 'single-quotes' which hides it from the shell expansion you expect... it should be sed 's/'"${VALUE}"'/foo happy \1/' test.txt – Peter.O Sep 22 '11 at 4:32
up vote 5 down vote accepted

You can use shell variables:

complicated_regex='(([a-f0-9]{32})+([a-zA-Z0-9=]{{$i}})?)+'
sed s/^"$complicated_regex":"$complicated_regex"\$/'simple-output'/ my_file

I'm not sure about what you meant by $i, but you might need to put it outside single quotes:

complicated_regex='(([a-f0-9]{32})+([a-zA-Z0-9=]{{'"$i"'}})?)+'
share|improve this answer
    
Of course this works only if the sed command is invoked from the shell, but there is a similar solution with almost all programming languages. (And I don't think it's possible to use variables within sed.) – Stéphane Gimenez Sep 21 '11 at 18:59
    
Hrm. Trying this, backreferences appear to be broken. s/$complicated_regex/\1/ gives an error saying that is an invalid reference. – Cory Klein Sep 21 '11 at 19:30
    
Ah, maybe my fault, I'm used to zsh variable substitutions. See updated answer. – Stéphane Gimenez Sep 21 '11 at 19:37
    
You'll have to remove the anchors from the variable and put them in the sed script: sed "s/^${complicated_regex}:${complicated_regex}\$/simple-output/" my_file – glenn jackman Sep 21 '11 at 20:09
    
Duh! Yes, I forgot to check that I was provided a valid regex concatenation :-) – Stéphane Gimenez Sep 21 '11 at 20:13

The easiest way to drop in a shell variable value to sed and not worry about how your backslash-escaping will need to change for the rest of your sed script, is to stuff everything into single quotes except the variable, and put that in double quotes.

All of the following code examples assume: VALUE='foo \([a-z]\+\)'

The following broken code fails because the variable VALUE isn't expanded:

sed 's/"${VALUE}"/foo happy \1/' test.txt

The following broken code fails because the backslash on \1 gets eaten by the shell (because it's in double quotes rather than single quotes) before sed ever sees it:

sed "s/${VALUE}/foo happy \1/" test.txt

The following code works as expected:

sed 's/'"${VALUE}"'/foo happy \1/' test.txt

Tho following code also works:

sed "s/${VALUE}/foo happy \\1/" test.txt

So does the following:

sed s/"${VALUE}"/foo\ happy\ \\1/ test.txt

But why get complicated? Single quotes around a sed script make everything much clearer, especially for the non-shell-scripting-gurus reading your code. My favored way is, again, to drop out of single quotes to double quotes just for the variable expansion and jump right back to single quotes:

sed 's/'"${VALUE}"'/foo happy \1/' test.txt
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.