Regex alternation/or operator (foo|bar) in GNU or BSD Sed

Question

I cannot seem to make it work. GNU sed documentation says to escape the pipe, but that doesn't work, nor does using a straight pipe without the escape. Adding parens makes no difference.

$ echo 'cat
dog
pear
banana
cat
dog' | sed 's/cat|dog/Bear/g'
cat
dog
pear
banana
cat
dog

$ echo 'cat
dog
pear
banana
cat
dog' | sed 's/cat\|dog/Bear/g'
cat
dog
pear
banana
cat
dog

Answer 1

By default sed uses POSIX Basic Regular Expressions, which don't include the | alternation operator. You can switch it into using Extended Regular Expressions, which do include | alternation, with -E (or -r in some older versions of some implementations). You can use:

echo 'cat dog pear banana cat dog' | sed -E -e 's/cat|dog/Bear/g'

and it will work on compliant systems. (-e optionally marks the sed script itself - you can leave it out, it just guards against some kinds of mistake)

Portability to very old seds is complicated, but you can also switch to awk if you need it, which uses EREs everywhere.

Answer 2

This happens because (a|b) is an extended regular expression, not a Basic Regular Expression. Use the -E option to deal with this.

echo 'cat
dog
pear
banana
cat
dog'|sed -E 's/cat|dog/Bear/g'

From the sed man page:

 -E      Interpret regular expressions as extended (modern) regular
         expressions rather than basic regular expressions (BRE's).

Note that -r is another flag for the same thing, but -E is more portable and will even be in the next version of POSIX specifications.

Answer 3

The portable way to do this - and the more efficient way - is with addresses. You can do this:

printf %s\\n cat dog pear banana cat dog |
sed -e '/cat/!{/dog/!b' -e '}' -e 'c\
Bear'

In this way if the line does not contain the string cat and does not contain the string dog sed branches out of the script, autoprints its current line and pulls in the next to begin the next cycle. It therefore does not perform the next instruction - which in this example changes the entire line to read Bear but it could do anything.

It's probably worth noting also that any statement following the !b in that sed command can only match on a line containing either the string dog or cat - so you can perform further tests without any danger of matching a line that doesn't - which means you can now apply rules to only one or the other as well.

But that's next. Here's output from the above command:

###OUTPUT###
Bear
Bear
pear
banana
Bear
Bear

You can also portably implement a lookup table with backreferences.

printf %s\\n cat dog pear banana cat dog |
sed '1{x;s/^/ cat dog /;x;}
G;s/^\(.*\)\n.* \1 .*/Bear/;P;d'

It's a lot more work to setup for this simple example case, but it can make for much more flexible sed scripts in the long run.

In the first line I exchange hold space and pattern space then insert the string <space>cat<space>dog<space> into hold space before exchanging them back.

From then on and on every following line I Get hold space appended to pattern space, then check to see if all of the characters from the beginning of the line until the newline I just added at the end match a string surrounded by spaces after it. If so I replace the entire lot with Bear and if not there is no harm done because I next Print only up to the first occurring newline in pattern space then delete it all.

###OUTPUT###
Bear
Bear
pear
banana
Bear
Bear

And when I say flexible, I mean it. Here it is replacing cat with BrownBear and dog with BlackBear:

printf %s\\n cat dog pear banana cat dog |
sed '1{x;s/^/ 1cat Brown 2dog Black /;x;}
     G;s/^\(.*\)\n.* [0-9]\1 \([^ ]*\) .*/\2Bear/;P;d'

###OUTPUT###
BrownBear
BlackBear
pear
banana
BrownBear
BlackBear

You can of course expand a great deal on the contents of the lookup table - I picked up the idea from Greg Ubben's usenet emails on the subject when, in the 90's, he described how he constructed a crude calculator out of a single sed s/// statement.

Answer 4

Here's a technique that does not make use of any implementation specific options to sed (e.g. -E , -r). Instead of describing the pattern as a single regex cat|dog, we can simply run sed twice:

echo 'cat
dog
pear
banana
cat
dog' | sed -e 's/cat/Bear/g' -e 's/dog/Bear/g'

It's an obvious workaround really, but worth sharing. It naturally generalizes to more than two pattern strings, though a very long chain of sed's isn't too good looking.

I often use sed -i (note though that this is not implemented in all implementations) to make changes in files. Here, a long list of pattern strings may be incorporated nicely, as each temporary result is saved to the file:

for pattern in cat dog owl; do
    sed -i "s/${pattern}/Bear/g" myfile
done

Answer 5

this is a pretty old question, but in case someone wants to try, there is a fairly low effort way to do this in sed with sed files. Each option can be listed on a separate line, and sed will evaluate each one. It's a logical equivalent of or. For example, to remove lines that contain a certain code:

you can say : sed -E '/^\/\*!(40103|40101|40111).*\/;$/d'

or put this in your sed file:

/^\/\*!40103.*\/;$/d
/^\/\*!40101.*\/;$/d
/^\/\*!40111.*\/;$/d

Answer 6

While the title is referencing GNU or BSD sed, a moldy version of awk should always work. All awk implementations are expected to support EREs per the POSIX specification.

$ cat yearly.txt

2020
2021
2022
2023

$ awk '{sub(/2020|2021/,"old"); print}' yearly.txt

old
old
2022
2023

https://pubs.opengroup.org/onlinepubs/9699919799/utilities/awk.html

https://pubs.opengroup.org/onlinepubs/9699919799/utilities/sed.html