Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

In programing we often see the use of Regular Expressions.

One of the most common forms is:

newText = text.replace( /regex/, 'replacementString' )

If stdin is text and stdout is newText, what would the bash equivalent be to the above code?

share|improve this question
add comment

4 Answers 4

up vote 5 down vote accepted

For simple uses you can do it like this:

newText=${text/SEARCH/replacement}

as it is described here, but for more complex expressions sed is the way to go as alex described earlier.

share|improve this answer
add comment

The most straightforward answer is sed's s command. You need to convert the regexp syntax to Basic regular expressions, and the substitution will be applied successively on each line. You can use \1 through \9 to refer to parenthesized groups in the original string. Add the g modifier to replace all occurrences; otherwise only the first occurrence is replaced.

sed -e 's/basic regexp/replacement string/g'

A more flexible utility is awk. It too processes its input line by line by default, but you can change the record separator with -vRS=… (standard awk wants a single character, or an empty value to mean two or more newlines; Gawk (the GNU version) accepts a regular expression). The sub function performs a single replacement, and gsub replaces all occurences. The replacement string is interpreted literally except for \ and &; if you want to refer to parenthesized groups, you can use the match and subtring functions.

awk '{gsub(/regexp/, "replacement string")}'

Bash has built-in support for regexp matching: [[ text =~ regexp ]]. You can construct a replacement text using matched substrings stored in the BASH_REMATCH array. Use read or cat to obtain the input and printf to emit the output. The following pseudo-code performs multiple replacements (warning, untested; the code is supposed to perform multiple replacement from left to right as is usual, I hope I got it right).

# The end marker must not have a prefix that is a suffix of a match of the regexp,
# and must not start or end with a newline
end_marker='EOF'
text=$(cat; echo "$end_marker")
while [[ $text =~ regexp(.*)$ ]]; then
    printf %s%s "${text%"$BASH_REMATCH[0]"}" "replacement string"
    text=$BASH_REMATCH[$#BASH_REMATCH]
  fi
done
printf %s "${text%"$end_marker"}"

(A few words of explanation: the end marker is to avoid trailing newlines being stripped by the command substitution. ${text%"$BASH_REMATCH[0]"} extracts the part of the text that came before the match. Note that we can't use ^(.*) at the beginning of the regexp or we'd get the last match instead of the first. After the match, we iterate over the suffix. Finally we print the unmatched leftover, minus the end marker.)

If you're satisfied with wildcard matching and limited replacement text abilities, bash also has ${variable/pattern/replacement}. Double the first slash to replace all occurrences. Patterns do have the power of regular expressions (but with an unusual syntax) if the extglob option is set.

share|improve this answer
add comment
  • man sed
  • sed s/regex/replacementString/g
share|improve this answer
add comment

you can use tools like sed and awk, but in my opinion they are very antiquated and only useful for narrowly defined tasks.

a better bet is to redirect STDIN to a perl one-liner or script. perl's regex support is so good that most other languages now support some compatibility with them. there are even a2p and s2p tools to transform sed and awk directly into perl. using perl allows you to use the entirety of CPAN to help you solve your problems.

and if you don't like perl, you can use python in a similar capacity.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.