4

This command:

echo '$$foo=bar' | sed -E "s/(\$\$foo=).*/\1$(echo hello)/"

outputs:

$$foo=bar

If I change the sed's quotes to single and enclose the command in double quotes:

echo '$$foo=bar' | sed -E 's/(\$\$foo=).*/\1'"$(echo hello)"'/'

it outputs the desired result:

$$foo=hello

So I guess the problem lies in the extended regular expressions and quotes, unless I'm missing something obvious.

  1. Is concatenating single and double quotes in sed command substitution good practice (sed 's/foo/'"$(command)"'/')?

  2. How can I escape double dollar signs in sed with extended regular expressions?

6
  • First, you can just surround $$foo=bar with singles quotes as in echo '$$foo=bar'. That will escape the dollar signs. Second, what is the desired output? First you say that $$foo=hello is the desired result but then you state the that substitution goes fine when it outputs $$hello=bar. Which one do you want? Aug 27, 2019 at 22:04
  • @NasirRiley but then you state the that substitution goes fine when it outputs $$hello=bar. It's an example of a substitution with double dollar signs that works. Aug 27, 2019 at 22:16
  • First, you can just surround $$foo=bar with singles quotes as in echo '$$foo=bar'. You are right. Aug 27, 2019 at 22:24
  • What is your desired output? What makes the last one "fine"? You are confusing double quotes with command substitution ($(foo)). What are you actually trying to do here?
    – terdon
    Aug 27, 2019 at 22:32
  • @terdon Sorry if I haven't been clear, but Kusalananda's answer covers my question. I hope reading it will clarify things. Aug 27, 2019 at 22:47

1 Answer 1

5

POSIX has this to say about the difference in how $ is interpreted in basic and extended regular expressions:

Basic regular expressions (BREs):

A <dollar-sign> ( $ ) shall be an anchor when used as the last character of an entire BRE. The implementation may treat a <dollar-sign> as an anchor when used as the last character of a subexpression. The <dollar-sign> shall anchor the expression (or optionally subexpression) to the end of the string being matched; the <dollar-sign> can be said to match the end-of-string following the last character.

Extended regular expressions (EREs):

A <dollar-sign> ( $ ) outside a bracket expression shall anchor the expression or subexpression it ends to the end of a string; such an expression or subexpression can match only a sequence ending at the last character of a string. For example, the EREs ef$ and (ef$) match ef in the string abcdef, but fail to match in the string cdefab, and the ERE e$f is valid, but can never match because the f prevents the expression e$ from matching ending at the last character.

Conclusion: In a BRE, the $ character matches itself unless it's the last character of the expression or sub-expression (in which case it anchors the (sub-)expression to the end of the line). In an ERE, the $ character always anchors to the end of the line.

When you use

sed -E "s/(\$\$foo=).*/\1$(echo hello)/"

your ERE (since you use -E) is ($$foo=).* and this expression will never match (the POSIX text above contains the example e$f which is similar).

Your command

sed "s/\$\$foo/\$\$hello/"

uses the BRE $$foo which will match the literal string $$foo since the $ characters are not at the end of the expression.

To match a single $ character in an extended regular expression, use \$ or [$]. To escape that for the shell in a double quoted string, use \\\$ (an escaped backslash followed by an escaped dollar sign) or [\$], i.e.,

sed -E "s/(\\\$\\\$foo=).*/\1$(echo hello)/"

or

sed -E "s/([\$][\$]foo=).*/\1$(echo hello)/"

(The backslash in \1 does not need escaping since backslashes only act as an escape character in a double quoted string if followed by a dollar sign, backtick, double quote, another backslash, or a newline; so the \1 is literal, not an escaped 1; reference here).

Short answers:

  1. You either single quote the bits that need single quoting (and concatenate that with a double quoted string containing your shell expansions), or you escape what needs to be escaped in a single double quoted string. This is a matter of taste. I'd be more concerned with using a command substitution in the expression, as that is a code injection vulnerability unless you have full control over the string that is inserted.

  2. \\\$ or [\$] in a double quoted string. \$ or [$] in a single quoted string.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .