0

I'm having a problem with sed, when using its 's' (search and replace command) with its 'e' flag, as in

sed 's/.../.../e'

The problem is that the characters $, (, and ) in the standard input appear to confuse it.


I first ran into it here and have now boiled it down to this:

echo '$-' | sed 's/x//e'               # works when pattern is NOT matched
$-

echo '$-' | sed 's/-//e'               # fails when pattern is matched: gags on $
sh: $: command not found

echo '-' | sed 's/-//e'                # works when $ isn't present

echo '-(' | sed 's/-//e'               # Also fails when '(' or ')' are found
sh: -c: line 1: syntax error: unexpected end of file

echo '(' | sed 's/-//e'                # But works with this
(

echo '$-' | sed 's/-/echo xx/e'        # also fails
sh: xx: command not found

echo '$-' | sed 's/\(-\)/echo '\1'/e'  # also fails
sh: 1: command not found

Here's what the GNU manual says:

3.3 The s Command

...

The s command can be followed by zero or more of the following flags:

...

e

This command allows one to pipe input from a shell command into pattern space. If a substitution was made, the command that is found in pattern space is executed and pattern space is replaced with its output. A trailing newline is suppressed; results are undefined if the command to be executed contains a NUL character. This is a GNU sed extension.


bash --version
GNU bash, version 5.0.3(1)-release (x86_64-pc-linux-gnu)

sed --version
sed (GNU sed) 4.7
Packaged by Debian

What am I missing or doing wrong?

3
  • @Kamil Maciorowski, thanks, but no. What I'm trying to do is to pipe a long list of pathname lines thru a sed filter which operates on some parts of them, even calling a bash function to help rewrite them. Looks like I might have to figure out another way to do this, as it appears that the e flag doesn't exactly do what it says in the manual. It seems that the entire output is executed, not just the replacement part. Feb 23 '20 at 21:12
  • OK. For future reference: to avoid such misunderstandings it's good to state your expected output. E.g. I cannot tell what you expected from echo '$-' | sed 's/-//e'. Feb 23 '20 at 21:14
  • From your comments I deduce maybe you misinterpreted "pattern space". See this part of the manual: first, sed reads one line from the input stream, removes any trailing newline, and places it in the pattern space. Then commands are executed. With echo '$-' | sed 's/-/echo xx/e' the pattern space is not echo xx. The pattern space is $- at first, then s does its job and the pattern space becomes $echo xx. Feb 23 '20 at 21:27
3

Running

echo '$-' | sed 's/-//e'

removes the - from $-, leaving just $, and the result is run as a shell command, because of /e.

The result is the same is if you try to run $ from the command line:

sh: $: command not found

And similarly for all other lines.

2
  • Thanks, but in echo '$-' | sed 's/-/echo xx/e' I would have thought that the meaning of 'the command that is found in pattern space is executed' would be to execute echo xx. But I think I see what you mean: It is not executing just the echo xx, but rather the entire result, or something like $$(echo xx). But if so then it makes it very hard to predict what will be executed, and thus this appears to make it insecure in a way too. Feb 23 '20 at 21:05
  • 1
    Yes, that's why you seldom find /e in sed in practice (unlike a similar but different /e in Perl, see perlop).
    – choroba
    Feb 23 '20 at 22:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.