0

Below sed command helps add text Hello to the end of each line of the input

<complex_query> | sed "s,$,Hello,"
Output:
myvar1: Hello
myvar2: Hello

I now wish to have the output of grep wow data.txt | cut -d: -f2 in place of Hello

How is that possible?

Below is something i tried but does not work.

<complex_query> | sed "s,$,(grep wow data.txt  | cut -d: -f2),"

Expected Desired output:

myvar1: <output of grep wow data.txt  | cut -d: -f2 command>
myvar2: <output of grep wow data.txt  | cut -d: -f2 command>
4
  • Does sed "s,$,$(grep wow data.txt | cut -d: -f2)," do what you want? It's not clear if that just generates one line you want to put at the end of every line of output, or you want it to use different lines of output for different input lines from <complex_query>.
    – frabjous
    Apr 25 at 2:40
  • My requirement is -> it generates one line I want to put at the end of every line of output @frabjous
    – Ashar
    Apr 25 at 2:41
  • @frabjous this works -> sed "s,$,$(grep wow data.txt | cut -d: -f2)," please post it as an answer
    – Ashar
    Apr 25 at 2:49
  • Feel free to accept Stéphane's much more thorough answer. (Though also feel free to upvote my comment.)
    – frabjous
    Apr 25 at 12:21

1 Answer 1

4

To use the output of a command (minus trailing newline characters) in an argument of another command, you'd use command substitution, a feature introduced by the Bourne shell in the late 70s.

In the Bourne shell, the syntax was `cmd...`. So was it in csh.

In modern sh and sh-like shells such as ksh (where it comes from)¹, zsh, bash, ash-based shells, the syntax is $(...) instead (though they do also support the Bourne syntax for backward compatibility).

In rc-like shells, the syntax is `{cmd...}

In the fish shell, it's (cmd ...) though that can't be used inside double quotes. Since version 3.4.0, fish also supports $(cmd...) which can be used inside double quotes.

Command substitution generally splits the output of the command in ways that vary between shells resulting in several arguments passed to the commands, some (including most Bourne-like shells) even perform globbing on the resulting words (!). That can usually be prevented by placing the command substitution inside double quotes (not in rc-like shells² nor in fish < 3.4.0³).

So here:

  • POSIX-like shells or fish 3.4.0+:

    <complex_query> | sed "s,\$,$(grep wow data.txt | cut -d: -f2),"
    
  • rc-like shells:

    nl = '
    '
    <complex_query> | sed 's,$,'``($nl){grep wow data.txt | cut -d: -f2},
    
  • csh/tcsh:

    <complex_query> | sed "s,\$,`grep wow data.txt | cut -d: -f2`,"
    
  • fish < 3.4.0

    <complex_query> | sed 's,$,'(grep wow data.txt | cut -d: -f2 | string collect),
    

Now, beware the & and \ characters are special in the replacement part of sed's s command, so would have to be escaped with \ along with , (the separator used here) and newline if present there (if newline can be present, beware of splitting in csh/rc as noted above).

See How to ensure that string interpolated into `sed` substitution escapes all metachars about that.

Another approach is to use perl instead of sed to not have to worry about special characters:

<complex_query> | REPL="$(cmd...)" perl -lpe '$_ .= $ENV{REPL}'

(here assuming a POSIX-like shell)

Or even let perl collect cmd's output itself (using its own `...`, and trimming one trailing newline if any with chomp).

<complex_query> | perl -lpe 'BEGIN{chomp ($repl = `cmd...`)}; $_ .= $repl'

¹ ksh93 and recent versions of mksh also support a ${ ...; } form which unlike the other one, but like in fish doesn't introduce a subshell environment.

² rc-like shells have only one type of quotes: '...' which are strong quotes (no expansion can happen within). You'd prevent the splitting by using ``(){cmd ...} (using the ``(sep1 sep2){cmd ...} syntax where you specify the list of separators for splitting, here empty) though that would also prevent trailing newline trimming.

³ also note that in csh/tcsh, "`cmd ...`" still splits on newlines.

4
  • Please check my edit, I am assuming that's what you meant but I don't know the rc shells, so I wanted to make sure you were notified.
    – terdon
    Apr 25 at 8:48
  • @terdon, rolled back. The syntax is `cmd or `{more complex commands} (or ``(sep){cmd...} to specify the separators in place of $ifs). Apr 25 at 8:54
  • Good thing I checked then. Seriously, the syntax for command substitution is a single, unbalanced backtick!? That's weirder than the hanging quotes of Lisp!
    – terdon
    Apr 25 at 8:59
  • @terdon, what's the use of the second `? rc has syntax for enclosing commands: {...}, the second ` would be superfluous. Apr 25 at 9:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.