124

I am trying to get the output of a pipe into a variable. I tried the following things:

echo foo | myvar=$(</dev/stdin)
echo foo | myvar=$(cat)
echo foo | myvar=$(tee)

But $myvar is empty.

I don’t want to do:

myvar=$(echo foo)

Because I don’t want to spawn a subshell.

Any ideas?

Edit: I don’t want to spawn a subshell because the command before the pipe needs to edit global variables, which it can’t do in a subshell. Can it? The echo thing is just for simplification. It’s more like:

complex_function | myvar=$(</dev/stdin)

And I don’t get, why that doesn’t work. This works for example:

complex_function | echo $(</dev/stdin)
11
  • 1
    I don't understand what you're trying to do since none of your examples are correct syntax. What pipe? What is myvar supposed to contain? Could you give an example with a real command and explain what output you want to save? And what do you have against subshells anyway?
    – terdon
    Commented Jan 17, 2017 at 10:22
  • I don’t even understand why $myvar does not contain foo in my examples. After all, foo should be in stdin. I simplified the example on purpose. The echo foo thing is actually a more complicated command changing global variables, which won’t work if it’s in a subshell.
    – Parckwart
    Commented Jan 17, 2017 at 10:26
  • 3
    Sorry, but you're pretty much stuck with using subshells, even if you don't want to use them. Each command in pipes is executed in subshells too, see stackoverflow.com/a/5760832/3701431 Commented Jan 17, 2017 at 10:35
  • 3
    Yes, you can edit variables in a subshell and no, you can't assign the output if a command to a variable without a subshell. This is what's known as an XY problem. Please edit your question and explain what you are actually trying to do. Give an example of code that reproduces your problem and we should be able to help you out.
    – terdon
    Commented Jan 17, 2017 at 10:55
  • 1
    @Parckwart no, all commands in a pipeline are executed in subshells. See the "Pipelines" section in man bash. Just give us a complete example and we can help you out.
    – terdon
    Commented Jan 17, 2017 at 10:57

8 Answers 8

136

The correct solution is to use command substitution like this:

variable=$(complex_command)

as in

message=$(echo 'hello')

(or for that matter, message=hello in this case).

Your pipeline:

echo 'hello' | message=$(</dev/stdin)

or

echo 'hello' | read message

actually works. The only problem is that the shell that you're using will run the second part of the pipeline in a subshell. This subshell is destroyed when the pipeline exits, so the value of $message is not retained in the shell.

Here you can see that it works:

$ echo 'hello' | { read message; echo "$message"; }
hello

... but since the subshell's environment is separate (and gone):

$ echo "$message"

(no output)

One solution for you would be to switch to ksh93 which is smarter about this:

$ echo 'hello' | read message
$ echo "$message"
hello

Another solution for bash would be to set the lastpipe shell option. This would make the last part of the pipeline run in the current environment. This however does not work in interactive shells as lastpipe requires that job control is not active.

#!/bin/bash

shopt -s lastpipe
echo 'hello' | read message
echo "$message"
16
  • and what if later I wold like to pass multiline output from variable to another command? E. g. files=$(ls); echo $files | subcommand breaks original output and produces all items in one line while ls | subcommand sends files by one
    – oxfn
    Commented Nov 25, 2021 at 15:54
  • 1
    @oxfn echo "$files". Without the quoting, the shell would split the value on whitespace etc. (but even when quoting, echo may change the contents of the string, see Why is printf better than echo?). Note that saving the output of ls in a variable is pretty useless as there is no way of safely accessing the individual filenames (valid filenames can contain newlines). See Why *not* parse `ls` (and what to do instead)? and also When is double-quoting necessary?
    – Kusalananda
    Commented Nov 25, 2021 at 16:38
  • How would one use this with sed? E.g. echo $RANDOM | md5sum | head -c 20 | { read val; sed -i 's/somevalue/$var/g' myFile - doesn't work, it replaces with the string "$var" not the actual value.
    – geoidesic
    Commented Jan 24, 2022 at 21:45
  • @geoidesic Because the shell does not expand variables inside single-quoted strings. Use double quotes instead.
    – Kusalananda
    Commented Jan 24, 2022 at 22:20
  • 1
    @Erwann It's a compound command. Or more exactly, it is exactly a single compound command. Since it is one command, input can be piped or redirected into it and its output may be piped or redirected elsewhere. I'm using it in my answer to read a string arriving over a pipe and then to output that string. It would not be possible without using { ... }. You could see the contents (code) within { ... } as a separate script if that helps, at least the way I've used it above.
    – Kusalananda
    Commented Feb 24, 2022 at 7:34
14

Use command substitution:

myvar=`echo foo`

or

myvar=$(echo foo)
4
  • 8
    `echo foo` is just the deprecated alternative to $(echo foo), isn’t it?
    – Parckwart
    Commented Jan 17, 2017 at 10:19
  • 3
    @Parckwart yes. Well, not actually "deprecated", it's still supported. it's just that $() is better almost always.
    – terdon
    Commented Jan 17, 2017 at 10:20
  • 3
    command substitution still spawns a subshell, so it won't help the OP here. Commented Oct 23, 2018 at 10:37
  • 4
    The OP actually said "I don't want to do that...." Commented Jan 31, 2020 at 10:22
6

Given a function that modifies a global variable and outputs something on stdout:

global_variable=old_value
myfunction() {
  global_variable=new_value
  echo some output
}

In ksh93 or mksh R45 or newer you can use:

var=${
  myfunction
}

Then:

$ print -r -- "$global_variable, $var"
new_value, some output

${ ...; } is a form of command substitution that doesn't spawn a subshell. For commands that are builtin commands, instead of having them writing their output to a pipe (for which you'd need different processes to read and write on the pipe to avoid dead-locks), ksh93 just makes them not output anything but gather what they would have been outputting in to make up the expansion. mksh uses a temporary file instead.

$ ksh -c 'a=${ b=123; echo foo;}; print -r -- "$a $b"'
foo 123

fish's command substitution also behaves like ksh93's ${ ...; }:

$ fish -c 'set a (set b 123; echo foo); echo $a $b'
foo 123

In most other shells, you'd use a temporary file:

myfunction > file
var=$(cat file) # can be optimised to $(<file) with some shells

On Linux, and with bash 4.4 or older or zsh (that use temp files for <<<), you can do:

{
  myfunction > /dev/fd/3 &&
  var=$(cat<&3)
} 3<<< ''

In zsh, you can also do:

() {
   myfunction > $1
   var=$(<$1)
} =(:)

In Korn-like shells such as ksh, zsh or bash, command substitution, whether the $(cmd...) standard form or the $(<file) or ${ cmd...; } variants strip all trailing newline characters (from file or the output of cmd). See shell: keep trailing newlines ('\n') in command substitution for how to work around that.

In fish, set var (cmd) assigns each line of the output of cmd to separate elements of the $var array. $var will contain the same thing whether cmd outputs foo or foo<newline>. Since version 3.4.0, fish also supports set var "$(cmd)" which behaves like in Korn-like shells (removes all trailing newline characters).

11
  • interesting about the <<< temp file.. you can also write to "global" shell variables after using the shell options shopt -s lastpipe && set +m
    – alchemy
    Commented Apr 10, 2022 at 22:27
  • @alchemy, I don't see how lastpipe would help. Even if you meant myfunction | IFS= read -rd '' var, myfunction would still run in a subshell. Commented Apr 11, 2022 at 6:46
  • those two options do actually allow changing shell variables: see my answer unix.stackexchange.com/a/698694/346155
    – alchemy
    Commented Apr 12, 2022 at 2:13
  • 1
    (1) +1, because complex_function > tmpfile; myvar=$(<tmpfile) is probably the best answer to the question for bash (with the caveat that you may need to use myvar=$(cat tmpfile) in some other shells). How on earth did nobody else suggest this in four months? (1b) I’m surprised that you didn’t even link to an answer explaining how to preserve multiple newlines at the end of a command substitution. … (Cont’d) Commented Apr 24, 2022 at 6:40
  • 1
    @ak2, I don't expect there be. Is suppose I hadn't realised =() also worked or possibly it didn't work in older versions. Commented Jul 17, 2023 at 9:27
4

Here is a simple way to do it in bash -

complexFunction(){
cat <<_endOfHereFile_
The lazy dog
jumped over 
the moon
_endOfHereFile_
}

{ read -d '' message; }< <(complexFunction)
echo "${message}"

The result of echo "${message}" is

The lazy dog
jumped over 
the moon

The complex function could be any function emitting to stdout. It happens to use a here file in this example, but that is not important.

The call read -d '' reads stdin up to the delimiter, which since it is the empty character `` means reading until the end of input.

The syntax {lhs;}< <(rhs) redirects the stdout of rhs to the stdin of lhs, where lhs enjoys the shared variable namespace so that echo ${message} works as desired.

9
  • (1) You totally missed the point of the question.  It says “I don’t want to spawn a subshell because the command before the pipe needs to edit global variables, which it can’t do in a subshell.”  Your (misnamed!) “complexFunction” is a single command!  Try adding othervar=$(date) to your function, and then do echo "$othervar" after doing your command with <(complexFunction).  Or add a cd to your function, and then do pwd after doing your command.  When you do <(…), you are running a subshell, just like when you do $(…). … (Cont’d) Commented Apr 24, 2022 at 6:43
  • (Cont’d) … (2) The braces in your { read -d '' message; } command are not necessary, since read is a single, simple command, and not a compound or complex command.  (3) Since you aren’t using read -r, backslashes in the data will cause problems.  (4) Since you aren’t using IFS=, leading and trailing whitespace can be lost. Commented Apr 24, 2022 at 6:43
  • @G-ManSays'ReinstateMonica' Good point. What the author wants - to be able to set environment variables in multiple segments of a pipeline, in bash, is impossible. Yet a selected answer exists - setting shopt -s lastpipe in bash, or using ksh. This < < syntax is actually better than shopt -s lastpipe because the effect doesn't linger, so it is helpful. While you are correct, I can't see why you posted here and not the selected answer. Commented Apr 24, 2022 at 19:32
  • (5) What do you mean by “the selected answer”?  There is no accepted answer. (6) What do you mean by “This < < syntax is actually better than shopt -s lastpipe because the effect doesn't linger, so it is helpful.”?  Do you mean “<<”?  And the OP wants a persistent variable, so how is a transient result helpful? Commented Apr 24, 2022 at 19:45
  • "selected"=>"most highly upvoted". The syntax is < <, not <<. The LHS is executed in the current shell, and the output from the RHS is redirected to input of the LHS. It is effectively an inverted two-stage pipeline. Did you run it in bash to check? It runs. Commented Apr 24, 2022 at 19:55
2

I would use a temporary file to store the complex function result, and then read the value from the temp file for processing.

# Create a temp file for storage
tmppath=$(mktemp)
# run the command and get the output to the temp file
complex_command > "${tmppath}"
# read the file into variable
message=$(cat "${tmppath}")
# use the variable
echo "$message"

rm -f "${tmppath}"

The usage of mktemp can refer to How create a temporary file in shell script?

4
  • Double-quote your variables when you use them so that their contents isn't parsed and word-split by the shell. For example echo $message should become echo "$message". (The curly braces are mostly unneeded.) Note that if $message begins with a dash (hyphen) all bets are off when you try to use echo anyway Commented May 7, 2021 at 6:53
  • 2
    Shouldn't message=$(echo ${tmppath}) be message=$(cat "$tmppath") to get the contents of the temporary file rather then its name Commented May 7, 2021 at 6:56
  • (1) I’m giving you a +1 for posting the best answer, complete with mktemp andrm, even though Stéphane Chazelas posted the bare bones of that answer four years earlier. Please get into the habit of reading all the existing answers before you post a new one. It’s OK to post a new answer improving on a previous post (IMO, you did that), but you should cite any such previous posts. … (Cont’d) Commented Apr 24, 2022 at 6:51
  • (Cont’d) … (2) @roaima is right: you should double-quote all your variables (e.g., complex_command > "$tmppath" and rm -f "$tmppath"), and you don’t need any of those curly braces.  (3) You could improve this answer by testing whether mktemp succeeded before you use "$tmppath". Commented Apr 24, 2022 at 6:51
0

I am not really an expert, but have you tried the following?

echo "hello world" \
| { echo_out=$(< /dev/stdin); echo "echo output is: $echo_out"; } \
| cut -d":" -f 2
1
  • 3
    And the result of echo $echo_out after running this...? Commented Jul 6, 2020 at 22:25
0

You could just use the read command and Command Grouping with curley braces: echo foo | { read myvar; echo $myvar; } except you want to use "global variables", which I think you mean "shell variables" (declare -p to list, or set | grep myvar or set -o posix; set). https://www.gnu.org/software/bash/manual/html_node/Command-Grouping.html

Using Command Grouping executes them in the "current shell context", however that is still a subshell as soon as a pipe is used, even if the braces surround the entire line { echo foo | read myvar; echo $myvar; }.

So you have to use shell options. (it seems, because the command line is interactive by default):

shopt -s lastpipe      # sets shell option for lastpipe on
set +m                 # turns job control off, part of interactive shell
echo foo | read myvar; echo $myvar
# foo

With those shell options, chaining works read works echo foo | read myvar; echo $myvar" bar" | read myvar2; echo $myvar2, to produce foo bar.

PS, you can put those shell options in your .bashrc, but I'm not sure if there are downsides to that beside possible incompatibility with scripts that use interactive shell flag -i.

2
  • As far as I can tell, you’re just repeating what others have said. Commented Apr 24, 2022 at 7:22
  • @G-ManSays'ReinstateMonica' nah, it doen't work without set +m. And the other answer on Command Grouping wasnt very clear to me.
    – alchemy
    Commented Apr 25, 2022 at 4:43
0

Notes to editors: Please don't change the code. I'll report it.

I hope you remember your username and password because lot of people found that answer useful.. But not me, no Sr. and that's because in a Makefile rules are slightly different.

In my case I'm encoding some JSON data and providing that to curl so I'm going to share just the basic idea because it is already encoded if you use the right type.

# Remember, this is a Makefile!
# Assuming your environment knows the variables in brackets

more_tests:
    @LABORADO=$$(echo -n "{\"user\":\"$(ALWAYS)\",\"know\":\"$(NOTHING)\"}" | base64) && \
    echo $(SHELL) && \
    echo \"$$LABORADO\"

And this is the output:

> make more_tests 
/bin/sh
"eyJ1c2VyIjoic2hvdWxkIiwia25vdyI6ImhpbXNlbGYifQ=="

The shell variable was just a random environment variable that I picked as an example because I know whatever your environment is it will know that value (I hope).

Remember.. I know $ALWAYS, $NOTHING, but I know my environment :D)

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .