75

I am trying to get the output of a pipe into a variable. I tried the following things:

echo foo | myvar=$(</dev/stdin)
echo foo | myvar=$(cat)
echo foo | myvar=$(tee)

But $myvar is empty.

I don’t want to do:

myvar=$(echo foo)

Because I don’t want to spawn a subshell.

Any ideas?

Edit: I don’t want to spawn a subshell because the command before the pipe needs to edit global variables, which it can’t do in a subshell. Can it? The echo thing is just for simplification. It’s more like:

complex_function | myvar=$(</dev/stdin)

And I don’t get, why that doesn’t work. This works for example:

complex_function | echo $(</dev/stdin)
11
  • 1
    I don't understand what you're trying to do since none of your examples are correct syntax. What pipe? What is myvar supposed to contain? Could you give an example with a real command and explain what output you want to save? And what do you have against subshells anyway?
    – terdon
    Jan 17, 2017 at 10:22
  • I don’t even understand why $myvar does not contain foo in my examples. After all, foo should be in stdin. I simplified the example on purpose. The echo foo thing is actually a more complicated command changing global variables, which won’t work if it’s in a subshell.
    – Parckwart
    Jan 17, 2017 at 10:26
  • 3
    Sorry, but you're pretty much stuck with using subshells, even if you don't want to use them. Each command in pipes is executed in subshells too, see stackoverflow.com/a/5760832/3701431 Jan 17, 2017 at 10:35
  • 3
    Yes, you can edit variables in a subshell and no, you can't assign the output if a command to a variable without a subshell. This is what's known as an XY problem. Please edit your question and explain what you are actually trying to do. Give an example of code that reproduces your problem and we should be able to help you out.
    – terdon
    Jan 17, 2017 at 10:55
  • 1
    @Parckwart no, all commands in a pipeline are executed in subshells. See the "Pipelines" section in man bash. Just give us a complete example and we can help you out.
    – terdon
    Jan 17, 2017 at 10:57

8 Answers 8

84

The correct solution is to use command substitution like this:

variable=$(complex_command)

as in

message=$(echo 'hello')

(or for that matter, message=hello in this case).

Your pipeline:

echo 'hello' | message=$(</dev/stdin)

or

echo 'hello' | read message

actually works. The only problem is that the shell that you're using will run the second part of the pipeline in a subshell. This subshell is destroyed when the pipeline exits, so the value of $message is not retained in the shell.

Here you can see that it works:

$ echo 'hello' | { read message; echo "$message"; }
hello

... but since the subshell's environment is separate (and gone):

$ echo "$message"

(no output)

One solution for you would be to switch to ksh93 which is smarter about this:

$ echo 'hello' | read message
$ echo "$message"
hello

Another solution for bash would be to set the lastpipe shell option. This would make the last part of the pipeline run in the current environment. This however does not work in interactive shells as lastpipe requires that job control is not active.

#!/bin/bash

shopt -s lastpipe
echo 'hello' | read message
echo "$message"
14
  • and what if later I wold like to pass multiline output from variable to another command? E. g. files=$(ls); echo $files | subcommand breaks original output and produces all items in one line while ls | subcommand sends files by one
    – oxfn
    Nov 25, 2021 at 15:54
  • 1
    @oxfn echo "$files". Without the quoting, the shell would split the value on whitespace etc. (but even when quoting, echo may change the contents of the string, see Why is printf better than echo?). Note that saving the output of ls in a variable is pretty useless as there is no way of safely accessing the individual filenames (valid filenames can contain newlines). See Why *not* parse `ls` (and what to do instead)? and also When is double-quoting necessary?
    – Kusalananda
    Nov 25, 2021 at 16:38
  • How would one use this with sed? E.g. echo $RANDOM | md5sum | head -c 20 | { read val; sed -i 's/somevalue/$var/g' myFile - doesn't work, it replaces with the string "$var" not the actual value.
    – geoidesic
    Jan 24 at 21:45
  • @geoidesic Because the shell does not expand variables inside single-quoted strings. Use double quotes instead.
    – Kusalananda
    Jan 24 at 22:20
  • 1
    @Erwann It's a compound command. Or more exactly, it is exactly a single compound command. Since it is one command, input can be piped or redirected into it and its output may be piped or redirected elsewhere. I'm using it in my answer to read a string arriving over a pipe and then to output that string. It would not be possible without using { ... }. You could see the contents (code) within { ... } as a separate script if that helps, at least the way I've used it above.
    – Kusalananda
    Feb 24 at 7:34
11

Use command substitution:

myvar=`echo foo`

or

myvar=$(echo foo)
4
  • 4
    `echo foo` is just the deprecated alternative to $(echo foo), isn’t it?
    – Parckwart
    Jan 17, 2017 at 10:19
  • 2
    @Parckwart yes. Well, not actually "deprecated", it's still supported. it's just that $() is better almost always.
    – terdon
    Jan 17, 2017 at 10:20
  • 3
    command substitution still spawns a subshell, so it won't help the OP here. Oct 23, 2018 at 10:37
  • 4
    The OP actually said "I don't want to do that...." Jan 31, 2020 at 10:22
5

Given a function that modifies a global variable and outputs something on stdout:

global_variable=old_value
myfunction() {
  global_variable=new_value
  echo some output
}

In ksh93 or mksh R45 or newer you can use:

var=${
  myfunction
}

Then:

$ print -r -- "$global_variable, $var"
new_value, some output

${ ...; } is a form of command substitution that doesn't spawn a subshell. For commands that are builtin commands, instead of having them writing their output to a pipe (for which you'd need different processes to read and write on the pipe to avoid dead-locks), ksh93 just makes them not output anything but gather what they would have been outputting in to make up the expansion. mksh uses a temporary file instead.

$ ksh -c 'a=${ b=123; echo foo;}; print -r -- "$a $b"'
foo 123

fish's command substitution also behaves like ksh93's ${ ...; }:

$ fish -c 'set a (set b 123; echo foo); echo $a $b'
foo 123

In most other shells, you'd use a temporary file:

myfunction > file
var=$(cat file) # can be optimised to $(<file) with some shells

On Linux, and with bash 4.4 or older or zsh (that use temp files for <<<), you can do:

{
  myfunction > /dev/fd/3 &&
  var=$(cat<&3)
} 3<<< ''

In zsh, you can also do:

() {
   myfunction > $1
   var=$(<$1)
} =(:)

In Korn-like shells such as ksh, zsh or bash, command substitution, whether the $(cmd...) standard form or the $(<file) or ${ cmd...; } variants strip all trailing newline characters (from file or the output of cmd). See shell: keep trailing newlines ('\n') in command substitution for how to work around that.

In fish, set var (cmd) assigns each line of the output of cmd to separate elements of the $var array. $var will contain the same thing whether cmd outputs foo or foo<newline>. Since version 3.4.0, fish also supports set var "$(cmd)" which behaves like in Korn-like shells (removes all trailing newline characters).

9
  • interesting about the <<< temp file.. you can also write to "global" shell variables after using the shell options shopt -s lastpipe && set +m
    – alchemy
    Apr 10 at 22:27
  • @alchemy, I don't see how lastpipe would help. Even if you meant myfunction | IFS= read -rd '' var, myfunction would still run in a subshell. Apr 11 at 6:46
  • those two options do actually allow changing shell variables: see my answer unix.stackexchange.com/a/698694/346155
    – alchemy
    Apr 12 at 2:13
  • @alchemy, see if my latest edit makes it clearer what I actually meant. Apr 15 at 17:58
  • 1
    (1) +1, because complex_function > tmpfile; myvar=$(<tmpfile) is probably the best answer to the question for bash (with the caveat that you may need to use myvar=$(cat tmpfile) in some other shells). How on earth did nobody else suggest this in four months? (1b) I’m surprised that you didn’t even link to an answer explaining how to preserve multiple newlines at the end of a command substitution. … (Cont’d) Apr 24 at 6:40
3

Here is a simple way to do it in bash -

complexFunction(){
cat <<_endOfHereFile_
The lazy dog
jumped over 
the moon
_endOfHereFile_
}

{ read -d '' message; }< <(complexFunction)
echo "${message}"

The result of echo "${message}" is

The lazy dog
jumped over 
the moon

The complex function could be any function emitting to stdout. It happens to use a here file in this example, but that is not important.

The call read -d '' reads stdin up to the delimiter, which since it is the empty character `` means reading until the end of input.

The syntax {lhs;}< <(rhs) redirects the stdout of rhs to the stdin of lhs, where lhs enjoys the shared variable namespace so that echo ${message} works as desired.

8
  • (1) You totally missed the point of the question.  It says “I don’t want to spawn a subshell because the command before the pipe needs to edit global variables, which it can’t do in a subshell.”  Your (misnamed!) “complexFunction” is a single command!  Try adding othervar=$(date) to your function, and then do echo "$othervar" after doing your command with <(complexFunction).  Or add a cd to your function, and then do pwd after doing your command.  When you do <(…), you are running a subshell, just like when you do $(…). … (Cont’d) Apr 24 at 6:43
  • (Cont’d) … (2) The braces in your { read -d '' message; } command are not necessary, since read is a single, simple command, and not a compound or complex command.  (3) Since you aren’t using read -r, backslashes in the data will cause problems.  (4) Since you aren’t using IFS=, leading and trailing whitespace can be lost. Apr 24 at 6:43
  • @G-ManSays'ReinstateMonica' Good point. What the author wants - to be able to set environment variables in multiple segments of a pipeline, in bash, is impossible. Yet a selected answer exists - setting shopt -s lastpipe in bash, or using ksh. This < < syntax is actually better than shopt -s lastpipe because the effect doesn't linger, so it is helpful. While you are correct, I can't see why you posted here and not the selected answer. Apr 24 at 19:32
  • (5) What do you mean by “the selected answer”?  There is no accepted answer. (6) What do you mean by “This < < syntax is actually better than shopt -s lastpipe because the effect doesn't linger, so it is helpful.”?  Do you mean “<<”?  And the OP wants a persistent variable, so how is a transient result helpful? Apr 24 at 19:45
  • "selected"=>"most highly upvoted". The syntax is < <, not <<. The LHS is executed in the current shell, and the output from the RHS is redirected to input of the LHS. It is effectively an inverted two-stage pipeline. Did you run it in bash to check? It runs. Apr 24 at 19:55
2

I would use a temporary file to store the complex function result, and then read the value from the temp file for processing.

# Create a temp file for storage
tmppath=$(mktemp)
# run the command and get the output to the temp file
complex_command > "${tmppath}"
# read the file into variable
message=$(cat "${tmppath}")
# use the variable
echo "$message"

rm -f "${tmppath}"

The usage of mktemp can refer to How create a temporary file in shell script?

4
  • Double-quote your variables when you use them so that their contents isn't parsed and word-split by the shell. For example echo $message should become echo "$message". (The curly braces are mostly unneeded.) Note that if $message begins with a dash (hyphen) all bets are off when you try to use echo anyway
    – roaima
    May 7, 2021 at 6:53
  • 2
    Shouldn't message=$(echo ${tmppath}) be message=$(cat "$tmppath") to get the contents of the temporary file rather then its name
    – roaima
    May 7, 2021 at 6:56
  • (1) I’m giving you a +1 for posting the best answer, complete with mktemp andrm, even though Stéphane Chazelas posted the bare bones of that answer four years earlier. Please get into the habit of reading all the existing answers before you post a new one. It’s OK to post a new answer improving on a previous post (IMO, you did that), but you should cite any such previous posts. … (Cont’d) Apr 24 at 6:51
  • (Cont’d) … (2) @roaima is right: you should double-quote all your variables (e.g., complex_command > "$tmppath" and rm -f "$tmppath"), and you don’t need any of those curly braces.  (3) You could improve this answer by testing whether mktemp succeeded before you use "$tmppath". Apr 24 at 6:51
0

I am not really an expert, but have you tried the following?

echo "hello world" \
| { echo_out=$(< /dev/stdin); echo "echo output is: $echo_out"; } \
| cut -d":" -f 2
1
  • 2
    And the result of echo $echo_out after running this...?
    – roaima
    Jul 6, 2020 at 22:25
0

You could just use the read command and Command Grouping with curley braces: echo foo | { read myvar; echo $myvar; } except you want to use "global variables", which I think you mean "shell variables" (declare -p to list, or set | grep myvar or set -o posix; set). https://www.gnu.org/software/bash/manual/html_node/Command-Grouping.html

Using Command Grouping executes them in the "current shell context", however that is still a subshell as soon as a pipe is used, even if the braces surround the entire line { echo foo | read myvar; echo $myvar; }.

So you have to use shell options. (it seems, because the command line is interactive by default):

shopt -s lastpipe      # sets shell option for lastpipe on
set +m                 # turns job control off, part of interactive shell
echo foo | read myvar; echo $myvar
# foo

With those shell options, chaining works read works echo foo | read myvar; echo $myvar" bar" | read myvar2; echo $myvar2, to produce foo bar.

PS, you can put those shell options in your .bashrc, but I'm not sure if there are downsides to that beside possible incompatibility with scripts that use interactive shell flag -i.

2
  • As far as I can tell, you’re just repeating what others have said. Apr 24 at 7:22
  • @G-ManSays'ReinstateMonica' nah, it doen't work without set +m. And the other answer on Command Grouping wasnt very clear to me.
    – alchemy
    Apr 25 at 4:43
-1

As people mentioned already echo doesn't read from stdin so pipes would not work. However, you can use the read command and then use the assigned variable with echo. Take a look at this example.

# Extract the ip address with a one-line command
ip addr | awk '/inet/ && /lo/' | { read myIP; echo ${${myIP%/*}#* } }
127.0.0.1

Now let's go step by step

ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever

Now filter the line with the words inet for ipv4 and lo for your network device, this could have been eth0 or wlan0 or whatever your distro named your devices.

ip addr | awk '/inet/ && / lo/'
inet 127.0.0.1/8 scope host lo

I had to add an extra space before lo because I was getting two lines, the second line matched my filter with inet and global. Well actually I was trying to find the IP on eth0 which is easier to filter, but for the purposes of this example I tried lo
inet 111.222.233.244/20 brd 111.222.233.255 scope global eth0.

Here's where I had the same issue as you, I knew what I had to do to chop that string and get the IP, but echo alone would not make it for me. Finally, the solution was to read, assign and reuse that variable.

To understand how echo ${${myIP%/*}#* } worked, check the bash cheat sheet at https://devhints.io/bash

9
  • 2
    Note that you are using syntax specific to the zsh shell, while the user in the question uses the bash shell. Your substitution does not work in bash, so referring to a bash "cheat sheet" is a bit misleading. Also, ip -family inet -brief addr show eth0 | awk '{ sub("/.*", "", $3); print $3 }'.
    – Kusalananda
    Apr 23 at 21:15
  • 2
    (1) “echo doesn't read from stdin so pipes would not work” is a red herring; it has nothing to do with the question.  (2) You totally missed the point of the question.  It says “I don’t want to spawn a subshell because the command before the pipe needs to edit global variables, which it can’t do in a subshell.”  You’re not doing that.  Try adding othervar=$(date) to your pipeline, and then do echo "$othervar" afterwards.  Or add a cd, and then do pwd after doing your command.  As people mentioned already, when you pipe, bash creates subshell(s). … (Cont’d) Apr 24 at 7:22
  • 2
    (Cont’d) …  (3) It’s implicit that the OP wants the output of the “complex function” to be assigned to a persistent variable.  That is, echo "$myIP" should work on the next line of the script.  You have the kernel of a potentially useful workaround here, but it’s not useful because you don’t explain it.  (4) Your detailed explanation of your ‘‘complex’’ commands will not help anybody who comes to this page. Apr 24 at 7:22
  • I didn't realize up until now how much difference there is between bash and zsh... it increases complexity a lot. In zsh just echo mystring | read myvar works and the data will persist, simply as that.
    – Daniel N.
    Apr 24 at 20:04
  • 1
    (Cont’d) …  (5) He was referring to your ${${myIP%right}#left} construct, which doesn’t work in Bash.  I know that it doesn’t work in Bash, but (I’m a little embarrassed to admit) I didn’t even notice that you were doing that until today.  But that supports my point #4: your answer was so complex and wide-ranging (and unrelated to the question) that it was a forest, in which the trees could easily be overlooked.  (For future reference, you would need to do temp=${myIP%right}; echo "${temp#left}" in Bash.)  … (Cont’d) Apr 28 at 4:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.