Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have a variable which contains multiline output of a command. What's the most effecient way to read the output line by line from the variable?

For example:

jobs="$(jobs)"
if [ "$jobs" ]; then
    # read lines from $jobs
fi
share|improve this question

5 Answers 5

up vote 19 down vote accepted

You can use a while loop with process substitution:

while read -r line
do
    echo "$line"
done < <(jobs)

To read a multiline variable, a simple way is:

printf %s "$var" | while IFS= read -r line
do
   echo "$line"
done

Also, please don't call your variable jobs because that is a shell command and may cause confusion.

share|improve this answer
2  
If you want to keep all your whitespace, then use while IFS= read .... If you want to prevent \ interpretation, then use read -r –  Peter.O Mar 21 '11 at 15:41
    
I've fixed the points fred.bear mentioned, as well as changed echo to printf %s, so that your script would work even with non-tame input. –  Gilles Mar 21 '11 at 20:57
    
To read from a multiline variable, a herestring is preferable to piping from printf (see l0b0's answer). –  ata Nov 25 '11 at 20:34

To process the output of a command line by line (explanation):

jobs |
while IFS= read -r line; do
  process "$line"
done

If you have the data in a variable already:

printf %s "$foo" | …

printf %s "$foo" is almost identical to echo "$foo", but prints $foo literally, whereas echo "$foo" might interpret $foo as an option to the echo command if it begins with a -, and might expand backslash sequences in $foo in some shells.

Note that in some shells (ash, bash, pdksh, but not ksh or zsh), the right-hand side of a pipeline runs in a separate process, so any variable you set in the loop is lost. For example, the following line-counting script prints 0 in these shells:

n=0
printf %s "$foo" |
while IFS= read -r line; do
  n=$(($n + 1))
done
echo $n

A workaround is to put the remainder of the script (or at least the part that needs the value of $n from the loop) in a command list:

n=0
printf %s "$foo" | {
  while IFS= read -r line; do
    n=$(($n + 1))
  done
  echo $n
}

If acting on the non-empty lines is good enough and the input is not huge, you can use word splitting:

IFS='
'
set -f
for line in $(jobs); do
  # process line
done
set +f
unset IFS

Explanation: setting IFS to a single newline makes word splitting occur at newlines only (as opposed to any whitespace character under the default setting). set -f turns off globbing (i.e. wildcard expansion), which would otherwise happen to the result of a command substitution $(jobs) or a variable substitutino $foo. The for loop acts on all the pieces of $(jobs), which are all the non-empty lines in the command output. Finally, restore the globbing and IFS settings.

share|improve this answer
    
I have had trouble with setting IFS and unsetting IFS. I think the right thing to do is store the old value of IFS and set IFS back to that old value. I'm not a bash expert, but in my experience, this gets you back to the original bahavior. –  Bjorn Roche Jun 3 '13 at 20:06

In recent bash versions, use mapfile or readarray to efficiently read command output into arrays

$ readarray test < <(ls -ltrR)
$ echo ${#test[@]}
6305

Disclaimer: horrible example, but you can prolly come up with a better command to use than ls yourself

share|improve this answer
    
It's a nice way, but litters /var/tmp with temporary files on my system. +1 anyway –  eugene y Mar 22 '11 at 13:18
    
@eugene: that's funny. What system (distro/OS) is that on? –  sehe Mar 23 '11 at 0:41
    
It's FreeBSD 8. How to reproduce: put readarray in a function and call the function a few times. –  eugene y Mar 23 '11 at 8:16
    
Nice one, @sehe. +1 –  Teresa e Junior Feb 19 '13 at 21:58

Problem: if you use while loop it will run in subshell and all variables will be lost. Solution: use for loop

# change delimiter (IFS) to new line.
IFS_BAK=$IFS
IFS=$'\n'

for line in $variableWithSeveralLines; do
 echo "$line"

 # return IFS back if you need to split new line by spaces:
 IFS=$IFS_BAK
 IFS_BAK=
 lineConvertedToArraySplittedBySpaces=( $line )
 echo "{lineConvertedToArraySplittedBySpaces[0]}"
 # return IFS back to newline for "for" loop
 IFS_BAK=$IFS
 IFS=$'\n'

done 

# return delimiter to previous value
IFS=$IFS_BAK
IFS_BAK=
share|improve this answer
    
THANK YOU SO MUCH!! All of the above solutions failed for me. –  inquisitor Jun 11 '13 at 18:33
jobs="$(jobs)"
while IFS= read
do
    echo $REPLY
done <<< "$jobs"

References:

share|improve this answer
1  
-r is a good idea too; It prevents \` interpretation... (it is in your links, but its probably worth mentioning, just to round out your IFS=` (which is essential to prevent losing whitespace) –  Peter.O Mar 21 '11 at 15:33

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.