How can I read line by line from a variable in bash?

Question

I have a variable which contains multiline output of a command. What's the most effecient way to read the output line by line from the variable?

For example:

jobs="$(jobs)"
if [ "$jobs" ]; then
    # read lines from $jobs
fi

Answer 1

You can use a while loop with process substitution:

while read -r line
do
    echo "$line"
done < <(jobs)

To read a multiline variable, a simple way is:

printf %s "$var" | while IFS= read -r line
do
   echo "$line"
done

Also, please don't call your variable jobs because that is a shell command and may cause confusion.

Answer 2

To process the output of a command line by line (explanation):

jobs |
while IFS= read -r line; do
  process "$line"
done

If you have the data in a variable already:

printf %s "$foo" | …

printf %s "$foo" is almost identical to echo "$foo", but prints $foo literally, whereas echo "$foo" might interpret $foo as an option to the echo command if it begins with a -, and might expand backslash sequences in $foo in some shells.

Note that in some shells (ash, bash, pdksh, but not ksh or zsh), the right-hand side of a pipeline runs in a separate process, so any variable you set in the loop is lost. For example, the following line-counting script prints 0 in these shells:

n=0
printf %s "$foo" |
while IFS= read -r line; do
  n=$(($n + 1))
done
echo $n

A workaround is to put the remainder of the script (or at least the part that needs the value of $n from the loop) in a command list:

n=0
printf %s "$foo" | {
  while IFS= read -r line; do
    n=$(($n + 1))
  done
  echo $n
}

---

If acting on the non-empty lines is good enough and the input is not huge, you can use word splitting:

IFS='
'
set -f
for line in $(jobs); do
  # process line
done
set +f
unset IFS

Explanation: setting IFS to a single newline makes word splitting occur at newlines only (as opposed to any whitespace character under the default setting). set -f turns off globbing (i.e. wildcard expansion), which would otherwise happen to the result of a command substitution $(jobs) or a variable substitutino $foo. The for loop acts on all the pieces of $(jobs), which are all the non-empty lines in the command output. Finally, restore the globbing and IFS settings.

Answer 3

In recent bash versions, use mapfile or readarray to efficiently read command output into arrays

$ readarray test < <(ls -ltrR)
$ echo ${#test[@]}
6305

Disclaimer: horrible example, but you can prolly come up with a better command to use than ls yourself

Answer 4

jobs="$(jobs)"
while IFS= read
do
    echo $REPLY
done <<< "$jobs"

References:

• Here strings
• while
• IFS
• read

Answer 5

Problem: if you use while loop it will run in subshell and all variables will be lost. Solution: use for loop

# change delimiter (IFS) to new line.
IFS_BAK=$IFS
IFS=$'\n'

for line in $variableWithSeveralLines; do
 echo "$line"

 # return IFS back if you need to split new line by spaces:
 IFS=$IFS_BAK
 IFS_BAK=
 lineConvertedToArraySplittedBySpaces=( $line )
 echo "{lineConvertedToArraySplittedBySpaces[0]}"
 # return IFS back to newline for "for" loop
 IFS_BAK=$IFS
 IFS=$'\n'

done 

# return delimiter to previous value
IFS=$IFS_BAK
IFS_BAK=

Answer 6

I would go with:

jobs="$(jobs)"
IFS="`echo`"
for line in $jobs
do
  echo $line
done

(if the output from jobs could be > 16MB, I would stick it in a temp file, and traverse that).