I have a command returning multiple lines. For further processing I need to process each single line of those lines.

My current code works by modifying the IFS (Internal Field Separator):

ROWS=$(some command returning multiple lines)

O=$IFS #save original IFS
IFS=$(echo -en "\n\b") # set IFS to linebreak

for ROW in ${ROWS[@]}
  echo "$ROW"

IFS=$O #restore old IFS

I am wondering, is there a better way to access the single lines of the multiple lines output, one without modifying the IFS? Especially the readability of my script gets bad by modifying the IFS.

Update: I have troubles to get the answers working, e.g the one from choroba:

while IFS= read -r line ; do
    let var+=line #line 42
done << $(sqlite3 -list -nullvalue NULL -separator ',' /var/log/asterisk/master.db "${QUERY}")
echo "$var" # line 44

gives me

./bla.sh: row 44: Warning: here-document at line 43 delimited by end-of-file (wanted `$(sqlite3 -list -nullvalue NULL -separator , /var/log/asterisk/master.db ${QUERY})')
./bla.sh: row 42: let: echo "": syntax error: invalid arithmetic operator. (error causing character is \"""\").

Anyone can help me with this? Thanks!


What about read in a while loop?

some command returning multiple lines | while IFS= read -r line ; do
    echo "$line"

Be careful, though, the pipe is run in a subshell, which means you cannot change variable values for the rest of the script. If you need something to survive from your while loop, you can use bash's process substitution:

while IFS= read -r line ; do
    let var+=line
done < <(some command returning multiple lines)
echo "$var"
  • Using process substitution is a smart solution for this in bash, but it's worth noting that the subshell scoping of variables is only a problem in bash, doing the same thing in zsh would not have this issue at all. – Caleb Sep 11 '12 at 22:28
  • ksh doesn't have this issue either. – Didi Kohen Sep 12 '12 at 12:08
  • thank you guys, I tried this solution, but I can't get it working, please see my updated original post, thanks! – stefan.at.wpf Sep 13 '12 at 15:24
  • @stefan.at.wpf: You cannot place spaces and dollar signs around at will. They have meaning. – choroba Sep 14 '12 at 8:28

Setting IFS to only a newline is not enough. (Why are you also splitting at backspace characters, by the way?)

In your code, ${ROWS[@]} (which is a strange way to write $ROWSROWS is not an array) is not double-quoted. (If it was inside double quotes, you'd get a single string, since ROWS is not an array.) So the shell splits the value of the variable into fields at each IFS character, then treats each field as a glob pattern. For example, if one of the lines printed by the command contains the single character *, this will be replaced by the file names in the current directory.

You can turn off globbing with set -f. In most cases where you set IFS to use the shell's field splitting feature, you also need to turn off globbing. Set it back on with set +f.

The reliable idiom for reading a command's output line by line is while IFS= read -r.

some command returning multiple lines |
while IFS= read -r ROW; do

Note that most shells run each command of a pipeline in a separate subshell. So if you need to set variables and use them after the loop, wrap these commands in a group together with the loop. (Ksh and zsh are the exceptions, they run the last command of a pipeline in the parent shell.)

some command returning multiple lines | {
  while IFS= read -r ROW; do
  echo "There were $row_count rows."

You are mixing the here-document and here-string syntax in your update question.

Either use here-document:

while IFS= read -r line ; do
    let var+=line #line 42
done <<ENDMARK
$(sqlite3 -list -nullvalue NULL -separator ',' /var/log/asterisk/master.db "${QUERY}")

Or here-string:

while IFS= read -r line ; do
    let var+=line #line 42
done <<< $(sqlite3 -list -nullvalue NULL -separator ',' /var/log/asterisk/master.db "${QUERY}")

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.