Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have a command returning multiple lines. For further processing I need to process each single line of those lines.

My current code works by modifying the IFS (Internal Field Separator):

ROWS=$(some command returning multiple lines)

O=$IFS #save original IFS
IFS=$(echo -en "\n\b") # set IFS to linebreak

for ROW in ${ROWS[@]}
do
  echo "$ROW"
done

IFS=$O #restore old IFS

I am wondering, is there a better way to access the single lines of the multiple lines output, one without modifying the IFS? Especially the readability of my script gets bad by modifying the IFS.

Update: I have troubles to get the answers working, e.g the one from choroba:

while IFS= read -r line ; do
    let var+=line #line 42
done << $(sqlite3 -list -nullvalue NULL -separator ',' /var/log/asterisk/master.db "${QUERY}")
echo "$var" # line 44

gives me

./bla.sh: row 44: Warning: here-document at line 43 delimited by end-of-file (wanted `$(sqlite3 -list -nullvalue NULL -separator , /var/log/asterisk/master.db ${QUERY})')
./bla.sh: row 42: let: echo "": syntax error: invalid arithmetic operator. (error causing character is \"""\").

Anyone can help me with this? Thanks!

share|improve this question
    
choroba's answer says to do < <(some command returning multiple lines), but that's not what you are doing. –  Mikel Sep 13 '12 at 15:56

3 Answers 3

What about read in a while loop?

some command returning multiple lines | while IFS= read -r line ; do
    echo "$line"
done

Be careful, though, the pipe is run in a subshell, which means you cannot change variable values for the rest of the script. If you need something to survive from your while loop, you can use bash's process substitution:

while IFS= read -r line ; do
    let var+=line
done < <(some command returning multiple lines)
echo "$var"
share|improve this answer
    
Using process substitution is a smart solution for this in bash, but it's worth noting that the subshell scoping of variables is only a problem in bash, doing the same thing in zsh would not have this issue at all. –  Caleb Sep 11 '12 at 22:28
    
ksh doesn't have this issue either. –  David Kohen Sep 12 '12 at 12:08
    
thank you guys, I tried this solution, but I can't get it working, please see my updated original post, thanks! –  stefan.at.wpf Sep 13 '12 at 15:24
    
@stefan.at.wpf: You cannot place spaces and dollar signs around at will. They have meaning. –  choroba Sep 14 '12 at 8:28

Setting IFS to only a newline is not enough. (Why are you also splitting at backspace characters, by the way?)

In your code, ${ROWS[@]} (which is a strange way to write $ROWSROWS is not an array) is not double-quoted. (If it was inside double quotes, you'd get a single string, since ROWS is not an array.) So the shell splits the value of the variable into fields at each IFS character, then treats each field as a glob pattern. For example, if one of the lines printed by the command contains the single character *, this will be replaced by the file names in the current directory.

You can turn off globbing with set -f. In most cases where you set IFS to use the shell's field splitting feature, you also need to turn off globbing. Set it back on with set +f.

The reliable idiom for reading a command's output line by line is while IFS= read -r.

some command returning multiple lines |
while IFS= read -r ROW; do
  …
done

Note that most shells run each command of a pipeline in a separate subshell. So if you need to set variables and use them after the loop, wrap these commands in a group together with the loop. (Ksh and zsh are the exceptions, they run the last command of a pipeline in the parent shell.)

some command returning multiple lines | {
  while IFS= read -r ROW; do
    …
    row_count=$((row_count+1))
  done
  echo "There were $row_count rows."
}
share|improve this answer

You are mixing the here-document and here-string syntax in your update question.

Either use here-document:

while IFS= read -r line ; do
    let var+=line #line 42
done <<ENDMARK
$(sqlite3 -list -nullvalue NULL -separator ',' /var/log/asterisk/master.db "${QUERY}")
ENDMARK

Or here-string:

while IFS= read -r line ; do
    let var+=line #line 42
done <<< $(sqlite3 -list -nullvalue NULL -separator ',' /var/log/asterisk/master.db "${QUERY}")
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.