0

I have a file called test containing

test
test

and I am running this command

while read line
do
echo "$line"
done </tmp/test

This should output "test" two times but it only outputs it once. Using while IFS= read -r line doesn't change anything. Something that fixes it is to output a third empty line, but it should work regardless.

closed as off-topic by Kusalananda, RalfFriedl, roaima, Toby Speight, schily Oct 30 '18 at 11:57

  • This question does not appear to be about Unix or Linux within the scope defined in the help center.
If this question can be reworded to fit the rules in the help center, please edit the question.

  • 9
    Is the second test line terminated? What's the output of od -vtc < /tmp/test? Note that lines have to end in a newline character, otherwise they're just garbage after the last line. – Stéphane Chazelas Nov 4 '16 at 13:02
  • As Stéphane Chazelas said : od output should appear as 0000000 t e s t \n t e s t \n – steve Nov 4 '16 at 14:49
  • 4
    I'm voting to close this question as off-topic because after almost two years, the user has not returned with more information about the data as requested in comments. – Kusalananda Oct 30 '18 at 6:37
  • Seems on topic as it's about "Shell scripting." Seems hostile to have closed as off topic when it seems on topic as per the help center.. Original user is not required to update as anyone with enough score can edit question with requested clarification. Then again, unix.se has always seemed hostile compared with stackoverflow or others – JGurtz May 15 at 18:43
1

The most probable problem is that your file is not correctly terminated in a newline.

This could test it:

tail -n 1 fileToTest | wc -l

If the result is 1, there is a new line, if 0, then not. This will correct the file:

sed -i -e '$a\' fileToTest
  • That's not a valid test. printf 'a\nb' prints a<newline>b with traditional tail implementations. Just check whether wc -l < fileToTest returns 1 or 2 in this case. – Stéphane Chazelas Oct 30 '18 at 7:33
1

Try:

while IFS='' read -r line || [ "$line" ]

Yes, it is reading from standard input (stdin), the <test redirection goes to stdin.

What might be happening is that read is designed to work with text files. Correct text files must end on a newline.

To make a file end on a newline turns out to be very fast:

[ -n "$(tail -c1 file)" ] && printf '\n' >>file 

But if you can not edit the file to add a newline (if missing), the solution is to test (additionally) if something was read:

while read line || [ "$line" ]

That will make the loop execute if read failed (got to the end of the file without reading a newline (by default)) but something was read in (in "$line").

That will work correctly for files that do end on a newline or not.

No, a IFS='' read will not (directly) affect the reading of the last line, in fact, the IFS='' (compared to a default IFS[a] of space, tab, newline) will only affect the split into several variables (beside affecting the removal of leading and/or trailing white-space (if IFS contains only white space[b])). As there is only one variable ("$line") there is no splitting to be performed.

The -d option to read (in bash since 2.04) will not help either. There is no specific "end of file" character to match (the last byte could be any).

The only options left are either:

  • to repair the file to make it a correct text file
  • test if anything was read into the variable line.

The correct script, then, will be:

#!/bin/bash
while IFS='' read -r line || [ "$line" ]
do
    printf '%s\n' "$line"
done <test

IFS='' used to avoid problems with un-common values of IFS.


[a][b]Of course, If IFS may have any value, many other characters may get removed. Try (for values that end on 'x')

printf "test\ntesx\ntest" | while IFS="x" read -r line || [ "$line" ]; do echo "$line"; done

No, zsh does something different.

Not the answer you're looking for? Browse other questions tagged or ask your own question.