Say I have this file:

hello world

This program


for i in $(cat $1); do
    echo "tester: $i"


tester: hello
tester: world
tester: hello
tester: world

I'd like to have the for iterate over each line individually ignoring whitespaces though, i.e. the last two lines should be replaced by

tester: hello world

Using quotes for i in "$(cat $1)"; results in i being assigned the whole file at once. What should I change?

6 Answers 6


With for and IFS:


IFS=$'\n'       # make newlines the only separator
set -f          # disable globbing
for i in $(cat < "$1"); do
  echo "tester: $i"

Note however that it will skip empty lines as newline being an IFS-white-space character, sequences of it count as 1 and the leading and trailing ones are ignored. With zsh and ksh93 (not bash), you can change it to IFS=$'\n\n' for newline not to be treated specially, however note that all trailing newline characters (so that includes trailing empty lines) will always be removed by the command substitution.

Or with read (no more cat):


while IFS= read -r line; do
  echo "tester: $line"
done < "$1"

There, empty lines are preserved, but note that it would skip the last line if it was not properly delimited by a newline character.

  • 7
    thanks, I didn't know one could < into a whole loop. Although it makes perfectly sense now I saw it Commented Feb 7, 2011 at 11:06
  • 2
    I see IFS \ read -r line' in second example. Is really IFS=` needed ? IMHO it enough to say : while read -r line; do echo "tester: $line"; done < "$1" Commented Mar 19, 2012 at 16:45
  • 5
    @GrzegorzWierzowiecki IFS= turns off the stripping of leading and trailing whitespace. See In while IFS= read.., why does IFS have no effect? Commented Jan 12, 2014 at 0:29
  • 1
    @BenMares To prevent globbing expressions possibly appearing in the text we are reading from being expanded to matching file names. Try, for instance, printf '%s\n' '*o' 'bar' >afile; touch foo; IFS=$'\n'; for i in $(cat afile); do echo "$i"; done.
    – fra-san
    Commented Mar 4, 2020 at 8:00
  • 3
    A while IFS= read -r line || [ "$line" ]; do will process a trailing line not properly delimited by a newline character (but it will be added back).
    – user232326
    Commented Mar 4, 2020 at 16:15

(9 years later:)
Both provided answers would fail on files without a newline at the end, this will effectively skip the last line, produce no errors, would lead to disaster (learned hard way:).

The best concise solution I found so far that "Just Works" (in both bash and sh):

while IFS='' read -r LINE || [ -n "${LINE}" ]; do
    echo "processing line: ${LINE}"
done < /path/to/input/file.txt

For more in-depth discussion see this StackOverflow discussion: How to use "while read" (Bash) to read the last line in a file if there’s no newline at the end of the file?

Beware: this approach adds an additional newline to the last line if there is none already.

  • files with characters after the last newline are not text files, and those characters don't constitute a line. In many cases those bogus characters are better left ignored or removed, though there are cases where you may want to treat it as a extra line, so it's good you show how. Commented Mar 10, 2023 at 7:53
  • Note that files with NUL characters in them are also not text files, and except in zsh that loop would also fail if the intention was to keep them as if they were allowed in text files. Commented Mar 10, 2023 at 7:59

If you can avoid it, don't especially if it's to process text.

Most text utilities are already designed to process text one line at a time, and, at least for the GNU implementations, do it efficiently, correctly and handle error conditions nicely. Piping one to another which runs them in parallel also means you can leverage more than one processor to do the job.


<input.txt sed 's/^/tester /' > output.txt


<input.txt awk '{print "tester", $0}' > output.txt

More on that at: Why is using a shell loop to process text considered bad practice?

If it's not about text processing and you do need to run some command per line of a file, also note GNU xargs where you can do:

xargs -rd'\n' -I@ -a input.txt cp -- @ @.back

for instance.

With the bash shell, you can get each line of a file into an array with the readarray builtin:

readarray -t lines < input.txt &&
  for line in "${lines[@]}"; do
    do-some-non-text-processing-you-cannot-easily-do-with-xargs "$line" || break

POSIXly, you can use IFS= read -r line to read one line off some input, but beware that if you redirect the whole while read loop with the input file on stdin, then commands inside the loop will also have their stdin redirected to the file, so best is to use a different fd which you close inside the loop:

  IFS= read -r line <&3 ||
    [ -n "$line" ] # to cover for an unterminated last line.
    do-some-non-text-processing-you-cannot-easily-do-with-xargs "$line" ||
      break # abort upon failure if relevant
  } 3<&-
done 3< input.txt > output.txt

read -r line removes leading and trailing whitespace characters from the line that it reads provided they are in the $IFS variable, though only the yash shell honours that POSIX requirement. With most shells, that's limited to space and tab. ksh93 and recent versions of bash do it for all single-byte characters considered as whitespace in the locale.

So to read a line and also strip leading and trailing blanks, you can do: IFS=$' \t' read -r line. With ksh93, yash¹ or recent versions of bash. IFS=$' \t\r' would also strip the trailing CR character found in text files from the Microsoft world.

¹ though yash doesn't support the $'...' syntax yet, you'd need IFS=$(printf ' \t\r') there.


For what it is worth, I need to do that quite often, and can never remember the exact way of using while IFS= read..., so I defined the following function in my bash profile:

# iterate the line of a file and call input function
iterlines() {
    (( $# < 2 )) && { echo "Usage: iterlines <File> <Callback>"; return; }
    local File=$1
    local Func=$2
    n=$(cat "$File" | wc -l)
    for (( i=1; i<=n; i++ )); do
        "$Func" "$(sed "${i}q;d" "$File")"

This function first determines the number of lines in the file, then uses sed to extract line after line, and passes each line as a single string argument to any given function. I suppose this might get really inefficient with large files, but that hasn't been a problem for me so far (suggestions on how to improve this welcome of course).

The usage is pretty sweet IMO:

>> cat example.txt # note the use of spaces, whitespace, etc.

This is a sentence.
"wi\th quotes"
>> iterlines example.txt echo # preserves quotes, $ and whitespace

This is a sentence.
"wi\th quotes"
>> x() { echo "$#"; }; iterlines example.txt x # line always passed as single input string

To read all the lines, regardless of whether they are ended with a new line or not:

cat "somefile" | { cat ; echo ; } | while read line; do echo $line; done

Source : My open source project https://sourceforge.net/projects/command-output-to-html-table/


There is an answer here that may be useful to people who this doesn't work for

loop through file by row in tcsh

  • Not an answer as these usually contain some sort of explanation and/or a procedure to follow. A comment at best but not even a good one?
    – muthuh
    Commented Jan 26, 2022 at 12:27

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .