60

Say I have this file:

hello
world
hello world

This program

#!/bin/bash

for i in $(cat $1); do
    echo "tester: $i"
done

outputs

tester: hello
tester: world
tester: hello
tester: world

I'd like to have the for iterate over each line individually ignoring whitespaces though, i.e. the last two lines should be replaced by

tester: hello world

Using quotes for i in "$(cat $1)"; results in i being assigned the whole file at once. What should I change?

69

With for and IFS:

#!/bin/bash

IFS=$'\n'       # make newlines the only separator
set -f          # disable globbing
for i in $(cat < "$1"); do
  echo "tester: $i"
done

Note however that it will skip empty lines as newline being an IFS-white-space character, sequences of it count as 1 and the leading and trailing ones are ignored. With zsh and ksh93 (not bash), you can change it to IFS=$'\n\n' for newline not to be treated specially, however note that all trailing newline characters (so that includes trailing empty lines) will always be removed by the command substitution.

Or with read (no more cat):

#!/bin/bash

while IFS= read -r line; do
  echo "tester: $line"
done < "$1"

There, empty lines are preserved, but note that it would skip the last line if it was not properly delimited by a newline character.

  • 5
    thanks, I didn't know one could < into a whole loop. Although it makes perfectly sense now I saw it – Tobias Kienzler Feb 7 '11 at 11:06
  • 1
    I see IFS \ read -r line' in second example. Is really IFS=` needed ? IMHO it enough to say : while read -r line; do echo "tester: $line"; done < "$1" – Grzegorz Wierzowiecki Mar 19 '12 at 16:45
  • 4
    @GrzegorzWierzowiecki IFS= turns off the stripping of leading and trailing whitespace. See In while IFS= read.., why does IFS have no effect? – Gilles Jan 12 '14 at 0:29
0

For what it is worth, I need to do that quite often, and can never remember the exact way of using while IFS= read..., so I defined the following function in my bash profile:

# iterate the line of a file and call input function
iterlines() {
    (( $# < 2 )) && { echo "Usage: iterlines <File> <Callback>"; return; }
    local File=$1
    local Func=$2
    n=$(cat "$File" | wc -l)
    for (( i=1; i<=n; i++ )); do
        "$Func" "$(sed "${i}q;d" "$File")"
    done
}

This function first determines the number of lines in the file, then uses sed to extract line after line, and passes each line as a single string argument to any given function. I suppose this might get really inefficient with large files, but that hasn't been a problem for me so far (suggestions on how to improve this welcome of course).

The usage is pretty sweet IMO:

>> cat example.txt # note the use of spaces, whitespace, etc.
a/path

This is a sentence.
"wi\th quotes"
$End
>> iterlines example.txt echo # preserves quotes, $ and whitespace
a/path

This is a sentence.
"wi\th quotes"
$End
>> x() { echo "$#"; }; iterlines example.txt x # line always passed as single input string
1
1 
1
1
1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.