69

I am trying to read the output of a command in bash using a while loop.

while read -r line
do
    echo "$line"
done <<< $(find . -type f)

The output I got

ranveer@ranveer:~/tmp$ bash test.sh
./test.py ./test1.py ./out1 ./test.sh ./out ./out2 ./hello
ranveer@ranveer:~/tmp$ 

After this I tried

$(find . -type f) | 
while read -r line
do
    echo "$line"
done 

but it generated an error test.sh: line 5: ./test.py: Permission denied.

So, how do I read it line by line because I think currently it is slurping the entire line at once.

Required output:

./test.py
./test1.py
./out1
./test.sh
./out
./out2
./hello
3

4 Answers 4

86

There's a mistake, you need < <(command) not <<<$(command)

< <( ) is a Process Substitution, $() is a command substitution and <<< is a here-string.

0
14

Note that there's nothing stopping file names from containing newline characters. The canonical way to run a command for each file found by find is.

find . -type f -exec cmd {} \;

And if you want things done in bash:

find . -type f -exec bash -c '
  for file do
    something with "$file"
  done' bash {} +

Also, the canonical way to call the "read" command in scripts (if you don't want it to do extra processing on the input) is:

IFS= read -r var

-r is to stop read from treating backslash characters specially (as an escape character for separators and newline), And IFS= to set the list of separators to the empty string for read (otherwise if any whitespace character was in that list, they would be stripped from the beginning and end of the input).

Using loops in shells is usually a bad idea (not how things are done in shells where you make several tools work collectively and concurrently to a task rather than running one or more tools hundreds of times in sequence).

0
14

There's no need for command substitution if you want to use pipe. read reads from stdin by default, so you just pipe into it:

find . -type f -print0 |
while read -r -d '' line
do
    echo "$line"
done

or in one line

find . -type f -print0 | while read -r -d '' line; do echo "$line"; done

Using -print0 and -d '' is a protective measure against potential newline character in filenames, which by default indicates end of line to read (this can be changed with -d). '' used above is effectively null byte in bash (equivalent to $'\0').

Another already mentioned approach is to use find's -exec option. Most likely that would be the best option performance-wise (remember to use -exec cmd {} + variant so that it won't fork a process for each line being processed). But it really depends on what you do inside the while loop and how much data you process. The difference might be negligible or acceptable for your case.

0
1

Remember that invoking commands has a cost (irrespective of what they do). This can have very negative impact on performance if you put them into a loop that will have multiple iterations. In case you need to process the output in some way, consider using awk. It is very powerful and can quickly handle a lot of input:

find . -type f | awk '{print}'

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.