I have a text.txt file like this


I want to write a script that loops over each line and echo out

modified line1
modified line2
modified line3

This is the script which is a very common solution:

while IFS= read -r line; do
  echo modified $line
done <<< $(cat ~/text.txt)

But the output I got was:

modified line1 line2 line3

What went wrong?

  • 2
    the <<< $(cat ...) command substitution + here string is completely unnecessary and messing things up, but I guess that you're using some older (< 4.0?) version of bash, which will ifs-split+glob the word following <<<. – mosvy May 25 '19 at 10:08
  • Thanks for pointing out this. Now that I think about it, it's definitely unnecessary. But could you explain what you meant by ifs-split+glob the word following <<<? – Tran Triet May 25 '19 at 10:26
  • 1
    The shell will split unquoted variables using the characters from the IFS variable and then glob-expand the resulting words (eg. a='* .*'; echo $a). This should not happen on a variable used as a here-string (eg. a='* .*'; cat <<<$a should print just * .*), but it does happen on some older buggy versions of bash. – mosvy May 25 '19 at 11:26

The issue is in the last line, you don't need the variable (command substitution) or cat, since read already can read the file. If you instead do this:

while IFS= read -r line; do
  echo modified $line
done < ~/text.txt

It works.

Additionally, your command would work if you quoted the variable like:

"$(cat ~/text.txt)"

since bash disregards newlines in variables unless you quote them. But doing it this way is overcomplicating it.

  • what is "the variable of cat"? – mosvy May 25 '19 at 10:03
  • @mosvy I don't know. I never mentioned "the variable of cat". I said he doesn't need the cat command or the variable (the result of the cat command) in the last line. – DisplayName May 25 '19 at 10:17
  • This really works. Thank you. I understand that using command substitution and heredoc is not necessary here, but I don't understand why that combination doesn't work. Would you mind explaining that? – Tran Triet May 25 '19 at 10:24
  • 2
    @TranTriet bash disregards newlines in variables unless you quote them. If you had you used double quotes around the command like "$(cat /tmp/text.txt)" it would work, but this is overcomplicating it. – DisplayName May 25 '19 at 10:31
  • 1
    This answer is correct, but it doesn't address why the command substitution is a problem (or a potential problem) here. I'd recommend expanding the answer to explain that. Also, it seems this problem only exists in older versions of baah, so including that information would be very useful as well. – filbranden May 25 '19 at 12:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.