5

I would assume that setting IFS='X' would cause bash to split the string fooXbarXbaz into three words: foo, bar, and baz.

However, it only works if the string is provided to the for-loop through command substitution: $(echo fooXbarXbaz):

$ IFS='X'; for x in fooXbarXbaz; do echo Y${x}Z; done
Yfoo bar bazZ
$ IFS='X'; for x in $(echo fooXbarXbaz); do echo Y${x}Z; done
YfooZ
YbarZ
YbazZ

Can someone explain why the first example command fails to split fooXbarXbaz into three words, while the second example is successful?

1
  • You can also use IFS='X'; vars=fooXbarXbaz; for x in $vars; do echo Y${x}Z; done May 20, 2020 at 11:24

3 Answers 3

8

$IFS is only used for word splitting after unquoted expansions. There is no expansion in for x in fooXbarXbaz. There is however an unquoted expansion in echo Y${x}Z, which means that echo is called with the three arguments Yfoo, bar and bazZ. The echo utility prints each of its arguments with a space in-between, so you get Yfoo bar bazZ.

In the second example, the string is the output of a command substitution, which is an expansion, so the string is split for the loop.

The IFS variable is also used for splitting the input to the read utility.

5
  • 1
    Does IFS-based word splitting only occur after expansions in general?
    – Shuzheng
    May 20, 2020 at 11:04
  • @Shuzheng Yes, correct. I have corrected the text to say "unquoted expansions". Quoted expansions are not split. Compare echo $s and echo "$s" with IFS=X; s=helloXworld
    – Kusalananda
    May 20, 2020 at 11:05
  • @Shuzheng (If I'm correctly reading your question) the (only, AFAIK) use of IFS for splitting that may not be occurring after an expansion is with the read utility. Try playing with (IFS=X; read x y; echo "x: $x"; echo "y: $y").
    – fra-san
    May 20, 2020 at 11:22
  • @fra-san That is a correct reading of the manual. I did not mention this as it was not applicable on this particular question, the way I read it. That's splitting input rather than words in a variable or a string.
    – Kusalananda
    May 20, 2020 at 11:30
  • @Kusalananda I agree. Actually, my "If I'm correctly reading your question" only referred to the question in the OP's comment to your answer - I haven't been really clear, sorry.
    – fra-san
    May 20, 2020 at 11:35
4

Can someone explain why the first example command fails to split fooXbarXbaz into three words

Because shells aren't supposed to work like that any more.

In some ancient times, word splitting happened for plain words too, e.g. with heirloom-sh:

heirloom-sh$ for foo in 123o456o789; do echo $foo; done
ech: not found
ech: not found
ech: not found

Oops, I mean:

heirloom-sh$ for foo in 123o456o789; do "echo" $foo; done
123
456
789

(Even there the keywords and variable names didn't get split on the os.)

While using o or X in IFS is a bit extreme, it's probably for the best that splitting now only happens for expansions.

If you use splitting on purpose, remember that unquoted expansions are also processed for globbing, so things like * may cause surprises. Arrays usually work better for handling multiple values.

3

The GNU bash documentation clearly explains this behavior under Command Substitution

If the substitution appears within double quotes, word splitting and filename expansion are not performed on the results.

Since the second case has a a unquoted substitution, the resultant string fooXbarXbaz is subjected to word splitting by the shell by the value of IFS set, which splits the word into multiple words.

In the first case for x in fooXbarXbaz;, the string is subjected to pathname expansion (aka glob expansion). The shell performs this at a stage much later than word-splitting, after finishing all possible expansions (see Shell Expansions), so the resultant string is treated literally.

To add further take the case of variable expansion $var, which happens before pathname expansion, undergoes same process like your OP

IFS='X'; var=fooXbarXbaz; for x in $var; do echo Y${x}Z; done

An unquoted variable expansion like command substitution undergoes word splitting by the value of IFS. That's why generally we insist on quoting all expansions to avoid unforseen word splitting done by the shell.

1
  • 1
    Also remember that an unquoted expansion is split + glob. If you only want the splitting, you need to remove the quotes, but also to disable globbing. So: IFS=X; set -o noglob; for x in $var; do printf '%s\n' "Y${x}Z"; done (also remember: don't use echo to display arbitrary content). May 20, 2020 at 14:22

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .