18

When I set the IFS variable to a space, bash treats multiple spaces as one space (myprogram is a program that prints the command line arguments it receives):

IFS=" "
x="hello   hi   world"
./myprogram $x
argv[1] = hello
argv[2] = hi
argv[3] = world

But when I set the IFS variable to a comma, bash does not treat multiple commas as one comma:

IFS=","
x="hello,,,hi,,,world"
./myprogram $x
argv[1] = hello
argv[2] = 
argv[3] = 
argv[4] = hi
argv[5] = 
argv[6] = 
argv[7] = world

Why is that?

1
  • Just for reference, "IFS" means Internal Field Separator.
    – pr1268
    Dec 30, 2017 at 13:36

1 Answer 1

20

This is documented in man bash. A single occurrence of any character in IFS that is not whitespace delimits a field.

From man bash:

The shell treats each character of IFS as a delimiter, and splits the results of the other expansions into words using these characters as field terminators. If IFS is unset, or its value is exactly <space><tab><newline>, the default, then sequences of <space>, <tab>, and <newline> at the beginning and end of the results of the previous expansions are ignored, and any sequence of IFS characters not at the beginning or end serves to delimit words. If IFS has a value other than the default, then sequences of the whitespace characters space, tab, and newline are ignored at the beginning and end of the word, as long as the whitespace character is in the value of IFS (an IFS whitespace character). Any character in IFS that is not IFS whitespace, along with any adjacent IFS whitespace characters, delimits a field. A sequence of IFS whitespace characters is also treated as a delimiter. If the value of IFS is null, no word splitting occurs. [Emphasis added.]

Examples: field splitting

If IFS has no whitespace characters, then whitespace is included in the fields:

$ ( IFS=',' x='one , two,three'; printf "<%s>\n" $x )
<one >
< two>
<three>

If IFS has both blanks and a comma, then sequences of blanks, followed by a comma, followed by sequences of blanks are treated as a single delimiter:

$ ( IFS=' ,' x='one , two,three'; printf "<%s>\n" $x )
<one>
<two>
<three>

Sequences of commas are interpreted as sequences of empty fields:

$ ( IFS=' ,' x='one,,,two,three'; printf "<%s>\n" $x )
<one>
<>
<>
<two>
<three>

Examples: leading and trailing whitespace

If IFS contains no whitespace, then any leading and trailing whitespace is kept in the fields:

$ ( IFS=',' x='  one , two,three  ,'; printf "<%s>\n" $x )
<  one >
< two>
<three  >

If IFS does contain blanks, then any leading or trailing sequences of blanks are removed:

$ ( IFS=' ,' x='  one , two,three  ,'; printf "<%s>\n" $x )
<one>
<two>
<three>
5
  • maybe also worth emphasizing "then sequences of the whitespace characters space, tab, and newline are ignored at the beginning and end of the word, as long as the whitespace character is in the value of IFS"
    – Jeff Schaller
    Dec 30, 2017 at 2:49
  • @JeffSchaller Excellent idea: I just added a section on that.
    – John1024
    Dec 30, 2017 at 3:27
  • what if you have a tab-separated file with some missing values? i.e. you don't want sequences of tabs to be treated as a single tab. Also, the fields contain commas so can't use that as a delimiter. Is the only solution to use some other delimiter (not tabs)?
    – Davos
    Aug 27, 2018 at 7:42
  • @Davos For data with each field delimited by a single tab, it may be more natural to use other tools that handle this easily such as awk with the -F'\t' option or cut. Alternatively, if you have a recent version of bash, you may be able to parse the fields using readarray with the -d$'\t' option.
    – John1024
    Aug 28, 2018 at 1:26

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .