Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I might have something absolutely wrong, but it looks convincing to me, that setting IFS as one of the commands in the pre-do/done list has absolutely no effect.
The outer IFS (outside the while construct) prevails in all examples shown in the script below..

What's going on here? Have I got the wrong idea of what IFS does in this situation? I expected the array-split results to be as shown in the "expected" column.


#!/bin/bash
xifs() { echo -n "$(echo -n "$IFS" | xxd -p)"; } # allow for null $IFS 
show() { x=($1) 
         echo -ne "  (${#x[@]})\t |"
         for ((j=0;j<${#x[@]};j++)); do 
           echo -n "${x[j]}|"
         done
         echo -ne "\t"
         xifs "$IFS"; echo
}
data="a  b   c"
echo -e "-----   --  -- \t --------\tactual"
echo -e "outside        \t  IFS    \tinside" 
echo -e "loop           \t Field   \tloop" 
echo -e "IFS     NR  NF \t Split   \tIFS (actual)" 
echo -e "-----   --  -- \t --------\t-----"
IFS=$' \t\n'; xifs "$IFS"; echo "$data" | while         read; do echo -ne '\t 1'; show "$REPLY"; done 
IFS=$' \t\n'; xifs "$IFS"; echo "$data" | while IFS=    read; do echo -ne '\t 2'; show "$REPLY"; done 
IFS=$' \t\n'; xifs "$IFS"; echo "$data" | while IFS=b   read; do echo -ne '\t 3'; show "$REPLY"; done
IFS=" ";      xifs "$IFS"; echo "$data" | while         read; do echo -ne '\t 4'; show "$REPLY"; done 
IFS=" ";      xifs "$IFS"; echo "$data" | while IFS=    read; do echo -ne '\t 5'; show "$REPLY"; done 
IFS=" ";      xifs "$IFS"; echo "$data" | while IFS=b   read; do echo -ne '\t 6'; show "$REPLY"; done
IFS=;         xifs "$IFS"; echo "$data" | while         read; do echo -ne '\t 7'; show "$REPLY"; done 
IFS=;         xifs "$IFS"; echo "$data" | while IFS=" " read; do echo -ne '\t 8'; show "$REPLY"; done 
IFS=;         xifs "$IFS"; echo "$data" | while IFS=b   read; do echo -ne '\t 9'; show "$REPLY"; done
IFS=b;        xifs "$IFS"; echo "$data" | while IFS=    read; do echo -ne '\t10'; show "$REPLY"; done
IFS=b;        xifs "$IFS"; echo "$data" | while IFS=" " read; do echo -ne '\t11'; show "$REPLY"; done
echo -e "-----   --  -- \t --------\t-----"

Output:

-----   --  --   --------       actual   
outside           IFS           inside                assigned   
loop             Field          loop    #              inner
IFS     NR  NF   Split          IFS     #  expected    IFS
-----   --  --   --------       -----   #  ---------  --------
20090a   1  (3)  |a|b|c|        20090a  #                              
20090a   2  (3)  |a|b|c|        20090a  #  |a  b   c|  IFS=
20090a   3  (3)  |a|b|c|        20090a  #  |a  |   c|  IFS=b
20       4  (3)  |a|b|c|        20      #                          
20       5  (3)  |a|b|c|        20      #  |a  b   c   IFS=
20       6  (3)  |a|b|c|        20      #  |a  |   c|  IFS=b
         7  (1)  |a  b   c|             #                          
         8  (1)  |a  b   c|             #  |a|b|c|     IFS=" "
         9  (1)  |a  b   c|             #  |a  |   c|  IFS=b
62      10  (2)  |a  |   c|     62      #  |a  b   c|  IFS=
62      11  (2)  |a  |   c|     62      #  |a|b|c|     IFS=" "
-----   --  --   --------       -----      ---------   -------                        
share|improve this question

2 Answers 2

up vote 9 down vote accepted

(Sorry, long explanation)

Yes, the IFS variable in while IFS=" " read; do … has no effect on the rest of the code.

Let's first precise that in shell's command-lines there are usually two different kinds of variables involved:

  • shell variables (which only exist within a shell, and are local to the shell)
  • environment variables, which exist for every process. Those are usually preserved upon fork() and exec(), so child processes inherit them.

When you call a command with:

  A=foo B=bar command

the command is executed in within an environment where (environment) variable A is set to foo and B is set to bar. But with this command line, the current shell variables A and B are are left unchanged.

This is different from:

A=foo; B=bar; command

Here, A and B shell variables are defined and the command is run without A and B environment variables defined. Values of A and B are unaccessible from command.

Though, if the shell variables are export-ed, the corresponding environment variables are synchronized with their respective shell variables. Example:

export A
export B
A=foo; B=bar; command

In this code, both shell variables and the shell's environment variables are set to foo and bar. Since environment variables are inherited by sub-processes, command will be able to access their values.

To jump back to your original question, in:

IFS='a' read

only read is affected. And in fact, in this case, read don't care about the value of the IFS variable. It uses IFS only when you ask the line to be split (and stored in several variables), like in:

echo "a :  b :    c" | IFS=":" read i j k; \
    printf "i is '%s', j is '%s', k is '%s'" "$i" "$j" "$k"

IFS is not used by read unless it is called with arguments. (Edit: This is not exactly true: whitespace characters, i.e. space and tab, present in IFS are always ignored at the beginning/end of the input line. )

share|improve this answer
    
What a great explanation! It is so simple! I've been bemused by that 'no semi-colon' syntax for months; and it is simply a case of it meaning a local variable!.. rozcietrzewiacz opened the pathway for me (big-time) in the other question ... and you have just put the icing on the cake... I've been up all night on this one, and it has certainly been worth it for such good and clear answers! .. Thank you.. –  Peter.O Aug 17 '11 at 21:04
    
Uhm. I had to read that edit comment several times before I got it – you mean to say that whitespace characters which are present in $IFS are removed at the beginning/end of the input line, I presume? (Which is how it works.) –  zrajm Jul 28 at 5:22

Put it simple, you must read to more than one variable at a time for the IFS=<something> read ... construct to have a visible effect in your examples1.

You miss the scope of read in the examples. There is no modification of IFS inside the loop in your test cases. Allow me to point exactly, where does the second IFS have its effect in each of your lines:

 IFS=$' \t\n'; xifs "$IFS"; echo "$data" | while IFS=b   read; do echo ...
                                                      ^      ^
                                                      |      |
                                          from here --'       `- to here :)

It is just as with any program executed in the shell. The variable you (re)define at the command-line affects the program execution. And only that (since you do not export). Therefore, to make a use of redefined IFS in such line, you'd have to ask read to assign values to more than one variable. Have a look a these examples:

 $ data="a  b   c"
 $ echo "$data" | while           read A B C; do echo \|$A\|$B\|\|$C\|; done
 |a|b||c|
 $ echo "$data" | while IFS=      read A B C; do echo \|$A\|$B\|\|$C\|; done
 |a b c||||
 $ echo "$data" | while IFS='a'   read A B C; do echo \|$A\|$B\|\|$C\|; done
 || b c|||
 $ echo "$data" | while IFS='ab'  read A B C; do echo \|$A\|$B\|\|$C\|; done
 || || c|

1 As I've just learned from Gilles, there might actually be a benefit of setting IFS='' (blank) when reading only one field: it avoids truncation of whitespace at the beginning of the line.

share|improve this answer
    
Good.. Thanks... I got it this time.. and I love your sketch :) –  Peter.O Aug 17 '11 at 20:50
    
OK, now I've read your comment that see you haven't noticed my answer to that issue in the other question. Maybe you could just revert the other one and delete this, since it really is one general issue? –  rozcietrzewiacz Aug 17 '11 at 20:52
    
Yes, the two questions do have a related theme, but the title of the other one is "Why is IFS= read used in preference to just re-setting the IFS environment variable". I didn't have awareness, then, that local variables could be set by the caller of a command. That was the answer to that question. It did evolve further to addressing this question's main point, but by the time I realized that, I had already asked this question... Perhaps the two questions are as similar as two sed questions, so my feeling it to keep it asis... More titles for googlers to google. –  Peter.O Aug 17 '11 at 21:47

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.