11

I have a file in the following format and it is tab separated

a   k   testis  adult   male    8 week  rRNA
b   k   testis  adult   male    8 week  rRNA
c   k   testis  adult   male    8 week  rRNA

I want to do some operation on each line so I am using a while loop.I want to split each line on tab and then store let's say 6th column which is 8 week in a variable. I am using this code but I am not able to get what I want

while read -r line; do tmp=(${line///}); col6=${tmp[5]}; echo "$col6"; done < file.txt

This gives me 8 and not 8 week. 8 week has a space in between 8 and week and hence I want to split the line on tab.

12

1 Answer 1

18

The array assignment tmp=(${line///}) splits the value on whatever characters IFS contains, which by default includes tabs, and spaces and newlines. (I don't see what the empty substitution does.) To split only on tabs, set IFS to this:

foo=$'a\tk\testis\tadult\tmale\t8 week\tRNA'
IFS=$'\t'
tmp=($foo)
echo "${tmp[5]}"

Though that still leaves globbing as an issue, and since you are already using while read, you could use read -a tmp (in Bash only, replace -a with -A with ksh/zsh/yash), it splits the input line based on IFS, and stores the resulting fields as elements of the named array:

$ while IFS=$'\t' read -r -a tmp ; do
    echo "${tmp[5]}"
done <<< $'a\tk\testis\tadult\tmale\t8 week\tRNA'

That prints 8 week. The other upside with this is that the change in IFS is only in effect for the duration of the read, not for the rest of the script.

Beware however that read strips empty fields when using tab as a delimiter. In zsh, you could replace IFS=$'\t' with IFS=$'\t\t' to stop that from happening.

Of course, if we know the number/meaning of the fields, we could just have read split them to separate named variables:

... IFS=$'\t' read -r col1 col2 col3 ...

Or, if you only want to print that one column, use cut:

cut -d$'\t' -f 6  < file.txt

If you have empty columns, cut -d$'\t' and IFS=$'\t' have different behaviour with regard to them. Cut will treat each individual tab as a distinct separator, while read will take consecutive tabs as just a single separator. That is, the string foo<tab><tab>bar will be read as just two columns by read, but three columns by cut.

You can't change this for tabs, but printing characters are always recognized as distinct separators, so you could change the tabs to some character that doesn't appear in the data, and then use that as the separator, e.g. ... | tr '\t' : | IFS=: read -r -a tmp or so.

8
  • can you tell what -a tmp is doing?? Dec 13, 2017 at 18:20
  • 2
    @user3138373 the -a flag tells read to store what it reads as an array.
    – terdon
    Dec 13, 2017 at 18:23
  • This will not work if you have mutliple consequent empty columns as described here stackoverflow.com/questions/9736202/…
    – giorgio79
    Jan 17, 2021 at 8:41
  • @giorgio79, yes, you're right, because space and tab in IFS are treated differently from other characters (newline too, maybe, but it doesn't matter for read here). If there are empty columns (which there aren't in the question here), you'd need to do something like ... | tr '\t' : | IFS=: read -r -a tmp to not have the separator be tab or space.
    – ilkkachu
    Jan 17, 2021 at 11:17
  • Thanks @ikkachu I went with a script meanwhile. For reference, I bumped into this issue with the geonames cities dump that is tab separated but has lots of empty cols.
    – giorgio79
    Jan 17, 2021 at 12:42

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .