3

How to set IFS to two bytes value in Bash ?

IFS=',;'

will delimit each part of which resides between two simple , and/or simple ; instead of ,; as a delimiter
How is emulated/workaround way, so that gives a solution? Thanks before

  • 1
    It's not clear (to me) whether you want either , or ; to be a delimiter, or the two character combination ,; to be the delimiter – roaima Oct 18 at 17:14
  • 2
    @roaima since they describe that using IFS='.,' will "delimit each part of which resides between two simple , and/or simple ; instead of ,; as a delimiter", I think the OP wants to set the string ,; as the delimiter. Hence the "two bytes" in the title. – terdon Oct 18 at 17:20
  • @Terdon I still can't see it. word ,; word ,; word or word , word ; word? – roaima Oct 18 at 17:22
  • 1
    Presumably, word ,; word ,; word @roaima, otherwise the IFS=',;' which they have in the question would work. – terdon Oct 18 at 17:25
  • 1
    Be careful not to confuse characters and bytes, since there are multi-byte characters. In your example, a character is a byte, but an ö may be a multi byte character in some locales. – schily Oct 18 at 19:26
1

IFS is a set of single-character separators, so with IFS=,;, either of ; or , would work as a separator, and a,b,;c;d would have five fields. If you want to use just the combination ,; as a single separator, you'll have to do it manually. One way is to replace that ,; pair with some single character you then do put in IFS:

s='a,b,;c;d'
IFS=#
fields=(${s//,;/#})

${s//,;/#} replaces all substrings ,; with #, and the unquoted expansion then splits the result. Now the array fields contains a,b and c;d. Note that it will also use the resulting words as globs (filename wildcards). You may want to prevent that with set -f / set -o noglob, but note that as well assigning to IFS, that has a global effect.

Or you could use sed, especially if you have a pipe there to begin with:

sed -e 's/,;/#/g'
| improve this answer | |
  • @StéphaneChazelas, agh, I meant to put the @ there. But no matter, you'd already posted the Zsh solution. :) – ilkkachu Oct 18 at 18:25
2

You could switch to zsh instead of bash and use its s (for split) parameter expansion flag:

$ string='foo,;bar,;,;baz'
$ words=("${(@s[,;])string}")
$ typeset -p words
typeset -a words=( foo bar '' baz )

Note that it's splitting not delimiting, foo,; would be split into foo and the empty string, not just foo like bash's IFS splitting would (with single characters only).

Also note that in bash (and zsh, but not all shells), word splitting is done on the characters of $IFS, not bytes. For instance, with IFS='é', Stéphane would be split into St and phane even in locales where é is encoded on two bytes (like in those locales where the charmap is UTF-8, the most common these days).

| improve this answer | |
  • A shell that does not support multi-byte characters may be called basically POSIX compliant for tiny embedded systems, but is "illegal" on a platform that claims UNIX compatibility. So using dash as system shell degrades a platform. BTW: ksh88. ksh93, yash the Bourne Shell and bash support multi byte characters and thus allow things like IFS=ö on multi byte locales. BTW: my test script for multi byte support fails with dash, mksh and zsh. – schily Oct 18 at 18:53
  • @schily, note that both ksh93 and bosh fail on bosh -c 'IFS=é; echo "$*"' sh St phane (in multibyte locales (not tested with the latest bosh though)). – Stéphane Chazelas Oct 18 at 19:37
  • @schily, still a problem with bosh 2020/10/07 a+. – Stéphane Chazelas Oct 18 at 19:46
  • I am not sure what you like to test with that command, but you need to remove the double quotes for a test for multi byte support and you would need to write your name completely including the é. Result: one space -> multi byte support present. two spaces -> no multi byte support.BTW: the Bourne Shell added the basics for multi byte support in 1988 and completed it in 1993 – schily Oct 18 at 19:53
2

A bash (version 4.3+) function:

split() {
    local string=$1 fs=$2
    local -n fields=$3
    fields=()
    while [[ $string =~ (.*)"$fs"(.*) ]]; do
        fields=( "${BASH_REMATCH[2]}" "${fields[@]}" )
        string=${BASH_REMATCH[1]}
    done
    fields=( "$string" "${fields[@]}" )
}

usage:

$ string="field1,;field2,field2b,;field3a;field3b,;,;field4"
$ split "$string" ",;" result
$ declare -p result
declare -a result=([0]="field1" [1]="field2,field2b" [2]="field3a;field3b" [3]="" [4]="field4")

It will fail, like many naive attempts to implement CSV parsing, with the separator enclosed in quotes:

$ split 'Thoughtfully, he said "Hello, friend."' , x
$ declare -p x
declare -a x=([0]="Thoughtfully" [1]=" he said \"Hello" [2]=" friend.\"")
| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy