2

I'm working on a simple bash script that fixes a duplicate naming issue on a retropie.

The script itself grabs any name that is mentioned more than once in a gameslist.xml file, then stores those in an array for later use.

I end up looping over this array in indexes like so:

pi@retropie:~ $ for game in ${game_array[@]:0:10} ; do echo $game; done

which pulls the 1st to the 10th element (i.e. ${game_array[9]}) however the output gets joined to one line:

pi@retropie:~ $ for game in ${game_array[@]:0:10} ; do echo $game; done
R.B.I. Baseball '94 World Series Baseball '95 Mega Games 1 Bill Walsh College Football T2: The Arcade Game Sonic & Knuckles + Sonic the Hedgehog Sega Top Five Pyramid Magic Tecmo Super Baseball Super Chinese Tycoon

But if I loop over the entire array, it outputs on new lines as expected:

pi@retropie:~ $ for game in ${game_array[@]}; do echo $game; done | head -10
R.B.I. Baseball '94
World Series Baseball '95
Mega Games 1
Bill Walsh College Football
T2: The Arcade Game
Sonic & Knuckles + Sonic the Hedgehog
Sega Top Five
Pyramid Magic
Tecmo Super Baseball
Super Chinese Tycoon

the field seperator has been set to new line with IFS='$\n' which is why the second one works, but I can't for the life of me figure out why it isn't with the first one?

here's the full test script for context:

#!/bin/bash

user_input=$1
while [ -z "$user_input" ]; do
        echo "please enter the name of the system you want to fix the game list for"
        echo "(as it is labelled in /home/pi/RetroPie/roms)"
        read -r user_input
done

ls "/home/pi/RetroPie/roms/$user_input" >/dev/null 2>&1

if  [ "$?" -ne 0 ]; then
        echo "this doesn't appear to be a system installed here. exiting."
        exit 1
fi

games_to_fix()
{
        IFS=$'\n'
        console=$1
        filepath="/opt/retropie/configs/all/emulationstation/gamelists/$console/gamelist.xml"
        game_array=($(fgrep "<name>" "$filepath" | sort | uniq -c | sort -rn | awk  '$1 > 1 {print $0}'| cut -d ">" -f 2 | cut -d "<" -f 1))
        number_to_fix=($(fgrep "<name>" "$filepath" | sort | uniq -c | sort -rn | awk  '$1 > 1 {print $1}'))
}

get_new_name()
{
        mYpath=$1
        new_name=$(echo $mYpath | cut -d ">" -f 2 | cut -d "<" -f 1 | sed -e 's/\.\///g' | sed -e 's/\.7z//g')
}

games_to_fix $user_input

IFS=$'\n'
index=0
for i in ${number_to_fix[@]}; do
        loop=1
        for game in ${game_array[@]:$index:$i}; do
        #       for ind in $(eval echo {1..$i}); do
                line_number=$(fgrep -n "<name>$game</name>"  $filepath | awk '{print $1}' | cut -d : -f 1 | sed -e "${loop}q;d")
                path_line_number=$(expr $line_number - 1 )
                path=$(sed "${path_line_number}q;d" $filepath | cut -d : -f 2)
                get_new_name "$path"
                sed -i "${line_number}s/$game/$new_name/g" $filepath
                ((loop++))
        done
        index=$(expr index + $i);
done
  • I'm sceptical. What's in the array? declare -p game_array – glenn jackman Jul 31 '18 at 17:47
  • Can you try to reduce your code to a minimal testable example that demonstrates the issue? as it stands, it's rather hard to understand exactly what you are asking. Are you sure the problem cannot be solved by proper quoting - for example arr=("foo bar" "baz" "bam foo" "bar") ; for item in "${arr[@]:1:2}"; do echo "$item"; done – steeldriver Jul 31 '18 at 17:56
3

In brief: you should use quotes around array expansions like that, unless you explicitly want field/word splitting. "$@" expands each positional parameter to a separate word, and similarly so does "${a[@]}". By extension, that should work in the same way for "${a[@]:0:2}".


That said, it still seems like an inconsistency in Bash, and what you used should work in your case (since there are no glob characters in the values, and field splitting is taken care of by IFS being set properly).

Taking the full array works:

$ IFS=$'\n'
$ a=("foo bar" "asdf ghjk")
$ printf "%s\n" ${a[@]}
foo bar
asdf ghjk

Slicing it doesn't work for an array, but works for $@:

$ printf "%s\n" ${a[@]:0:2}
foo bar asdf ghjk

$ set -- "aa bb" "cc dd"
$ printf "%s\n" ${@:1:2}
aa bb
cc dd

It does work in ksh and zsh, which sort of underlines Bash being inconsistent here (zsh of course would have its own syntax for the equivalent):

$ ifs=$'\n' ksh -c 'IFS="$ifs"; a=("foo bar" "asdf ghjk"); printf "%s\n" ${a[@]:0:2}'
foo bar
asdf ghjk
$ ifs=$'\n' zsh -yc 'IFS="$ifs"; a=("foo bar" "asdf ghjk"); printf "%s\n" ${a[@]:0:2}'
foo bar
asdf ghjk

The quoted version works in Bash, too, and is better when you just want the values as-is, since you don't need to depend on IFS. The default IFS works fine here even though the array elements have spaces:

$ unset IFS                         # uses default of $' \t\n'
$ printf "%s\n" "${a[@]:0:2}"
foo bar
asdf ghjk

It looks as though the unquoted ${a[@]:0:2} joins the elements with spaces, a bit like what happens in Bash in contexts where word-splitting doesn't happen (e.g. str=${a[@]}). And then it tries to split the result with IFS, as usual. E.g. here, it splits on the newline in the middle of the second array element:

$ IFS=$'\n'
$ a=("foo bar" $'new\nline' "asdf ghjk");
$ printf ":%s\n" ${a[@]:0:3}
:foo bar new
:line asdf ghjk

As said above, you really should use the quotes around array expansions in most cases, though still, one would assume ${a[@]:n:m} would result in multiple words, just like ${a[@]} does.

The behaviour here seems present in Bash 4.4.12(1)-release and 5.0.0(1)-alpha. I posted a bug report about it.

  • In zsh, you'd rather use $a[1,2] (or "${(@)a[1,2]}" or "${a[@][1,2]}" to preserve empty elements). – Stéphane Chazelas Aug 1 '18 at 12:11
  • @StéphaneChazelas, doesn't "${a[@]:1:2}" work fine in zsh too? Of course zsh has its own way of doing that, I'm not arguing against that. Just using zsh as a point of comparison for something I see as a bug in Bash (even though the whole thing is something that probably shouldn't usually be used) – ilkkachu Aug 1 '18 at 13:23
  • Yes, I'm just saying there are more idiomatic/native/sensible ways to do it in zsh (${var:x:y} was added much later in zsh and only for compatibility with ksh/bash as it had its own better way to do it before them). bash copied it from ksh93, you may want to check the status there. – Stéphane Chazelas Aug 1 '18 at 13:26
  • Using an unquoted ${a[@]} causes the problem, and the problem is not a bug. For example: from man ksh The meaning of $ and $@ is identical when not quoted*. Both unquoted expansions should become one string separated by spaces. – Isaac Aug 2 '18 at 7:56
  • @Isaac, no, they shouldn't. Bash's reference puts it in plain text re $*: "($*) Expands to the positional parameters, starting from one. When the expansion is not within double quotes, each positional parameter expands to a separate word. " and the text in POSIX is similar (for $@): "Expands to the positional parameters, starting from one, initially producing one field for each positional parameter that is set. When the expansion occurs in a context where field splitting will be performed, ... each of the non-empty fields shall be further split as described in Field Splitting." – ilkkachu Aug 2 '18 at 8:19
2

quote
quote

Quote

Quote

Quote!!

This works:

$ game_array=("foo bar" "baz" "bam foo" "bar")

$ for game in "${game_array[@]:0:10}" ; do echo "$game"; done
foo bar
baz
bam foo
bar

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.