1

I have following script which is supposed to solve the problem from the title, but obviously it isn't working to assign the values to the keys. Is the reason an accidental error or is there a substantial mistake in the script?

The foldernames are names of gem-files like gem-file-foo-1.2.3

The key is supposed to be gem-file-foo in this example and the value(s) the version number 1.2.3 or a string of multiple version numbers if there are multiple versions of the same gem.

It doesn't output any keys with echo "${!my_gems[@]}" ... why not?

#!/bin/bash

directory=$GEM_HOME/gems
declare -A my_gems

get_gemset_versions () {

last_key=""
values=""

FIND_RESULTS=$(find $directory -maxdepth 1 -type d -regextype posix-extended -regex "^${directory}\/[a-zA-Z0-9]+([-_]?[a-zA-Z0-9]+)*-[0-9]{1,3}(.[0-9]{1,3}){,3}\$")

 printf "%s\n" $FIND_RESULTS | sort | 
  while read -r line; do
    line=${line##*/}
            
    KEY="${line%-*}"
            
    VALUE="${line##*-}"
  
        if [[ $last_key -eq "" ]]; then
            last_key=$KEY
        fi
        
        if [[ $last_key -eq $KEY ]]; then
            values="$values ${VALUE}"
        else
            values="${VALUE}"
            last_key=$KEY
        fi
        
        my_gems[$KEY]=$values
    done
 
    echo "${!my_gems[@]}"

 }


get_gemset_versions

Also, the logic with $last_key and $key to summerize equal gem-packages seems to be faulty. This is not necessarily part of the question, but it would be nice if you would point out if I am applying some faulty logics here.

Thanks

8
  • 1
    -eq is an integer equality operator; string equality is = (POSIX) or == (bash extended test) May 29, 2021 at 12:41
  • Thanks. It seems like the input to $my_gems[key] happened just locally and isn't available anymore after the while loop ???
    – von spotz
    May 29, 2021 at 13:33
  • Ok thanks. Could you maybe please tell me how I can save the output from printf "%s\n" $FIND_RESULTS | sort in a variable such that I can use the redirecting syntax to give input to the while loop? Or how I can manipulate the process substitution syntax to sort the result-set of find (see the second solution here stackoverflow.com/a/67739521/4307872) ?
    – von spotz
    May 29, 2021 at 13:51
  • @vonspotz, there's a few workarounds in the answers to the question steeldriver linked. (Also, $(...) is command substitution, not process substitution which is a different thing.)
    – ilkkachu
    May 29, 2021 at 13:53

1 Answer 1

3

You have:

printf "%s\n" $FIND_RESULTS | sort | 
  while read -r line; do
    ...
    done
 
    echo "${!my_gems[@]}"

where, regardless of the indentation, the echo is outside the pipeline. Bash runs all parts of a pipeline in subshells by default, so the assignments within the while loop aren't visible after the pipeline ends. Shellcheck.net also warns about that:

Line 32:
        my_gems[$KEY]=$values
        ^-- SC2030: Modification of my_gems is local (to subshell caused by pipeline).

Sadly it doesn't give workarounds.

In Bash, you can either enable the lastpipe option, or replace the pipe with process substitution:

shopt -s lastpipe
echo test | while read line; do
    out=$line
  done
echo "out=$out"

or

while read line; do
  out=$line
done < <(echo test)
echo "out=$out"

(lastpipe probably won't work if you try it in an interactive shell, as it's tied to job control not being enabled.)

See: Why is my variable local in one 'while read' loop, but not in another seemingly similar loop?


In any case, this seems a bit odd:

FIND_RESULTS=$(find ...)

 printf "%s\n" $FIND_RESULTS 

find outputs filenames separated by newlines, which is fine as long as you know no filenames contain any. But here, the round-trip through the variable and the word splitting from the unquoted expansion also splits any filenames with spaces.

You could just run find ... | while ... directly. Or while ...; done < <(find...).

Also, note that you almost always want to use while IFS= read -r line; do, to prevent read from breaking leading and tailing whitespace. Well, I hope your filenames don't contain those either, but in any case.

I can't find a good reference question right now, but that's specific to IFS containing whitespace. Other leading and trailing separators are not removed with a read to just one field. E.g. IFS=": " read -r foo <<< "::foobar " leaves foo with the literal ::foobar. The colons are kept, but the trailing spaces gone.

1
  • you can use grouping braces: pipeline | { while read line; do out=$line; done; echo $line; } but that only extends the availability of the variable a little bit. May 31, 2021 at 15:24

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .