1

I have a bash script (seen here) that I'm using to sort a list of files in subdirectories by filename.

This script appends sorted files in hooks/pre-relevant/, then files named hooks/pre-relevant, then files named hooks/relevant, then sorted files in hooks/relevant/, using the append_hook function below:

hooks=()

# Run a hook script, if it's executable, with the input from this invocation
append_hook () {
  if [[ -f "$1" && -x "$1" ]]; then
    hooks+=("$1")
  fi
}

I'm testing it with this directory structure:

testing-range/plugins/
|-- bar
|   `-- hooks
|       |-- irrelevant
|       |-- only-one
|       `-- relevant
|-- dickory
|   `-- hooks
|       `-- pre-relevant
|-- doc
|   `-- hooks
|       |-- perl-envsubst
|       `-- relevant
|           |-- 00
|           `-- 20
|
|-- factory
|   `-- hooks
|       `-- relevant
`-- hickory
    `-- hooks
        |-- pre-relevant
        |   `-- 30
        `-- relevant
            `-- 10

When I use this multi-step array code to append the files:

append_numbered_hooks () {
  # Gather every file into an array, prefixing each item with the
  # filename of the script (its position in sort order)
  local filelist=()
  for hook in "$PLUSHU_ROOT"/plugins/*/hooks/"$hookname"/*; do
    filelist+=("${hook##*/}/$hook")
  done

  # Sort the array numerically
  printf '%s\0' "${filelist[@]}" | sort -nz | IFS= read -rd '' -a filelist

  # Append each script in sorted order
  for hookline in "${filelist[@]}"; do
    append_hook "${hookline#*/}"
  done
}

I get almost the correct output:

hickory/hooks/pre-relevant/30
dickory/hooks/pre-relevant
bar/hooks/relevant
factory/hooks/relevant
doc/hooks/relevant/00
doc/hooks/relevant/20
hickory/hooks/relevant/10

Yet, when I append the files using this simpler pipeline:

append_numbered_hooks () {
  for hook in "$PLUSHU_ROOT"/plugins/*/hooks/"$1"/*; do
    printf '%s\0' "${hook##*/}/$hook"
  done | sort -nz | while IFS= read -rd '' hookline; do
    append_hook "${hookline#*/}"
  done
}

The numbered files don't show up in a for "${hooks[@]}" loop:

dickory/hooks/pre-relevant
bar/hooks/relevant
factory/hooks/relevant

What's going on? The correct values should be:

hickory/hooks/pre-relevant/30
dickory/hooks/pre-relevant
bar/hooks/relevant
factory/hooks/relevant
doc/hooks/relevant/00
hickory/hooks/relevant/10
doc/hooks/relevant/20
Improve this question
1
  • Process substitution is almost always a workaround for a badly ordered process. set -- "$PLUSHU_ROOT"/*/hook[s]/*relevant/[0-9]*[0-9]; while [ -n "$1" ]; do [ -z "${1##*/hook\[*}" ] || env - "$1" ; shift; done should be pretty close to what you need, though maybe you have to sort it first. But that's easily done, because all of the returns are in your shell "$@" array.
    – mikeserv
    Aug 13, 2014 at 23:24

1 Answer 1

Reset to default
6

From http://www.gnu.org/software/bash/manual/bashref.html#Pipelines

Each command in a pipeline is executed in its own subshell

So, you are altering a variable in a subshell. When the subshell exits, those alterations vanish.

You may find process substitutions helpful.

append_numbered_hooks () {
  while IFS= read -rd '' hookline; do
    append_hook "${hookline#*/}"
  done < <(
    for hook in "$PLUSHU_ROOT"/plugins/*/hooks/"$1"/*; do
      printf '%s\0' "${hook##*/}/$hook"
    done | 
    sort -nz
  )
}
Improve this answer
5
  • If that's the case, then why does the first approach (which also reads in a pipeline) work just fine?
    – Stuart P. Bentley
    Aug 13, 2014 at 13:05
  • 1
    @StuartP.Bentley, your printf '%s\0' "${filelist[@]}" | sort -nz | IFS= read -rd '' -a filelist does nothing, because the read updates the $filelist of a subshell. Possibly your filelist was alread sorted (globs do sort)
    – Stéphane Chazelas
    Aug 13, 2014 at 13:11
  • @StéphaneChazelas hah, good point- it actually wasn't sorted (the correct sort value I listed was PICNIC noise)!
    – Stuart P. Bentley
    Aug 13, 2014 at 13:15
  • @StuartP.Bentley, note that you can also use zsh or ksh93 instead of bash which don't have the problem (and zsh has many interesting features you could make use of like builtin ways of numerically sorting lists (filelist=(${(on)filelist})).
    – Stéphane Chazelas
    Aug 13, 2014 at 13:17
  • @StéphaneChazelas You can also use shopt -s lastpipe in Bash: mywiki.wooledge.org/BashFAQ/024
    – Stuart P. Bentley
    Aug 13, 2014 at 13:34

Not the answer you're looking for? Browse other questions tagged or ask your own question.