8

An associative array I have has arbitrary keys, including keys containing backquotes, brackets, etc:

$ typeset -A arr
$ key='`'
$ arr[$key]=backquote
$ echo $arr[$key]
backquote

I now need to unset arr[$key]. Looking through the web, I tried:

$ unset "arr[$key]"
unset: arr[`]: invalid parameter name
$ unset "arr[${(b)key}]"
unset: arr[`]: invalid parameter name

…with no luck. Now, I'm somewhat lucky in that this provides an error message; in the following situation, nothing seems to fail but the outcome:

$ typeset -A arr
$ key='?~>#'
$ arr[$key]=symbols
$ echo "$arr[$key]"
symbols
$ unset "arr[${(b)key}]"
$ echo "$arr[$key]"
symbols

In fact, any of the symbols in ?~># would have triggered the same behavior.

Any clarification on what is happening and how to get the expected behavior?

Note: This question is related to a few threads on the ZSH mailing list with similar title as this question (here and there).

1 Answer 1

9

Wow, this is a mess.

Since Bart Schaefer's 2016 patch, merged as patch 37914 in commit 95663e936596933d529a648ed3d6c707d1a1dffe and first released in zsh 5.4, experimentally, unset "arr[$key]" works as long as key doesn't contain any of the following characters: \`()[]. These six characters must be prefixed with a backslash, and other characters must not (for example unset 'arr[\*] arr[\;]' attempts to unset the keys \* and \;, not * and ;). This is not what the any of the quoting parameter expansion flags (${(b)key}, ${(q)key} and its variants) do.

Furthermore, there's an additional wrinkle: I can't find a way to unset the empty key. unset 'arr[]' is an error and anything else unsets a non-empty key. The only workaround I've found to unset the empty key is to reassign the array in full, using a subscript flag to filter out the unwanted keys (as suggested by Stéphane Chazelas in a 2018 zsh-workers thread).

The following function works in zsh ≥5.4 (tested in zsh 5.8), and only copies the array if it needs to remove the empty key.

# Usage: unset_keys ARRAY [KEY]...
# ARRAY must be the name of an associative array parameter.
# Equivalent to unset 'ARRAY[KEY1]' 'ARRAY[KEY2]' ...
# except that this function works correctly even with keys containing
# special characters or is empty. See
# https://unix.stackexchange.com/questions/626393/in-zsh-how-do-i-unset-an-arbitrary-associative-array-element
function unset_keys {
  emulate -LR zsh
  if [[ -${(Pt)1}- != *-association-* ]]; then
    return 120 # Fail early if $1 is not the name of an associative array
  fi
  if ((${#@[2,$#]:#?*})); then
    if [[ -n ${${(P)1}[${:-}]+y} ]]; then
      # Copy all entries with non-empty keys
      : "${(AAP)1::=${(@kv)${(P)1}[(I)?*]}}"
    fi
    set -- $@ # Remove empty keys from the to-do list
  fi
  if (($# < 2)); then
    return 0
  fi
  set -- "$1" "${@[2,$#]//\\/\\\\}"
  set -- "$1" "${@[2,$#]//\`/\\\`}"
  set -- "$1" "${@[2,$#]//\(/\\(}"
  set -- "$1" "${@[2,$#]//\)/\\)}"
  set -- "$1" "${@[2,$#]//\[/\\[}"
  set -- "$1" "${@[2,$#]//\]/\\]}"
  noglob unset $1[${^@[2,$#]}]
}

Here's a simpler function that just does one copy no matter what.

# Usage: unset_keys ARRAY [KEY]...
# ARRAY must be the name of an associative array parameter.
# Equivalent to unset 'ARRAY[KEY1]' 'ARRAY[KEY2]' ...
# except that this function works correctly even with keys containing
# special characters or is empty. See
# https://unix.stackexchange.com/questions/626393/in-zsh-how-do-i-unset-an-arbitrary-associative-array-element
function unset_keys {
  emulate -LR zsh
  setopt extended_glob
  if [[ -${(Pt)1}- != *-association-* ]]; then
    return 120 # Fail early if $1 is not the name of an associative array
  fi
  set -- "$1" "${(j:|:)${(@b)@[2,$#]}}"
  # Copy all entries except the specified ones
  : "${(AAP)1::=${(@kv)${(P)1}[(I)^($~2)]}}"
}

Before zsh 5.4, it was a different mess that I haven't explored.


Here's the test harness I used. I think it gives reasonable coverage, but I haven't spent any time polishing it.

set -e

test_keys=(
  '()safe' '(r)set' '(R)set' '(k)safe' # look like valid subscript flags
  '(a' '(a)' '(n:foo:)a' '(n:1)a' # look like invalid subscript flags
  'set' '"set"' \'set\' '\s\e\t'
  'safe' '"safe"' \'safe\' '\s\a\f\e'
  '\\' '\\\' '\\\\' '""' \'\'
  'two words' 'two  spaces' ' initial space' 'trailing space '
  $'\x80\\' $'\x80\`' $'\x80\~' # broken UTF-8
  ''
  '?~>#'
)
for ((i=0; i<255; i++)); do
  printf -v n '\\x%02x' $i
  eval "test_keys+=(\$'$n')"
done

function populate_test_array {
  for k in "${(@)test_keys}"; do
    arr[$k]=set
  done
}

function check_expected_keys {
  local -a actual_keys
  actual_keys=("${(@k)arr}")
  actual_keys=("${(@o)actual_keys}") # Sorting in one step seems to misplace the empty string at the end (zsh 5.8 on Ubuntu 20.04), so sort in two steps.
  local actual_list="${(j: :)${(@qqqq)actual_keys}}"
  local expected_list="${(j: :)${(@qqqq)expected_keys}}"
  if [[ "$actual_list" != "$expected_list" ]]; then
    <<EOF
Failure: unexpected list of keys after $1
  expected: $expected_list
  actual  : $actual_list
EOF
    ((++errors))
  fi
}

typeset -A arr
errors=0

populate_test_array
expected_keys=("${(@o)test_keys}")
test_keys=("${(@)test_keys:#safe}") # [safe] must stay until the end
for k in "${(@)test_keys}"; do
  unset_keys arr "$k"
  if (($+arr[$k])); then
    printf 'Failure: unset %s did not unset it\n' "${(qq)k}"
    ((++errors))
  else
    expected_keys=("${(@)expected_keys:#"$k"}")
  fi
  check_expected_keys "unset ${(qq)k}"
done

populate_test_array
unset_keys arr "${(@)test_keys}"
expected_keys=(safe)
check_expected_keys "unsetting all"

exit $((!!errors))
2
  • Goodness me, indeed! So the b flag was good for some cases, bad for some other. Certainly there should be a flag that is foolproof for this use case, no? Should this be reported to the ML?
    – Michaël
    Dec 28, 2020 at 23:00
  • 1
    @Michaël There should be a robust way, but I'm not sure that adding another parameter expansion flag is it. Since unset "array[$key]" has already had one change that fixed some cases and broke others, I think it would be better to make another change that works with one of the existing flags. Ideally one that also allows array expansion flags. This should be discussed on the mailing list (where I don't currently participate because I don't have any public email set up with my Stack Exchange identity). Dec 28, 2020 at 23:12

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .