In ZSH, how do I unset an arbitrary associative array element?

Question

An associative array I have has arbitrary keys, including keys containing backquotes, brackets, etc:

$ typeset -A arr
$ key='`'
$ arr[$key]=backquote
$ echo $arr[$key]
backquote

I now need to unset arr[$key]. Looking through the web, I tried:

$ unset "arr[$key]"
unset: arr[`]: invalid parameter name
$ unset "arr[${(b)key}]"
unset: arr[`]: invalid parameter name

…with no luck. Now, I'm somewhat lucky in that this provides an error message; in the following situation, nothing seems to fail but the outcome:

$ typeset -A arr
$ key='?~>#'
$ arr[$key]=symbols
$ echo "$arr[$key]"
symbols
$ unset "arr[${(b)key}]"
$ echo "$arr[$key]"
symbols

In fact, any of the symbols in ?~># would have triggered the same behavior.

Any clarification on what is happening and how to get the expected behavior?

Note: This question is related to a few threads on the ZSH mailing list with similar title as this question (here and there).

Answer 1

Wow, this is a mess.

Since Bart Schaefer's 2016 patch, merged as patch 37914 in commit 95663e936596933d529a648ed3d6c707d1a1dffe and first released in zsh 5.4, experimentally, unset "arr[$key]" works as long as key doesn't contain any of the following characters: \`()[]. These six characters must be prefixed with a backslash, and other characters must not (for example unset 'arr[\*] arr[\;]' attempts to unset the keys \* and \;, not * and ;). This is not what the any of the quoting parameter expansion flags (${(b)key}, ${(q)key} and its variants) do.

Furthermore, there's an additional wrinkle: I can't find a way to unset the empty key. unset 'arr[]' is an error and anything else unsets a non-empty key. The only workaround I've found to unset the empty key is to reassign the array in full, using a subscript flag to filter out the unwanted keys (as suggested by Stéphane Chazelas in a 2018 zsh-workers thread).

The following function works in zsh ≥5.4 (tested in zsh 5.8), and only copies the array if it needs to remove the empty key.

# Usage: unset_keys ARRAY [KEY]...
# ARRAY must be the name of an associative array parameter.
# Equivalent to unset 'ARRAY[KEY1]' 'ARRAY[KEY2]' ...
# except that this function works correctly even with keys containing
# special characters or is empty. See
# https://unix.stackexchange.com/questions/626393/in-zsh-how-do-i-unset-an-arbitrary-associative-array-element
function unset_keys {
  emulate -LR zsh
  if [[ -${(Pt)1}- != *-association-* ]]; then
    return 120 # Fail early if $1 is not the name of an associative array
  fi
  if ((${#@[2,$#]:#?*})); then
    if [[ -n ${${(P)1}[${:-}]+y} ]]; then
      # Copy all entries with non-empty keys
      : "${(AAP)1::=${(@kv)${(P)1}[(I)?*]}}"
    fi
    set -- $@ # Remove empty keys from the to-do list
  fi
  if (($# < 2)); then
    return 0
  fi
  set -- "$1" "${@[2,$#]//\\/\\\\}"
  set -- "$1" "${@[2,$#]//\`/\\\`}"
  set -- "$1" "${@[2,$#]//\(/\\(}"
  set -- "$1" "${@[2,$#]//\)/\\)}"
  set -- "$1" "${@[2,$#]//\[/\\[}"
  set -- "$1" "${@[2,$#]//\]/\\]}"
  noglob unset $1[${^@[2,$#]}]
}

Here's a simpler function that just does one copy no matter what.

# Usage: unset_keys ARRAY [KEY]...
# ARRAY must be the name of an associative array parameter.
# Equivalent to unset 'ARRAY[KEY1]' 'ARRAY[KEY2]' ...
# except that this function works correctly even with keys containing
# special characters or is empty. See
# https://unix.stackexchange.com/questions/626393/in-zsh-how-do-i-unset-an-arbitrary-associative-array-element
function unset_keys {
  emulate -LR zsh
  setopt extended_glob
  if [[ -${(Pt)1}- != *-association-* ]]; then
    return 120 # Fail early if $1 is not the name of an associative array
  fi
  set -- "$1" "${(j:|:)${(@b)@[2,$#]}}"
  # Copy all entries except the specified ones
  : "${(AAP)1::=${(@kv)${(P)1}[(I)^($~2)]}}"
}

Before zsh 5.4, it was a different mess that I haven't explored.

---

Here's the test harness I used. I think it gives reasonable coverage, but I haven't spent any time polishing it.

set -e

test_keys=(
  '()safe' '(r)set' '(R)set' '(k)safe' # look like valid subscript flags
  '(a' '(a)' '(n:foo:)a' '(n:1)a' # look like invalid subscript flags
  'set' '"set"' \'set\' '\s\e\t'
  'safe' '"safe"' \'safe\' '\s\a\f\e'
  '\\' '\\\' '\\\\' '""' \'\'
  'two words' 'two  spaces' ' initial space' 'trailing space '
  $'\x80\\' $'\x80\`' $'\x80\~' # broken UTF-8
  ''
  '?~>#'
)
for ((i=0; i<255; i++)); do
  printf -v n '\\x%02x' $i
  eval "test_keys+=(\$'$n')"
done

function populate_test_array {
  for k in "${(@)test_keys}"; do
    arr[$k]=set
  done
}

function check_expected_keys {
  local -a actual_keys
  actual_keys=("${(@k)arr}")
  actual_keys=("${(@o)actual_keys}") # Sorting in one step seems to misplace the empty string at the end (zsh 5.8 on Ubuntu 20.04), so sort in two steps.
  local actual_list="${(j: :)${(@qqqq)actual_keys}}"
  local expected_list="${(j: :)${(@qqqq)expected_keys}}"
  if [[ "$actual_list" != "$expected_list" ]]; then
    <<EOF
Failure: unexpected list of keys after $1
  expected: $expected_list
  actual  : $actual_list
EOF
    ((++errors))
  fi
}

typeset -A arr
errors=0

populate_test_array
expected_keys=("${(@o)test_keys}")
test_keys=("${(@)test_keys:#safe}") # [safe] must stay until the end
for k in "${(@)test_keys}"; do
  unset_keys arr "$k"
  if (($+arr[$k])); then
    printf 'Failure: unset %s did not unset it\n' "${(qq)k}"
    ((++errors))
  else
    expected_keys=("${(@)expected_keys:#"$k"}")
  fi
  check_expected_keys "unset ${(qq)k}"
done

populate_test_array
unset_keys arr "${(@)test_keys}"
expected_keys=(safe)
check_expected_keys "unsetting all"

exit $((!!errors))