Iterate Through Sets of Command Arguments in Bash

Question

I have multiple "sets" of arguments for one command that I need to run in sequence (well, in parallel, technically). I also need to repeat the same logic after running each command.

#!/bin/bash

local pids=()

# Run commands in parallel.
my_command "$URL_ONE"   "$URL_ONE_TEXT"    "${TMP_DIR}/some_dir" &
pids+=("$!")

my_command "$URL_ONE"   "$URL_TWO_TEXT"    "${TMP_DIR}/some_similar_dir" &
pids+=("$!")

my_command "$URL_TWO"   "$URL_TWO_TEXT"    "${TMP_DIR}/third_dir" &
pids+=("$!")

my_command "$URL_THREE" "$URL_THREE_TEXT"  "${TMP_DIR}/fourth_dir" &
pids+=("$!")

# ...

# Wait for parallel commands to complete and exit if any fail.
for pid in "${pids[@]}"; do
    wait "$pid"
    if [[ $? -ne 0 ]]; then
        errecho "Failed."
        exit 1
    fi
done

Rather than repeating pids+=("$!") and other portions so frequently, what I'd like to do is define an array/set of arguments, loop through it, and execute the same logic for each set of arguments. For example:

#!/bin/bash

# This wouldn't actually work...
ARG_SETS=(
    ("$URL_ONE"   "$URL_ONE_TEXT"   "${TMP_DIR}/some_dir")
    ("$URL_ONE"   "$URL_TWO_TEXT"   "${TMP_DIR}/some_similar_dir")
    ("$URL_TWO"   "$URL_TWO_TEXT"   "${TMP_DIR}/third_dir")
    ("$URL_THREE" "$URL_THREE_TEXT" "${TMP_DIR}/fourth_dir")
)
for arg1 arg2 arg3 in "$ARG_SETS[@]"; do
    my_command "$arg1" "$arg2" "$arg3" &
    pids+=("$!")
done

But Bash does not support multidimensional arrays. Does anyone have any ideas for a good pattern to make this cleaner, or do something similar in design to my second example? Thanks!

Answer 1

This approach uses three arrays, one for each argument of the my_command:

pids=()

a=("$URL_ONE"               "$URL_ONE"                      "$URL_TWO"                  "$URL_THREE")
b=("$URL_ONE_TEXT"          "$URL_TWO_TEXT"                 "$URL_TWO_TEXT"         "$URL_THREE_TEXT")
c=("${TMP_DIR}/some_dir"    "${TMP_DIR}/some_similar_dir"   "${TMP_DIR}/third_dir"  "${TMP_DIR}/fourth_dir")

for i in ${!a[@]}
do
    my_command "${a[$i]}" "${b[$i]}" "${c[$i]}" &
    pids+=("$!")
done

# Wait for parallel commands to complete and exit if any fail.
for pid in "${pids[@]}"
do
    if ! wait "$pid"
    then
        errecho "Failed."
        exit 1
    fi
done

Alternate style

Depending on how many commands are to be run, you may want to consider the following alternative for defining the arrays:

a=();              b=();                   c=()
a+=("$URL_ONE");   b+=("$URL_ONE_TEXT");   c+=("${TMP_DIR}/some_dir")
a+=("$URL_ONE");   b+=("$URL_TWO_TEXT");   c+=("${TMP_DIR}/some_similar_dir")
a+=("$URL_TWO");   b+=("$URL_TWO_TEXT");   c+=("${TMP_DIR}/third_dir")
a+=("$URL_THREE"); b+=("$URL_THREE_TEXT"); c+=("${TMP_DIR}/fourth_dir")

Answer 2

I don't really see the need for arrays, and would go for something simple like:

f(){
  my_command "$@" &
  pids+=("$!")
}

f "$URL_ONE"   "$URL_ONE_TEXT"   "${TMP_DIR}/some_dir"
f "$URL_ONE"   "$URL_TWO_TEXT"   "${TMP_DIR}/some_similar_dir"
f "$URL_TWO"   "$URL_TWO_TEXT"   "${TMP_DIR}/third_dir"
f "$URL_THREE" "$URL_THREE_TEXT" "${TMP_DIR}/fourth_dir"