How to procedurally generate command in bash script using string interpolation with spaces?

Question

I'm trying to create a simple script that uses a list of months like this:

(Jan Feb)

To generate and execute this command:

python ExpenseManager.py -p Inputs/Jan\ 2019\ Debit.CSV Inputs/Jan\ 2019\ Credit.CSV -p Inputs/Feb\ 2019\ Debit.CSV Inputs/Feb\ 2019\ Credit.CSV

This is the program I've written to that effect:

#!/usr/bin/env bash
clear
months=(Jan Feb)
args=()
for month in ${months[@]}; do
  args=(${args[@]} -p "Inputs/${month}\\ 2019\\ Debit.CSV" "Inputs/${month}\\ 2019\\ Credit.CSV")
done
python ExpenseManager.py "${args[@]}"
exit 0

And this, in theory, is working. When I echo the resulting command, I get the exact command I want:

python ExpenseManager.py -p Inputs/Jan\ 2019\ Debit.CSV Inputs/Jan\ 2019\ Credit.CSV -p Inputs/Feb\ 2019\ Debit.CSV Inputs/Feb\ 2019\ Credit.CSV

Now when I copy/paste the command created by this program and execute it, it works fine. However when I have Bash execute the command, it includes the backslashes that I use to include the escape backslash in Bash:

Namespace(filepairs=[['Inputs/Jan\', '2019\', 'Debit.CSV', 'Inputs/Jan\', '2019\', 'Credit.CSV'], ['Inputs/Feb\ 2019\ Debit.CSV', 'Inputs/Feb\ 2019\ Credit.CSV']]

I've tried several solutions to get this to work:

• I've tried making the args a single string and building off of that

• I've tried surrounding the args with single quotes and using double quotes around spaces like this:

args=(${args[@]} -p 'Inputs/${month}" "2019" "Debit.CSV "Inputs/${month}" "2019" "Credit.CSV")

• I've tried separating each part of the arg that requires a space using quotes like this:

  args=(${args[@]} -p "Inputs/${month}" "2019" "Debit.CSV" "Inputs/${month}\ 2019\ Credit.CSV")

I've looked at other solutions here and elsewhere but nothing seems to do the trick. So rather than continue to get stuck on this I was hoping someone could tell me the magic trick to have Bash execute this procedurally-built, interpolated command?

Answer 1

Don't use echo to see what command is being executed. It prints the command after parsing, that is after quotes and escapes have been applied and removed); therefore, if the output of echo includes quotes and/or escapes like you'd expect to see in a raw command line (i.e. before parsing), it indicates that something is terribly wrong. Compare the output from these two echo commands:

$ month=Jan
$ var="Inputs/${month}\\ 2019\\ Debit.CSV"
$ echo $var
Inputs/Jan\ 2019\ Debit.CSV
$ echo Inputs/Jan\ 2019\ Debit.CSV
Inputs/Jan 2019 Debit.CSV

In the first, the escapes are printed, indicating that they weren't parsed, applied, and removed. In the second, they're gone, indicating that they were parsed, applied, and removed.

So, how to fix it? Two rules: 1) don't put quotes or escapes in variables (except in weird cases where the string's going to go through an extra level of parsing), and 2) instead, put double-quotes around all variable references (including ${args[@]}) (again, there are some weird exceptions). Also, you can add to an array with array+=("new" "elements").

Here's the fixed script:

#!/usr/bin/env bash
clear
months=(Jan Feb)
args=()
for month in "${months[@]}"; do
  args+=(-p "Inputs/${month} 2019 Debit.CSV" "Inputs/${month} 2019 Credit.CSV")
done
python ExpenseManager.py "${args[@]}"
exit 0