Calling script from bash has problem with arguments

Question

I have a script that generates an xml file and then executes a perl script, passing a series of arguments to it. Here's the key code:

CMD="./dnscurl.pl --keyname $KEYNAME  -- -X POST -H \"Content-Type: text/xml; charset=UTF-8\" --upload-file /tmp/file.xml https://route53.amazonaws.com/2010-10-01/hostedzone/$HZID/rrset"
echo "Executing..."
echo $CMD
RESULT=`$CMD`

When this executes, the result indicates that something is happening to the embedded arguments after the -H param sent to the dnscurl.pl script:

curl: (6) Couldn't resolve host 'text'

However, I can take the exact command I echo out, and run it from the shell prompt, and the script operates correctly. It seems that when I run it with backtic using RESULT=$CMD the parameters are not getting passed with the double quotes around the "Content-Type: etc...".

The command I'm running is this one (only the keyname and hostid params were changed)

./dnscurl.pl --keyname mykey -- -X POST -H "Content-Type: text/xml; charset=UTF-8" --upload-file /tmp/file.xml https://route53.amazonaws.com/2010-10-01/hostedzone/ZZZZZZZ/rrset

One thing I'm curious about is the use of the '--' they are passing. I realize that is specific to the dnscurl.pl script handling, but I'm not clear on why that is being passed. Is that due to get around positional argument handling? The script was written to handle 3 different use cases. I don't think it's relevant, but just for completeness, this script is being used to make calls to AWS's route 53 rest api.

Answer 1

Your quoting is wrong. When you write $CMD with no quotes, the value of $CMD is broken up into “words” at each whitespace sequence¹ (the words can contain any non-whitespace character including punctuation), and then each word undergoes globbing (i.e. wildcard expansion). Note that quotes in the value of CMD, in particular, are untouched: quotes have a meaning in the syntax of shell scripts but not in variable substitution. After this, the first word becomes the name of the command to execute, and subsequent words are the command's arguments.

In your example, assuming $KEYNAME is somekeyname and $HZID is somehzid, then the command and arguments are:

./dnscurl.pl
--keyname
$KEYNAME
--
-X
POST
-H
"Content-Type:
text/xml;
charset=UTF-8"
--upload-file
/tmp/file.xml
https://route53.amazonaws.com/2010-10-01/hostedzone/somehzid/rrset

Note that text/xml; appears as the first non-option argument; clearly the Perl script passes that argument down to curl. The -- in the argument list is unrelated to your problem.

There's no way to stuff a command line into a variable. A (simple) variable is the wrong tool for that: it contains a string, but a command line is a list of strings (the command, and its arguments). You can stuff a command name and its argument into an array variable:

CMD=(./dnscurl.pl --keyname "$KEYNAME"  --
     -X POST -H "Content-Type: text/xml; charset=UTF-8"
     --upload-file /tmp/file.xml
     "https://route53.amazonaws.com/2010-10-01/hostedzone/$HZID/rrset")
RESULT=$("${CMD[@]}")

The strange-looking syntax "${CMD[@]}" expands the array variable CMD to the list of words in the array. The double quotes prevent expansion of words inside the array, and [@] is needed for historical reasons to tell the shell that you want to expand an array.

Another way to remember a command line, or an arbitrary shell snippet, for later use, is a function.

cmd () {
    ./dnscurl.pl --keyname "$KEYNAME"  -- \
                 -X POST -H "Content-Type: text/xml; charset=UTF-8" \
                 --upload-file /tmp/file.xml \
                 "https://route53.amazonaws.com/2010-10-01/hostedzone/$HZID/rrset"
}
RESULT=$(cmd)

¹ _{More precisely, according to the value of IFS.}