Passing paths with spaces as arguments

Question

I am having difficulty in passing some string variables having spaces in them as arguments to a program.
For debugging and showing the arguments being passed, I created a demo Python script -:

##### show_args.py #####

import sys

def main():
    # Display the arguments passed to the script
    print "Number of arguments =", len(sys.argv)
    for arg in sys.argv:
        print arg

if __name__ == '__main__':
    main()

Now, the script demonstrating the problem -:

path_with_spaces="$HOME/blah blah"
arg_list="$path_with_spaces/abc $path_with_spaces/xyz"

python show_args.py $arg_list

Output -:

Number of arguments = 5
show_args.py
/home/anmol/blah
blah/abc
/home/anmol/blah
blah/xyz

What I actually want is this -:

path_with_spaces="$HOME/blah blah"

python show_args.py "$path_with_spaces/abc" "$path_with_spaces/xyz"

Output -:

Number of arguments = 3
show_args.py
/home/anmol/blah blah/abc
/home/anmol/blah blah/xyz

To confirm that the problem was occurring only for paths with spaces in them, I created the following script -:

path_without_spaces="$HOME/blah"
arg_list="$path_without_spaces/abc $path_without_spaces/xyz"

python show_args.py $arg_list

Output -:

Number of arguments = 3
show_args.py
/home/anmol/blah/abc
/home/anmol/blah/xyz

While searching for the solutions to this problem, I encountered this answer, according to which the correct way is to put the arguments in an array variable rather than a string variable.
The script showing this new approach -:

path_with_spaces="$HOME/blah blah"
arg_list=("$path_with_spaces/abc" "$path_with_spaces/xyz")

python show_args.py "${arg_list[@]}"

Output -:

Number of arguments = 3
show_args.py
/home/anmol/blah blah/abc
/home/anmol/blah blah/xyz

Although, this solution is working correctly, I still want to know if there is a way with which we can accomplish the same thing using a string variable rather an array variable.

My system configuration -:

• Ubuntu 14.04 LTS
• Bash 4.3.11
• Gnome Terminal 3.6.2

Answer 1

In:

path_with_spaces="$HOME/blah blah"
arg_list="$path_with_spaces/abc $path_with_spaces/xyz"

python show_args.py $arg_list

You're using a scalar/string variable and using the split+glob operator (leaving that variable unquoted) to split the content of that variable and generate the arguments to pass to python.

The split part of the split+glob operator splits on the characters stored in the $IFS special parameter.

Here, you could disable the glob part and split on a character not found in your paths. For instance, if newline is not found in the paths:

path_with_spaces="$HOME/blah blah"
arg_list="$path_with_spaces/abc
$path_with_spaces/xyz"
IFS="
"
set -f # disable glob.
python show_args.py $arg_list

You could also use shell quoting and use eval to have the shell interpret those quotes (here using bash-specific features):

printf -v arg_list '%q ' "$path_with_spaces/abc" \
                         "$path_with_spaces/xyz"

eval "python show_args.py $arg_list"

Answer 2

Yes.

You can use the default path separator to separate paths. There are two kinds of characters which are illegal in a path-name:

• the \0NUL character

• the / path separator

As it happens, you can separate path-names on the separator with the shell's Internal Field Separator $IFS special parameter. When two or more separators occur in sequence, the resulting component will be a \0NUL valued argument - which cannot be a path-name.

To work you must root all paths directly (and so rule out a root //), and you must ensure you have canonical paths - or at the least that all //* have been squeezed to a single occurrence per (dots won't hurt - or help). Once you have done so:

---

paths='/path/one//path/t w o//p a t h/t h r e e/'
set -f --; IFS=/
for p in ${paths#/}$IFS
do  printf ${p:+/}%s\\n "$*"
    set -- ${p:+"$@"/$p}
done

---

/path/one
/path/t w o
/p a t h/t h r e e

---

You can get a canonical path in $PWD w/ cd.

---

cd -- /some//../screwy/../path/to///destination
printf %s\\n "$PWD"

---

/path/to/destination

---

If you use the -P switch cd will ensure an absolute, physical path to the current working directory is set to $PWD. Else with -L or by default cd will maintain any indirections implied by previous changes to symlinked directories as your shell session progresses.

And so you can most reliably collect the names in a tree by walking it.