69
$ ls -l /tmp/test/my\ dir/
total 0

I was wondering why the following ways to run the above command fail or succeed?

$ abc='ls -l "/tmp/test/my dir"'

$ $abc
ls: cannot access '"/tmp/test/my': No such file or directory
ls: cannot access 'dir"': No such file or directory

$ "$abc"
bash: ls -l "/tmp/test/my dir": No such file or directory

$ bash -c $abc
'my dir'

$ bash -c "$abc"
total 0

$ eval $abc
total 0

$ eval "$abc"
total 0
99

This has been discussed in a number of questions on unix.SE, I'll try to collect all issues I can come up with here. References at the end.


Why it fails

The reason you face those problems is word splitting and the fact that quotes expanded from variables don't act as quotes, but are just ordinary characters.

The cases presented in the question:

The assignment here assigns the single string ls -l "/tmp/test/my dir" to abc:

$ abc='ls -l "/tmp/test/my dir"'

Below, $abc is split on whitespace, and ls gets the two arguments "/tmp/test/my and dir" (with a quote at the front of the first and another at the back of the second):

$ $abc
ls: cannot access '"/tmp/test/my': No such file or directory
ls: cannot access 'dir"': No such file or directory

Here, the expansion is quoted, so it's kept as a single word. The shell tries to find a program literally called ls -l "/tmp/test/my dir", spaces and quotes included.

$ "$abc"
bash: ls -l "/tmp/test/my dir": No such file or directory

And here, $abc is split, and only the first resulting word is taken as the argument to -c, so Bash just runs ls in the current directory. The other words are arguments to bash, and are used to fill $0, $1, etc.

$ bash -c $abc
'my dir'

With bash -c "$abc", and eval "$abc", there's an additional shell processing step, which does make the quotes work, but also causes all shell expansions to be processed again, so there's a risk of accidentally running e.g. a command substitution from user-provided data, unless you're very careful about quoting.


Better ways to do it

The two better ways to store a command are a) use a function instead, b) use an array variable (or the positional parameters).

Using a function:

Simply declare a function with the command inside, and run the function as if it were a command. Expansions in commands within the function are only processed when the command runs, not when it's defined, and you don't need to quote the individual commands.

# define it
myls() {
    ls -l "/tmp/test/my dir"
}

# run it
myls

Using an array:

Arrays allow creating multi-word variables where the individual words contain white space. Here, the individual words are stored as distinct array elements, and the "${array[@]}" expansion expands each element as separate shell words:

# define the array
mycmd=(ls -l "/tmp/test/my dir")

# run the command
"${mycmd[@]}"

The syntax is slightly horrible, but arrays also allow you to build the command line piece-by-piece. For example:

mycmd=(ls)               # initial command
if [ "$want_detail" = 1 ]; then
    mycmd+=(-l)          # optional flag
fi
mycmd+=("$targetdir")    # the filename

"${mycmd[@]}"

or keep parts of the command line constant and use the array fill just a part of it, like options or filenames:

options=(-x -v)
files=(file1 "file name with whitespace")
target=/somedir

transmutate "${options[@]}" "${files[@]}" "$target"

The downside of arrays is that they're not a standard feature, so plain POSIX shells (like dash, the default /bin/sh in Debian/Ubuntu) don't support them (but see below). Bash, ksh and zsh do, however, so it's likely your system has some shell that supports arrays.

Using "$@"

In shells with no support for named arrays, one can still use the positional parameters (the pseudo-array "$@") to hold the arguments of a command.

The following should be portable script bits that do the equivalent of the code bits in the previous section. The array is replaced with "$@", the list of positional parameters. Setting "$@" is done with set, and the double quotes around "$@" are important (these cause the elements of the list to be individually quoted).

First, simply storing a command with arguments in "$@" and running it:

set -- ls -l "/tmp/test/my dir"
"$@"

Conditionally setting parts of the command line options for a command:

set -- ls
if [ "$want_detail" = 1 ]; then
    set -- "$@" -l
fi
set -- "$@" "$targetdir"

"$@"

Only using "$@" for options and operands:

set -- -x -v
set -- "$@" file1 "file name with whitespace"
set -- "$@" /somedir

transmutate "$@"

(Of course, "$@" is usually filled with the arguments to the script itself, so you'll have to save them somewhere before re-purposing "$@".)


Be careful with eval!

As eval introduces an additional level of quote and expansion processing, you need to be careful with user input. For example, this works as long as the user doesn't type in any single quotes:

read -r filename
cmd="ls -l '$filename'"
eval "$cmd";

But if they give the input '$(uname)'.txt, your script happily runs the command substitution.

A version with arrays is immune to that since the words are kept separate for the whole time, there's no quote or other processing for the contents of filename.

read -r filename
cmd=(ls -ld -- "$filename")
"${cmd[@]}"

References

| improve this answer | |
  • 2
    you can get around the eval quoting thing by doing cmd="ls -l $(printf "%q" "$filename")". not pretty, but if the user is dead set on using an eval, it helps. It's also very useful for sending the command though similar things, such as ssh foohost "ls -l $(printf "%q" "$filename")", or in the sprit of this question: ssh foohost "$cmd". – Patrick May 20 '18 at 19:39
  • Not directly related, but have you hard-coded the directory? In that case, you might want to look at alias. Something like: $ alias abc='ls -l "/tmp/test/my dir"' – Hopping Bunny May 23 '18 at 2:25
8

The safest way to run a (non-trivial) command is eval. Then you can write the command as you would do on the command line and it is executed exactly as if you had just entered it. But you have to quote everything.

Simple case:

abc='ls -l "/tmp/test/my dir"'
eval "$abc"

not so simple case:

# command: awk '! a[$0]++ { print "foo: " $0; }' inputfile
abc='awk '\''! a[$0]++ { print "foo: " $0; }'\'' inputfile'
eval "$abc"
| improve this answer | |
  • Hi @HaukeLaging, what is \ ' ' ? – jumping_monkey Apr 1 at 6:49
  • @jumping_monkey You cannot have a ' within a '-quoted string. Thus you have to (1) finish/interrupt the quoting, (2) escape the next ' in order to get it literally, (3) type the literal quote and (4) in case of an interruption resume the quoted string: (1)'(2)\(3)'(4)' – Hauke Laging Apr 15 at 16:45
  • Hi @HaukeLaging, interesting, and why not just abc='awk \'! a[$0]++ { print "foo: " $0; }\' inputfile' ? – jumping_monkey Apr 17 at 6:43
  • @jumping_monkey Let alone that I just explained to you why that does not work: Wouldn't it make sense to test code before posting it? – Hauke Laging Apr 17 at 16:31
3

The second quote sign break the command.

When I run:

abc="ls -l '/home/wattana/Desktop'"
$abc

It gave me an error.

But when I run

abc="ls -l /home/wattana/Desktop"
$abc

There is no error at all

There is no way to fix this at the time(for me) but you can avoid the error by not having space in directory name.

This answer said the eval command can be used to fix this but it doesn't work for me :(

| improve this answer | |
  • 1
    Yeah, that works as long as there's no need for e.g. filenames with embedded spaces (or ones containing glob characters). – ilkkachu May 20 '18 at 13:22
0

If this does not work with '', then you should use ``:

abc=`ls -l /tmp/test/my\ dir/`

Update better way:

abc=$(ls -l /tmp/test/my\ dir/)
| improve this answer | |
  • 2
    This stores the result of the command in a variable. The OP is (strangely) wanting to save the command itself in a variable. Oh, and you should really start to use $( command... ) instead of backticks. – roaima Sep 23 '19 at 15:21
  • Thank you very much for the clarifications and tips! – balon Sep 23 '19 at 15:42
0

How about a python3 one-liner ?

bash-4.4# pwd
/home
bash-4.4# CUSTOM="ls -altr"
bash-4.4# python3 -c "import subprocess; subprocess.call(\"$CUSTOM\", shell=True)"
total 8
drwxr-xr-x    2 root     root          4096 Mar  4  2019 .
drwxr-xr-x    1 root     root          4096 Nov 27 16:42 ..
| improve this answer | |
0

Another simple trick to run any (trivial/non-trivial) command stored in abc variable is:

$ history -s $abc

and push UpArrow or Ctrl-p to bring it in the command line. Unlike any other method this way you can edit it before execution if needed.

This command will append the variable content as new entry to the Bash history and you can recall it by UpArrow.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.