After some reading I try to answer by myself
Why does command substitution behave in such way?
$ a='echo x; echo y'
$ echo $($a) # expect 'x y'
Lets notice that the substitution of variable
$a is performed in the subshell produced during command substitution, not in the current shell (1,2,3). Hence the subshell executes the command
echo x; echo y (value of variable
a is inherited)
So now we only need to find out why does shell behave like this:
$ a='echo x; echo y'
x; echo y
According to Bash Reference Manual:
There are seven kinds of expansion … The order of expansions is: brace expansion; tilde expansion, parameter and variable expansion, arithmetic expansion, and command substitution (done in a left-to-right fashion); word splitting; and filename expansion
— from the section 3.5 Shell Expansions
The shell scans the results of parameter expansion, command substitution, and arithmetic expansion that did not occur within double quotes for word splitting.
The shell treats each character of $IFS as a delimiter, and splits the results of the other expansions into words using these characters as field terminators
— from the section 3.5.7 Word Splitting
(The default value of
The IFS variable is used to split only the results of expansion, not all words (see Word Splitting)
— from the Appendix B Major Differences From The Bourne Shell
A character that, when unquoted, separates words. A metacharacter is a space, tab, newline, or one of the following characters:
— from the section 2. Definitions
The brief description of the shell’s operation when it reads and executes a command. Basically, the shell does the following:
- Reads its input.
- Breaks the input into words and operators, obeying the quoting rules. These tokens are separated by metacharacters.
- Parses the tokens into simple and compound commands.
- Performs the various shell expansions.
- Performs any necessary redirections.
- Executes the command
— from the section 3.1.1 Shell Operation
Now we see that there actually are two different kinds of "word splitting" — the initial word splitting (step 2) and word splitting which is kind of shell expansions (step 4).
The initial word splitting (step 2) treats metacharacters like
) as delimiters; its outcome is parsed (step 3), so metacharacters and keywords are recognized.
Subsequently bash performs the various shell expansions (step 4) and, among others, "word splitting" — a kind of them. This type of word splitting treats only characters of
IFS as delimiters. It does not care about metacharacters. Its outcome is not parsed again, as well as outcome of any other expansion. Thus metacharecters and keywords are not recognized.
; within value of variable
a is not treated as command separator.
'echo' 'x;' 'echo' 'y'. Even if value of
'echo x ; echo y' the word
';' won't treated as metacharacter, so command
$a will become
'echo' 'x' ';' 'echo' 'y'.
Metacharacters and keywords (
while etc.) cannot be a result of expansions, but program names, builtin commands, functions and aliases do can. It may introduce some confusion, because it allows to store simple commands in variables, while not allowing to do same thing with compound commands.
$ 'echo' 'a' ';' 'echo' 'b' # ';' is a literal
a ; echo b
$ "$cmd" a ; $(printf echo) b # ';' is a metacharacter
$ cmd='echo a' # simple command is executed properly
$ cmd='echo a;' # but metacharacters are not recognized
$ $cmd echo b
a; echo b
$ echo a $(echo ';') echo b # ';' is a literal
a ; echo b
$ $(printf if) true; then echo a; fi # parsing error
bash: syntax error near unexpected token 'then'
$ $(printf if) true; $(printf then) echo a; $(printf fi) # looking for command "if"
bash: if: command not found
bash: then: command not found
bash: fi: command not found
How to perform the
command substitution for list of commands stored in a variable without using eval and bash -c (since eval is evil)?
Probably, the right answer is "this is evil too" so you don't need to avoid eval :)
Variables hold data. Functions hold code. Don't put code inside variables!..
— from the article I'm trying to put a command in a variable, but the complex cases always fail!
Bash shell command substitution
Why is a variable visible in a subshell?
Do parentheses really put the command in a subshell?
Bash Reference Manual
I'm trying to put a command in a variable, but the complex cases always fail!
conversation in debian bug tracking system