Linux command substitution order

Question

I'm confused with the command substitution. I think command substitution as like a programming language macro. The sub shell is executed first and the $(...) is substituted with it's standard output, before the parent command is evaluated. But is this all the truth?

When I try to execute the following command

echo {0..9} | xargs -n 2 $(echo 'echo | tac')

I would except to see the following output

8 9
6 7
4 5
2 3
0 1

But instead get this

| tac 0 1
| tac 2 3
| tac 4 5
| tac 6 7
| tac 8 9

So what causes | tac to be captured as an argument to the echo, and not as part of the command line? What is the exact mechanism and order, how the command line and command substitution is parsed, scoped and evaluated? Is it possible at all to build command lines dynamically in Linux in a metaprogramming style of way?

Edit

I know I could just use echo {0..9}| xargs -n 2 | tac. This question is theoretical, I'm interested why the sub shell example doesn't produce the same result

Answer 1

The genesis of the issue you are seeing is due to the way tokenization happens of a command line. In this particular case, much before the $(echo 'echo | tac') is expanded, the shell has already figured out that you intend to run a command (echo {…}) and pass it's output via a pipe (|) to another command (xargs -n 2 $(…)).

Then in the next stage does the filling in of $(…) and brace expansion {…} happens to generate the actual commands to be run. And in this stage if it results in a pipe character, so what, it's too late and it's clearly missed the bus.

Now it will be treated as not as a special metacharacter, but will be included in the xargs command line as just any ordinary (=> non-metachar) char.

Should you want to take another shot at it, then you need to eval it.

eval "echo {0..9} | xargs -n 2 $(echo 'echo | tac')"

This will output what you expect.

Answer 2

besides Roman said, your commnand should be like this

echo {0..9}| xargs -n 2 | tac

this produces

8 9
6 7 
4 5
2 3
0 1

with the pointless $(echo 'echo') this causes the error

PS. For the long answer, | redirects the output from one command to the next one, so echo {0..9} produces:

0 1 2 3 4 5 6 7 8 9

xargs -n 2 takes that input and produces

0 1
2 3
4 5 
6 7
8 9

and finally tac takes that input and produces the reverse expresion:

8 9
6 7
4 5
2 3
0 1

I hope this is a bit clearer.

EDIT:

echo {0..9} | xargs -n 2 $(echo 'echo | tac')
+ echo 0 1 2 3 4 5 6 7 8 9
++ echo 'echo | tac'
+ xargs -n 2 echo '|' tac
| tac 0 1
| tac 2 3
| tac 4 5
| tac 6 7
| tac 8 9

This is the command that you are sending to bash with your initial sintax, it's pretty clear what's happening, at the end you are sending xargs -n 2 echo '|' tac

Answer 3

At the risk of being redundant (but I don’t see where anybody else has said this plainly), the reason why your

echo {0..9} | xargs -n 2 $(echo 'echo | tac')

command does what it does is that it is equivalent to

echo {0..9} | xargs -n 2 'echo' '|' 'tac'

Compare to

echo {0..9} | xargs -n 2 echo foo bar

This happens because the $(echo …) is evaluated after the overall structure of the command line (echo (something) | xargs (args)) has been parsed, and so the output from $(echo …) can only add argument(s) to the xargs command.  To do what you want the way that you want to do it, you would need to say

echo {0..9} | eval xargs -n 2 $(echo 'echo | tac')

Note that eval can be dangerous.  Search this site for warnings about it, and heed them.