Why doesn't GNU parallel work with "bash -c"?

Question

% echo -e '1\n2' | parallel "bash -c 'echo :\$1' '' {}"
:1
:2
% echo -e '1\n2' | parallel bash -c 'echo :\$1' '' {}


%

I'd expect the second line to act the same.

Answer 1

parallel runs the command in a shell already (which shell it is is determined by parallel using heuristics (the intention being to invoke the same shell as the one parallel was invoked from). You can set the $PARALLEL_SHELL variable to fix the shell).

It's not a command you're passing to parallel like you would for the env or xargs command, but a shell command line (like you would for the eval command).

Like for eval, in parallel arg1 arg2, parallel is concatenating those arguments with spaces in between (so it becomes arg1 arg2) and that string is passed to <the-shell> -c.

For the arguments that are passed on parallel's stdin, parallel quotes them in the format expected by that particular shell (a difficult and error prone task which is why you'll find there have been a lot of bugs fixed around that in parallel's Changelog (some are still not fixed as of 2017-03-06)) and appends it to that command line.

So for instance, if called from within bash,

echo "foo'bar" | parallel echo foo

Would have parallel call bash -c with echo foo foo\'bar as the command line. And if called from within rc (or with PARALLEL_SHELL=rc) rc -c with echo foo foo''''bar.

In your:

parallel bash -c 'echo :\$1' '' {}

parallel concatenates those arguments which gives:

bash -c echo :$1  {}

And with the {} expanded and quoted in the right format for the shell you're calling parallel from, passes that to <that-shell> -c which will call bash -c echo with :$1 in $0 and the current argument in $1.

It's not how parallel works. Here, you'd probably want:

printf '1\n2\n' | PARALLEL_SHELL=bash parallel 'echo :{}'

To see what parallel does, you can run it under strace -fe execve (or the equivalent on your system if not Linux).

Here, you could use GNU xargs instead of parallel to get a simpler processing closer to what you're expecting:

printf '1\n2\n' | xargs -rn1 -P4 bash -c 'echo ":$1"' ''

See also the discussion at https://lists.gnu.org/archive/html/bug-parallel/2015-05/msg00005.html

Note that in bash -c 'echo foo' '' foo, you're making $0 the empty string for that inline-script. I would avoid that as that $0 is also used in error messages. Compare:

$ bash -c 'echo x > "$1"' '' /
: /: Is a directory

with.

$ bash -c 'echo x > "$1"' bash /
bash: /: Is a directory

Also note that leaving variables unquoted has a very special meaning in bash and that echo can generally not be used for arbitrary data.