4

I'm trying to use GNU Parallel to run a comman mutliple times with a combination of constant and varying arguments. But for some reason the constant arguments are split on white-space even though I've quoted them when passing them to parallel.

In this example, the constant argument 'a b' should be passed to debug-call as a single argument instead of two:

$ parallel debug-call 'a b' {} ::: {1..2}
[0] = '[...]/debug-call'
[1] = 'a'
[2] = 'b'
[3] = '1'
[0] = '[...]/debug-call'
[1] = 'a'
[2] = 'b'
[3] = '2'

debug-call is a simple script which prints each argument it has been passed in argv. Instead I would expect to see this output:

[0] = '[...]/debug-call'
[1] = 'a b'
[2] = '1'
[0] = '[...]/debug-call'
[1] = 'a b'
[2] = '2'

Is this a bug or is there a option to prevent GNU Parallel from splitting command line arguments before passing them on to the command?

1
  • 1
    Use parallel with -q... May 10, 2017 at 11:06

1 Answer 1

5

parallel runs a shell (which exact one depending on the context in which it is called, generally, when called from a shell, it's that same shell) to parse the concatenation of the arguments.

So:

parallel debug-call 'a b' {} ::: 'a b' c

is the same as

parallel 'debug-call a b {}' ::: 'a b' c

parallel will call:

your-shell -c 'debug-call a b <something>'

Where <something> is the arguments (hopefully) properly quoted for that shell. For instance, if that shell is bash, it will run

bash -c 'debug-call a b a\ b'

Here, you want:

parallel 'debug-call "a b" {}' ::: 'a b' c

Or

parallel -q debug-call 'a b' {} ::: 'a b' c

Where parallel will quote the arguments (in the correct (hopefully) syntax for the shell) before concatenating.

To avoid calling a shell in the first place, you could use GNU xargs instead:

xargs -n1 -r0 -P4 -a <(printf '%s\0' 'a b' c) debug-call 'a b'

That won't invoke a shell (nor any of the many commands ran by parallel upon initialisation), but you won't benefit from any of the extra features of parallel, like output reordering with -k.

You may find other approaches at Background execution in parallel

4
  • 1
    Thank you for you answer. -q seems to work. Is it possible to prevent my shell from getting called at all? I.e. having parallel pass the arguments directly to execvp(3) or similar? May 10, 2017 at 13:26
  • 1
    @Feuermurmel, see discussion at lists.gnu.org/archive/html/bug-parallel/2015-05/msg00005.html May 10, 2017 at 13:29
  • It seems that GNU Parallel is not the right tool for what I want to do. I will have another look at xargs. I remember that being even worse, but maybe I am mistaken. May 10, 2017 at 13:54
  • @Feuermurmel, see edit for GNU xargs example May 10, 2017 at 14:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.