4

I'm trying to use GNU Parallel to run a comman mutliple times with a combination of constant and varying arguments. But for some reason the constant arguments are split on white-space even though I've quoted them when passing them to parallel.

In this example, the constant argument 'a b' should be passed to debug-call as a single argument instead of two:

$ parallel debug-call 'a b' {} ::: {1..2}
[0] = '[...]/debug-call'
[1] = 'a'
[2] = 'b'
[3] = '1'
[0] = '[...]/debug-call'
[1] = 'a'
[2] = 'b'
[3] = '2'

debug-call is a simple script which prints each argument it has been passed in argv. Instead I would expect to see this output:

[0] = '[...]/debug-call'
[1] = 'a b'
[2] = '1'
[0] = '[...]/debug-call'
[1] = 'a b'
[2] = '2'

Is this a bug or is there a option to prevent GNU Parallel from splitting command line arguments before passing them on to the command?

  • 1
    Use parallel with -q... – don_crissti May 10 '17 at 11:06
4

parallel runs a shell (which exact one depending on the context in which it is called, generally, when called from a shell, it's that same shell) to parse the concatenation of the arguments.

So:

parallel debug-call 'a b' {} ::: 'a b' c

is the same as

parallel 'debug-call a b {}' ::: 'a b' c

parallel will call:

your-shell -c 'debug-call a b <something>'

Where <something> is the arguments (hopefully) properly quoted for that shell. For instance, if that shell is bash, it will run

bash -c 'debug-call a b a\ b'

Here, you want:

parallel 'debug-call "a b" {}' ::: 'a b' c

Or

parallel -q debug-call 'a b' {} ::: 'a b' c

Where parallel will quote the arguments (in the correct (hopefully) syntax for the shell) before concatenating.

To avoid calling a shell in the first place, you could use GNU xargs instead:

xargs -n1 -r0 -P4 -a <(printf '%s\0' 'a b' c) debug-call 'a b'

That won't invoke a shell (nor any of the many commands ran by parallel upon initialisation), but you won't benefit from any of the extra features of parallel, like output reordering with -k.

You may find other approaches at Background execution in parallel

  • 1
    Thank you for you answer. -q seems to work. Is it possible to prevent my shell from getting called at all? I.e. having parallel pass the arguments directly to execvp(3) or similar? – Feuermurmel May 10 '17 at 13:26
  • 1
    @Feuermurmel, see discussion at lists.gnu.org/archive/html/bug-parallel/2015-05/msg00005.html – Stéphane Chazelas May 10 '17 at 13:29
  • It seems that GNU Parallel is not the right tool for what I want to do. I will have another look at xargs. I remember that being even worse, but maybe I am mistaken. – Feuermurmel May 10 '17 at 13:54
  • @Feuermurmel, see edit for GNU xargs example – Stéphane Chazelas May 10 '17 at 14:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.