I'd like to use the feature of GNU parallel where it can execute the command and the list it's fed in parallel and spit it out after it's all done, however, I don't want to install GNU parallel across all of our servers.

Or perhaps a parallel version of xargs?

Is there a ksh implementation of what GNU Parallel does? In this case, it doesn't have to be done in order as GNU Parallel does -- just as long as all the output can be piped or stored. I'd also like to avoid using temporary files.

  • GNU Parallel is a single perl script. I am curious why you prefer installing a ksh script over installing a single perl script that is designed to have very few dependencies. Can you elaborate on that? (Disclosure: I am the author of GNU Parallel)
    – Ole Tange
    Jun 21, 2011 at 10:37

2 Answers 2


If you want to parallelize on a machine with multiple cores, you can just use (GNU) xargs, e.g.:

echo seq_[0-9][0-9].gz | xargs -n 1 -P 16 ./crunching

Meaning: xargs starts up to 16 processes in parallel of ./crunching using 1 token from stdin for each process.

You can also use split in combination with xargs.

Or you can create a simple Makefile for Job execution and call make -f mymf -j $CORES (you need temporary files for this solution).

PS: The GNU parallel manual also includes some comparisons with other tools, including xargs and make, interestingly they write:

(Very early versions of GNU parallel were coincidently implemented using make -j).

  • It is important for the asker that "all the output can be piped or stored". xargs is notoriously bad at that when run in parallel because the output can mix.
    – Ole Tange
    Jun 21, 2011 at 10:40
  • @maxschlepzig: I agree with your answer 100%, I always just use xargs. Many people just don't know GNU xargs is able to do parallel! Dec 29, 2011 at 19:09

Look at parallel --embed which embeds GNU Parallel into the same shell script that you use it from.

parallel --embed > new_script

then edit new_script.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .