7

I am having an issue trying to use parallel command on Ubuntu 10.04. I looked up the parallel documentation and few of the commands seem to run. In all cases I just get the command prompt back without any action being taken. e.g. I was trying to compress a bunch of files using bzip2


17:32 farhat HarshaNaveen$ parallel bzip2 ::: *fastq 
17:33 farhat HarshaNaveen$ ls *fastq|parallel bzip2 {}

Neither of these commands worked. Nor was there any error. The example given in the man file worked fine though. 


18:58farhat HarshaNaveen$ parallel sh -c "echo hi; sleep 2; echo bye" -- 1 2 3
hi
hi
hi
bye
bye
bye
18:58farhat HarshaNaveen$

What am I doing wrong?

Improve this question
1
  • 1
    You are probably using moreutils. As you are dealing with fastq-files you definitely want to install GNU Parallel because of the --pipe option. Watch youtube.com/watch?v=1ntxT-47VPA to see an example with fasta-files.
    – Ole Tange
    Commented Jun 21, 2011 at 15:12

3 Answers 3

Reset to default
3

Your first try is closest to being correct, but why the :::? If you change ::: to --, it will do what you want.

parallel has a specific, unusual structure to its command line. In the first half, you give it the command you want to run multiple times, and the part of the command line that will be the same every time. In the second half, you give it the parts that will be different each time the command is run. These halves are separated by --. Some experimentation shows that if parallel doesn't find the second half, it doesn't actually run any commands.

It's probably worth re-reading the man page carefully. Man pages have a terse, information-dense style that can take some getting used to. Also try reading some pages for commands you're already familiar with.

Improve this answer
5
  • Thanks, it is a command that I expect to use quite a bit so it is good to know the intricacies.
    – Farhat
    Commented Feb 20, 2011 at 5:08
  • The man page actually specifies ::: for the first example. The only difference I had is using bzip2 instead of gzip.
    – Farhat
    Commented Feb 20, 2011 at 7:14
  • This part of GNU parallel documentation should be updated then gnu.org/software/parallel/…
    – Farhat
    Commented Feb 20, 2011 at 7:25
  • Ah! I understand now. It looks like there are at least two versions of parallel with different syntax. The one we apparently both have on our systems is from the moreutils package and is not written by GNU. Something's still very strange, though, if you can type man parallel at the command line and get the version with the ::: syntax.
    – Jander
    Commented Feb 20, 2011 at 7:49
  • Wow. That's a very strange bug to run into.
    – Farhat
    Commented Mar 23, 2011 at 16:12
3

By default in ubuntu the parallel behaves as Tollef's implementation (why?). You can check /etc/parallel/config. On the command line you can override this setting and switch it into GNU mode with --gnu. With this switch all examples from the man page should work fine.

Improve this answer
0

I assume you are trying to run both bzip2 ::: *fastq and ls *fastq|parallel bzip2 {} in parallel. Any compelling reason you don't just open up two terminal windows and run them each in their own window?

Improve this answer
1
  • 1
    Depending on how many processors he has, and how many files *fastq matches, he would need quite a few more than two terminal windows. Check out the parallel man page.
    – Jander
    Commented Feb 16, 2011 at 15:51

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .