2

Hi I guess this is a very basic question:

Is there any specific syntax to do the "piping" when using "bsub"?

I am asking that because when I had an issue like this:

# try to run the same command successfully ran in another host
bsub fastx_trimmer -Q33 -f 1 -l 230 -i myfile.fastq | fastq_quality_trimmer -Q33 -t 18 -l 20 -o Trimmed_file.fastq &

but I met an error:

fastq_quality_trimmer: input file (-) has unknown file format (not FASTA or FASTQ), first character = J (74)

the second command seems not be able find out the output of the first command.

# try to run command without using "|"
bsub fastx_trimmer -Q33 -f 1 -l 230 -i myfile.fastq -o Trimmed_file.fastq
# seemed to work!

bsub fastq_quality_trimmer -Q33 -t 18 -l 20 -i Trimmed_file.fastq -o Trimme_file2.fastq &
# Also seemed to work!

# try to pipe again, did not work...
bsub fastx_trimmer -Q33 -f 1 -l 230 -i myfile.fastq | fastq_quality_trimmer -Q33 -t 18 -l 20 -o Trimme_file2.fastq &

# when call command without bsub, it seemed to work. 
fastx_trimmer -Q33 -f 1 -l 230 -i myfile.fastq | fastq_quality_trimmer -Q33 -t 18 -l 20 -o Trimme_file2.fastq &
# so the issue seemed to be "bsub"

I would like to use bsub because my host is using gate node (the previous host has no gate node, so it does not matter whether I use "bsub" or not), and I do not want to create "traffic jam" by running command in the gate node. Any suggestion?

3

The bsub command will output the job submission ID and the name of the queue that it has been submitted to, if the submission is successful.

bsub will not provide you with the output of fastx_trimmer, which means that it can not be used for direct input to fastq_quality_trimmer via a pipe.

You will need to submit the whole pipeline to the queue to have it work properly, either by putting the commands into a shell script and submit that (ideally, if you plan on running it many times), or by quoting the whole pipeline properly in the call to bsub.

2

I received suggestions from my friend by email. Two solutions: 1. put the real command in "".

bsub "fastx_trimmer -Q33 -f 1 -l 230 -i myfile.fastq | fastq_quality_trimmer -Q33 -t 18 -l 20 -o Trimmed_file.fastq”
  1. write a bash script for bsub.

I tried solution 1. it worked very well. Put it here in case other people have similar issues.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.