0

I am using a program called minimap for aligning reads to a reference genome (computational biology problem). This program requires as input two files as positional arguments: first a reference in .fasta format followed by a file with reads, in fastqformat.

The thing is that I have multiple fastq files. One option is to first concatenate them all with cat *.fastq > cat_all.fastq then call minimap. But I would like to concatenate them all and pipe this file to minimap, without saving it to cat_all.fastq, so that I don't waste storage space.

Is there a way I can store the cat output as a placeholder (without saving it) then pipe to minimap, something like?

cat *.fastq | minimap2 -ax map-ont /path/to/a/file.fasta {placeholder_for_the_cat_command} > output_file.sam

Below is an example of how I call minimap on a single (concatenated) file called PD180425_cat_all.fastq:

minimap2 -ax map-ont /path/to/a/file.fasta PD180425_cat_all.fastq > PD180425_aligned_minimap.sam
4

Yes, in the bash shell you may use a process substitution:

minimap -ax map-ont /path/to/fasta.file <( cat *.fastq ) >output.sam

The <( ... ) is a process substitution. It will be replaced by the pathname of a named pipe (something like /dev/fd/XXX) that will produce the output of the command within when reading from it. The output of the command in the process substitution is not stored on disk.

As long as the minimap tool does not need to jump back and forth in the fastq data, but just reads it sequentially, this will probably work.

Doing this without process substitutions (this would also work with sh or any POSIX shell):

mkfifo fastq_data
cat *.fastq >fastq_data &
minimap -ax map-ont /path/to/fasta.file fastq_data >output.sam
rm fastq_data

This does pretty much the same thing as the first command. It creates a named pipe and concatenates the fastq data to it (the cat is running as a background job until all its output has been read by minimap, and then terminates). The minimap tool is then invoked with the named pipe for the fastq data. When done, the named pipe is removed.

Reading from the fastq_data pipe means reading directly from the cat command, not from some temporary file. Again, the result of the cat is never stored on disk.

If the minimap tool for some reason needs a fastq file with a particular filename suffix, this may be the best option. Just name your named pipe as data.fastq or similar.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.