I used the below command

nohup cat fastp-filtered/OZBenth*_R1.fastp.fq.gz > fastp-filtered-merged4racon/merge_R1.fq.gz > merge_R1.out.log 2>&1 &

but unfortunately, the output went to merge_R1.out.log and not to merge_R1.fq.gz.

What did I miss?

  • What do you expect from cat fastp-filtered/OZBenth*_R1.fastp.fq.gz > fastp-filtered-merged4racon/merge_R1.fq.gz This will be mess from compressed files and with high probability will not be able to decompress – Romeo Ninov Mar 15 at 11:21
  • @RomeoNinov No it wouldn't. The concatenated file, when uncompressed, would be concatenation of the uncompressed files. This is likely what is intended and it will work. – Kusalananda Mar 15 at 12:43
  • cat <(tar zcf - a) <(tar zcf - b) | tar -ztf - shows a. – Weijun Zhou Mar 15 at 12:50
  • @WeijunZhou Yes, because two concatenated tar archives don't make much sense. Try cat <( gzip -cf <<<"HELLO" ) <( gzip -cf <<<"WORLD" ) | gzip -dc instead. The result would be the same as cat <( echo HELLO ) <( echo WORLD ) – Kusalananda Mar 15 at 12:58
  • @WeijunZhou What the user is dealing with here is a collection compressed text files containing genomic "fastq" sequence data. They are merging the compressed files together and will later used the merged file for something. There will be no issues with this. – Kusalananda Mar 15 at 13:00

Redirections are processed in a left-to-right fashion.

When executing a command like

somecommand >f1 >f2 >f3

the shell will first create or truncate (empty) the file f1, then f2 and lastly f3. Then it starts somecommand with its standard output connected to f3.

To save the output of a command to multiple files, you would have to duplicate the data for each output file. This is easiest done with tee:

somecommand | tee f1 f2 >f3

This sends the output of somecommand to tee which duplicates it into the two files f1 and f2. The tee utility also produces the same data on its own standard output, and in the command above we simply redirect this into f3.

In your case, you would use

nohup cat fastp-filtered/OZBenth*_R1.fastp.fq.gz 2>&1 |
tee fastp-filtered-merged4racon/merge_R1.fq.gz >merge_R1.out.log &

The 2>&1 makes the error stream from cat be sent to tee as well (this may not be wanted as any errors would corrupt the resulting archive).

tee truncates its output files, just like > would do. Use tee with its -a option to append data to an output file.

It's unlikely that you actually meant for all output from cat to be written to two files though, so assuming that you want to save the errors to the log file and the concatenated archives to the output file, you would instead do

nohup cat fastp-filtered/OZBenth*_R1.fastp.fq.gz \
    >fastp-filtered-merged4racon/merge_R1.fq.gz \
    2>merge_R1.out.log &

without the need for using tee at all.

  • Conceptually correct, but I don't think this is what the OP is asking for. – Weijun Zhou Mar 15 at 12:37
  • @WeijunZhou I just realized this. It's unclear what they want to write into the log file. – Kusalananda Mar 15 at 12:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.