1

I have attempted to write a shell script and it is giving an error which I cannot understand. The script goes like this:

#!/bin/bash
while read id 
do 
cat demuxR1/$id.barcode.in.R1.R1.fastq demuxR2/$id.barcode.in.R2.R2.fastq > alldemulti/$id.forwardread.fastq 
done < R1_new.txt

the files in demuxR1 and demuxR2 folder contain files enter image description here named as shown. The output has to be redirected into another folder called alldemulti. When I am running the script it complains

cat: demuxR1/SAE1903.barcode.in.R1.R1.fastq.barcode.in.R1.R1.fastq: No such file or directory

The R1_new.txt conatins the file names :

    SAE1903.barcode.in.R1.R1.fastq
    SAE1903.barcode.in.R1.R2.fastq
    SAE2000.barcode.in.R1.R1.fastq
    SAE2000.barcode.in.R1.R2.fastq
    SAE2103.barcode.in.R1.R1.fastq
    SAE2103.barcode.in.R1.R2.fastq
    SAE2203.barcode.in.R1.R1.fastq
    SAE2203.barcode.in.R1.R2.fastq
    SAE2303.barcode.in.R1.R1.fastq
    SAE2303.barcode.in.R1.R2.fastq
    SAE2403.barcode.in.R1.R1.fastq
    SAE2403.barcode.in.R1.R2.fastq
    SAE2603.barcode.in.R1.R1.fastq
    SAE2603.barcode.in.R1.R2.fastq
    SAE2803.barcode.in.R1.R1.fastq
    SAE2803.barcode.in.R1.R2.fastq

Can someone tell me what am I doing wrong here.

Content of demuxR1 directory:

total 4426752
-rwxrwxrwx 1 moloy moloy        561 Oct 15 08:46 barcode_R1.txt
-rwxrwxrwx 1 moloy moloy     231092 Oct 15 09:14 SAE1903.barcode.in.R1.R1.fastq
-rwxrwxrwx 1 moloy moloy     235734 Oct 15 09:14 SAE1903.barcode.in.R1.R2.fastq
-rwxrwxrwx 1 moloy moloy     917692 Oct 15 09:14 SAE2000.barcode.in.R1.R1.fastq
-rwxrwxrwx 1 moloy moloy     935522 Oct 15 09:14 SAE2000.barcode.in.R1.R2.fastq
-rwxrwxrwx 1 moloy moloy    1129320 Oct 15 09:14 SAE2103.barcode.in.R1.R1.fastq
-rwxrwxrwx 1 moloy moloy    1151554 Oct 15 09:14 SAE2103.barcode.in.R1.R2.fastq
-rwxrwxrwx 1 moloy moloy     747736 Oct 15 09:14 SAE2203.barcode.in.R1.R1.fastq
-rwxrwxrwx 1 moloy moloy     762462 Oct 15 09:14 SAE2203.barcode.in.R1.R2.fastq
-rwxrwxrwx 1 moloy moloy     446168 Oct 15 09:14 SAE2303.barcode.in.R1.R1.fastq
-rwxrwxrwx 1 moloy moloy     455044 Oct 15 09:14 SAE2303.barcode.in.R1.R2.fastq
-rwxrwxrwx 1 moloy moloy     378862 Oct 15 09:14 SAE2403.barcode.in.R1.R1.fastq
-rwxrwxrwx 1 moloy moloy     386440 Oct 15 09:14 SAE2403.barcode.in.R1.R2.fastq
-rwxrwxrwx 1 moloy moloy     960420 Oct 15 09:14 SAE2603.barcode.in.R1.R1.fastq
-rwxrwxrwx 1 moloy moloy     979364 Oct 15 09:14 SAE2603.barcode.in.R1.R2.fastq
-rwxrwxrwx 1 moloy moloy    1256116 Oct 15 09:14 SAE2803.barcode.in.R1.R1.fastq
-rwxrwxrwx 1 moloy moloy    1281112 Oct 15 09:14 SAE2803.barcode.in.R1.R2.fastq
-rwxrwxrwx 1 moloy moloy 2258319418 Oct 15 09:14 unknown.R1.fastq
-rwxrwxrwx 1 moloy moloy 2257956950 Oct 15 09:14 unknown.R2.fastq

Content of demuxR2 directory:

total 4427776
-rwxrwxrwx 1 moloy moloy        560 Oct 14 12:48 barcode_R2.txt
-rwxrwxrwx 1 moloy moloy     261854 Oct 15 10:11 SAE1903.barcode.in.R2.R1.fastq
-rwxrwxrwx 1 moloy moloy     256718 Oct 15 10:11 SAE1903.barcode.in.R2.R2.fastq
-rwxrwxrwx 1 moloy moloy    1284362 Oct 15 10:11 SAE2000.barcode.in.R2.R1.fastq
-rwxrwxrwx 1 moloy moloy    1259358 Oct 15 10:11 SAE2000.barcode.in.R2.R2.fastq
-rwxrwxrwx 1 moloy moloy    1208280 Oct 15 10:11 SAE2103.barcode.in.R2.R1.fastq
-rwxrwxrwx 1 moloy moloy    1184716 Oct 15 10:11 SAE2103.barcode.in.R2.R2.fastq
-rwxrwxrwx 1 moloy moloy    1028542 Oct 15 10:11 SAE2203.barcode.in.R2.R1.fastq
-rwxrwxrwx 1 moloy moloy    1008462 Oct 15 10:11 SAE2203.barcode.in.R2.R2.fastq
-rwxrwxrwx 1 moloy moloy     627558 Oct 15 10:11 SAE2303.barcode.in.R2.R1.fastq
-rwxrwxrwx 1 moloy moloy     615364 Oct 15 10:11 SAE2303.barcode.in.R2.R2.fastq
-rwxrwxrwx 1 moloy moloy     414098 Oct 15 10:11 SAE2403.barcode.in.R2.R1.fastq
-rwxrwxrwx 1 moloy moloy     405878 Oct 15 10:11 SAE2403.barcode.in.R2.R2.fastq
-rwxrwxrwx 1 moloy moloy    1231900 Oct 15 10:11 SAE2603.barcode.in.R2.R1.fastq
-rwxrwxrwx 1 moloy moloy    1208060 Oct 15 10:11 SAE2603.barcode.in.R2.R2.fastq
-rwxrwxrwx 1 moloy moloy    1784282 Oct 15 10:11 SAE2803.barcode.in.R2.R1.fastq
-rwxrwxrwx 1 moloy moloy    1749406 Oct 15 10:11 SAE2803.barcode.in.R2.R2.fastq
-rwxrwxrwx 1 moloy moloy 2256301114 Oct 15 10:11 unknown.R1.fastq
-rwxrwxrwx 1 moloy moloy 2256668336 Oct 15 10:11 unknown.R2.fastq
  • What's in R1_new.txt? – nohillside Oct 15 '18 at 13:55
  • it contain the file names – user1738234 Oct 15 '18 at 13:58
  • it contains these names : SAE1903.barcode.in.R1.R1.fastq SAE1903.barcode.in.R1.R2.fastq SAE2000.barcode.in.R1.R1.fastq SAE2000.barcode.in.R1.R2.fastq SAE2103.barcode.in.R1.R1.fastq SAE2103.barcode.in.R1.R2.fastq SAE2203.barcode.in.R1.R1.fastq SAE2203.barcode.in.R1.R2.fastq SAE2303.barcode.in.R1.R1.fastq SAE2303.barcode.in.R1.R2.fastq SAE2403.barcode.in.R1.R1.fastq SAE2403.barcode.in.R1.R2.fastq SAE2603.barcode.in.R1.R1.fastq SAE2603.barcode.in.R1.R2.fastq SAE2803.barcode.in.R1.R1.fastq SAE2803.barcode.in.R1.R2.fastq – user1738234 Oct 15 '18 at 14:00
  • 1
    Add updated information to the question itself, not in comments (where formatting gets lost). It looks like you're reading filenames, but your code expects only the SAE2000 bit in front of the first dot. Add the formatted input file in the question (or part of it) and all will be clearer. – Kusalananda Oct 15 '18 at 14:06
  • should I start a new thread I dont see any options for editing my original question – user1738234 Oct 15 '18 at 14:17
2

I assume your input file actually only contains one file name per line and looks like this:

SAE1903.barcode.in.R1.R1.fastq
SAE1903.barcode.in.R1.R2.fastq
SAE2000.barcode.in.R1.R1.fastq
...

Your script reads each line into id, so after the first line the value of id is SAE1903.barcode.in.R1.R1.fastq. If you now use this as $id.barcode.in.R1.R1.fastq the variable is replaced by the value, and then the rest of the string (.barcode.in.R1.R1.fastq) gets appended. Which then, of course, leads to the error because there is no file called SAE1903.barcode.in.R1.R1.fastq.barcode.in.R1.R1.fastq.

A simple solution (which assumes a file with an even number of lines) to overcome this could be

#!/bin/bash
while read line1; do
    read line2
    cat "demuxR1/$line1" "demuxR2/$line2" > "alldemulti/${line1%%.*}.forwardread.fastq"
done < R1_new.txt

The ${line1%%.*} will be replaced by e.g. SAE1903, i.e. the word before the first dot in the string read from the file.

  • @Kusalnanda I tried the script with the suggested changes but curiously it complains "cat: demuxR2/SAE1903.barcode.in.R1.R2.fastq: No such file or directory". There are no files by such a name in demuxR2. The script that I ran was #!/bin/bash while read line1; do read line2 cat "demuxR1/$line1" "demuxR2/$line2" > "alldemulti/SAE1903.forwardread.fastq" done < R1_new.txt. Myquestion is that for the next round of processing I need the output of all the ID's together in one file. How can I achieve this using this script ? – user1738234 Oct 16 '18 at 13:09
  • @user1738234 The script just is a copy of the one you've included in the question. In which directory is SAE1903.barcode.in.R1.R2.fastq (or more generally all the ...R2.fastq) file stored, and is the name correct? – nohillside Oct 16 '18 at 14:05
  • @user1738234 According to the screenshot in the question all input files seem to be in demuxR1. If this is the case replace demuxR2 with demuxR1 in the code snippet. If not, please update your question to show more clearly which files are in demuxR1 and which in demuxR2 (maybe just copy/paste the result of ls -l demuxR1 demuxR2) – nohillside Oct 16 '18 at 14:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.