0

How can I run the same command in mac terminal for multiple files in a folder? The files are named like 24538_7#1_paired1.fq, 24538_7#1_paired2.fq, 24538_7#2_paired1.fq, 24538_7#2_paired2.fq, 24538_7#3_paired1.fq, 24538_7#3_paired2.fq, and so on.

The command is:

STAR --runThreadN 12 --genomeDir indices/STAR --twopassMode Basic --readFilesIn data/24538_7#1_paired1.fq data/24538_7#1_paired2.fq --outFileNamePrefix results/STAR/ 

Since filename involves a counter, so, obviously the filenames need to be changed.

I was trying to write command, but it's giving segmentation fault. My effort for the command is below:

for file in 24538_7#*.fq; do STAR --runThreadN 12 --genomeDir indices/STAR --twopassMode Basic --readFilesIn data/"${file%.fq}_paired1.fq" data/"${file%.fq}_paired2.fq" --outFileNamePrefix results/STAR/ ; done

(PS- I use MacOS)

  • Put some debugging echo into the loop to see which execution of STAR gives you the segfault – nohillside Jun 12 '18 at 13:37
  • can you please elaborate @nohillside – Akhil Verma Jun 12 '18 at 13:40
  • Well, I assume that one of your calls to STAR crashes so you need to look at its input. – nohillside Jun 12 '18 at 13:44
  • Will each set of files only have 2 members in the pair? Both 1 and 2? – Jesse_b Jun 12 '18 at 13:52
  • Yes @Jesse_b, every file is paired. Please see question above for the type of file names. – Akhil Verma Jun 12 '18 at 13:56
1

Loop over all the paired1.fq files and for each such file, use the name to compute the name of the corresponding paired2.fq file. Then call your program with these:

for paired1 in data/*paired1.fq; do
    paired2="${paired1%1.fq}2.fq"  # remove 1.fq from end of name and replace with 2.fq

    if [ ! -f "$paired2" ]; then
        printf 'Missing file:\t%s\n' "$paired2" >&2
        continue
    fi

    prefix="${paired1%_*}" # remove last underscore and everything after
    prefix="${prefix##*/}" # remove directory name from prefix

    # If $paired1 is the string "data/24538_7#1_paired1.fq", then
    # $prefix should now be "24538_7#1"

    mkdir -p "results/STAR/$prefix"

    STAR --runThreadN 12 --genomeDir indices/STAR --twopassMode Basic \
         --readFilesIn "$paired1" "$paired2" \
         --outFileNamePrefix "results/STAR/$prefix/"
done
  • this is working now but I got only 1 output file. For that I think we may have to include a counter with command(--outFileNamePrefix) also. Can you help me with that? – Akhil Verma Jun 12 '18 at 14:50
  • @AkhilVerma I don't know what you mean by "counter". How does the --outFileNamePrefix option work? Note that I don't know what STAR is. – Kusalananda Jun 12 '18 at 14:58
  • by counter I mean * and I tried adding it after results/STAR* but it still gave only 1 output file. STAR is an alignment tool for RNA seq analysis. Regarding the --outFileNamePrefix: --outFileNamePrefix default: ./ string: output files name prefix (including full or relative path). Can only be defined on the command line. Link to STAR Manual :(github.com/alexdobin/STAR/blob/master/doc/STARmanual.pdf) – Akhil Verma Jun 12 '18 at 15:05
  • @AkhilVerma See updated answer. – Kusalananda Jun 12 '18 at 15:14
  • I ran it but still no luck with it. It's still giving only 1 output file. It's basically rewriting for each file. Can you please look into STAR manual and help me with it. (github.com/alexdobin/STAR/blob/master/doc/STARmanual.pdf) – Akhil Verma Jun 12 '18 at 15:24
0

Your parameter expansion isn't trimming enough.

You are setting file to:

file=24538_7#1_paired1.fq

Then trimming .fq:

$ echo ${file%.fq}
24538_7#1_paired1

Then adding _paired1.fq:

$ echo "${file%.fq}_paired1.fq"
24538_7#1_paired1_paired1.fq

Try like this:

for file in 24538_7#*.fq; do 
    STAR --runThreadN 12 --genomeDir indices/STAR --twopassMode Basic --readFilesIn data/"${file%_paired?.fq}_paired1.fq" data/"${file%_paired?.fq}_paired2.fq" --outFileNamePrefix results/STAR/
done

This is going to run twice per pair though since each pair has two files. I think that may not be what you need.


To run once per pair you could do something like this:

#!/bin/bash

files=( 24538_7#*.fq )
files=( printf '%s\n' "${files[@]#24538_7#}" | sort -n | awk -F_ '{print $1}' | uniq )

for n in "${files[@]}"; do
    STAR --runThreadN 12 --genomeDir indices/STAR --twopassMode Basic --readFilesIn data/"24538_7#${n}_paired1.fq" data/"24538_7#${n}_paired2.fq" --outFileNamePrefix results/STAR/ 
done
  • Sorry, but I'm getting error for each file like this: EXITING because of fatal input ERROR: could not open readFilesIn=/Volumes/Pegasus/Users/Akhil/SangerSeqData_Torp_Dec2017/Run24538_7_Plate3145_TorpWt/FastQ/testSTAR/24538_7#24538_7#1_paired1.fq_paired1.fq Jun 12 15:34:01 ...... FATAL ERROR, exiting @Jesse_b – Akhil Verma Jun 12 '18 at 14:36
  • @AkhilVerma: Which one are you using? – Jesse_b Jun 12 '18 at 14:55
  • The below one @Jesse_b – Akhil Verma Jun 12 '18 at 15:07
  • Are you running it exactly as I have posted? Can you remove the loop and just do: echo "${files[@]}" after the two variable assignments? It should just be a list of numbers – Jesse_b Jun 12 '18 at 15:21
  • After running files=( sort -n < <(printf '%s\n' "${files[@]#24538_7#}") | awk -F_ '{print $1}' | uniq ) I'm getting an error: -bash: syntax error near unexpected token <'` Because of which for n in "${files[@]}"; do STAR --runThreadN 12 --genomeDir indices/STAR --twopassMode Basic --readFilesIn data/"24538_7#${n}_paired1.fq" data/"24538_7#${n}_paired2.fq" --outFileNamePrefix results/STAR/ done is generating the other errors: EXITING because of fatal input ERROR: could not open readFilesIn=/Volumes/Pegasus/Users/Akhil/SangerSeqData_Torp_Dec2017/Run24538_7_Pl....(see above) – Akhil Verma Jun 12 '18 at 15:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.