3

You please imagine I have 16 separate folders named A1 to A16 inside each of them I have a file named aligned.sorted.bam

By these commands I want to convert aligned.sorted.bam to aligned.sam

module load samtools/1.3.2

samtools view -h -o aligned.sam aligned.sorted.bam

Then by below commands convert aligned.sam to counts.txt

module load htseq/0.6.1

htseq-count --stranded=no -q aligned.sam /local/software/DropSeq/STAR_Genomes/STAR_hg38_Genome/metadata/Homo_sapiens.GRCh38.dna.primary_assembly.gtf > counts.txt

Is there anyway having a script going through each folder and doing these commands step wise for me? I am too bad in linux

Thank you for any help

By @cas kindly help I wrote this script

#!/bin/sh

find /temp/hgig/fi1d18/bin/TruSeq300719-139385266/FASTQ_Generation_2019-08-03_04_28_02Z-190932857/335T/ -type f -name aligned.sorted.bam -print0 | \
  xargs -0r -n 1 -P 8 /temp/hgig/fi1d18/bin/TruSeq300719-139385266/FASTQ_Generation_2019-08-03_04_28_02Z-190932857/script.sh


    cd "$(/temp/hgig/fi1d18/bin/TruSeq300719-139385266/FASTQ_Generation_2019-08-03_04_28_02Z-190932857/335T/ "$1")"

    module load samtools/1.3.2
    samtools view -h -o aligned1.sam "$1"

    module load htseq/0.6.1
    htseq-count --stranded=no -q aligned1.sam /local/software/DropSeq/STAR_Genomes/STAR_hg38_Genome/metadata/Homo_sapiens.GRCh38.dna.primary_assembly.gtf > counts.txt

Then in terminal I typed

chmod +x script.sh

But no thing happened

Sorry by latest changes I made script.sh executable then I ran this command

 [fi1d18@cyan01 FASTQ_Generation_2019-08-03_04_28_02Z-190932857]$ find 353T -type f -name aligned.sorted.bam -print0 | \ xargs -0r -n 1 -P 8 script.sh
    -bash:  xargs: command not found

[fi1d18@cyan01 FASTQ_Generation_2019-08-03_04_28_02Z-190932857]$ find T{}/ -type f -name aligned.sorted.bam -print0 | \ xargs -0r -n 1 -P 8 script.sh
-bash:  xargs: command not found
find: `T{}/': No such file or directory
[fi1d18@cyan01 FASTQ_Generation_2019-08-03_04_28_02Z-190932857]$ find T{} -type f -name aligned.sorted.bam -print0 | \ xargs -0r -n 1 -P 8 script.sh
-bash:  xargs: command not found
find: `T{}': No such file or directory
[fi1d18@cyan01 FASTQ_Generation_2019-08-03_04_28_02Z-190932857]$ find T -type f -name aligned.sorted.bam -print0 | \ xargs -0r -n 1 -P 8 script.sh
-bash:  xargs: command not found
find: `T': No such file or directory
[fi1d18@cyan01 FASTQ_Generation_2019-08-03_04_28_02Z-190932857]$

    [fi1d18@cyan01 FASTQ_Generation_2019-08-03_04_28_02Z-190932857]$ ls
    305N  305T  310N  310T  324T  327T  335T  337N  337T  338T  344T  346T  349T  353B  353N  353T  script.sh
    [fi1d18@cyan01 FASTQ_Generation_2019-08-03_04_28_02Z-190932857]$

I removed \

[fi1d18@cyan01 FASTQ_Generation_2019-08-03_04_28_02Z-190932857]$ find T -type f -name aligned.sorted.bam -print0 | xargs -0r -n 1 -P 8 script.sh
find: `T': No such file or directory
[fi1d18@cyan01 FASTQ_Generation_2019-08-03_04_28_02Z-190932857]$ ls
305N  305T  310N  310T  324T  327T  335T  337N  337T  338T  344T  346T  349T  353B  353N  353T  script.sh
[fi1d18@cyan01 FASTQ_Generation_2019-08-03_04_28_02Z-190932857]$

Based on the @cas's latest commends

[fi1d18@cyan01 FASTQ_Generation_2019-08-03_04_28_02Z-190932857]$ find . -type f -name aligned.sorted.bam -print0 | xargs -0r -n 1 -P 8 script.sh
xargs: script.sh: Text file busy
xargs: script.shxargs: : Text file busyscript.sh
: Text file busy
xargs: script.sh: Text file busy
xargs: script.sh: Text file busy
xargs: script.shxargs: : Text file busyscript.sh
: Text file busy
[fi1d18@cyan01 FASTQ_Generation_2019-08-03_04_28_02Z-190932857]$
1

1 Answer 1

3

As a rough guess, something like the following might work:

find ./A{1..16}/ -type f -name aligned.sorted.bam -print0 | 
  xargs -0r -n 1 -P 8 ./myscript.sh

This will run multiple instances of myscript.sh in parallel (-P 8 runs 8 at once. adjust if you have more or less CPU cores/threads - on my threadripper, I'd use -P 32. or use -P 0 to run as many as possible), giving one filename argument to each instance of the script.

myscript.sh would be something like:

#!/bin/sh

cd "$(dirname "$1")"

module load samtools/1.3.2
samtools view -h -o aligned.sam "$1"

module load htseq/0.6.1
htseq-count --stranded=no -q aligned.sam /local/software/DropSeq/STAR_Genomes/STAR_hg38_Genome/metadata/Homo_sapiens.GRCh38.dna.primary_assembly.gtf > counts.txt

This has to be made executable with chmod +x myscript.sh. The xargs command above assumes it is in the current directory. You can put it in your PATH somewhere (~/bin/ is a good place for your own scripts, just add that to $PATH in your ~/.bashrc) and just run myscript.sh rather than ./myscript.sh.

I have no idea what module load ... does, or samtools or htseq-count, so it's entirely possible that i've got this part of the answer wrong. I assume you know what you want to run, so you can fix the script as required.


ps: I recommend testing with a very simple myscript.sh that doesn't write any files.

e.g. something like this:

#!/bin/sh

cd "$(dirname "$1")"

echo process-id $$ is in $(pwd), processing file "$1"
2
  • Sorry, the name of folders in my path are different for example 353N, 353T, 353B, 327T, etc so I did not figure out how to tell the path for different folders
    – Angel
    Sep 10, 2019 at 14:41
  • 1
    I used ./A{1..16}/ because you said I have 16 separate folders named A1 to A16. Anyway, it's easy enough to change the directory argument(s) to find.
    – cas
    Sep 10, 2019 at 15:14

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .