3

You please imagine I have 16 separate folders named A1 to A16 inside each of them I have a file named aligned.sorted.bam

By these commands I want to convert aligned.sorted.bam to aligned.sam

module load samtools/1.3.2

samtools view -h -o aligned.sam aligned.sorted.bam

Then by below commands convert aligned.sam to counts.txt

module load htseq/0.6.1

htseq-count --stranded=no -q aligned.sam /local/software/DropSeq/STAR_Genomes/STAR_hg38_Genome/metadata/Homo_sapiens.GRCh38.dna.primary_assembly.gtf > counts.txt

Is there anyway having a script going through each folder and doing these commands step wise for me? I am too bad in linux

Thank you for any help

By @cas kindly help I wrote this script

#!/bin/sh

find /temp/hgig/fi1d18/bin/TruSeq300719-139385266/FASTQ_Generation_2019-08-03_04_28_02Z-190932857/335T/ -type f -name aligned.sorted.bam -print0 | \
  xargs -0r -n 1 -P 8 /temp/hgig/fi1d18/bin/TruSeq300719-139385266/FASTQ_Generation_2019-08-03_04_28_02Z-190932857/script.sh


    cd "$(/temp/hgig/fi1d18/bin/TruSeq300719-139385266/FASTQ_Generation_2019-08-03_04_28_02Z-190932857/335T/ "$1")"

    module load samtools/1.3.2
    samtools view -h -o aligned1.sam "$1"

    module load htseq/0.6.1
    htseq-count --stranded=no -q aligned1.sam /local/software/DropSeq/STAR_Genomes/STAR_hg38_Genome/metadata/Homo_sapiens.GRCh38.dna.primary_assembly.gtf > counts.txt

Then in terminal I typed

chmod +x script.sh

But no thing happened

Sorry by latest changes I made script.sh executable then I ran this command

 [fi1d18@cyan01 FASTQ_Generation_2019-08-03_04_28_02Z-190932857]$ find 353T -type f -name aligned.sorted.bam -print0 | \ xargs -0r -n 1 -P 8 script.sh
    -bash:  xargs: command not found

[fi1d18@cyan01 FASTQ_Generation_2019-08-03_04_28_02Z-190932857]$ find T{}/ -type f -name aligned.sorted.bam -print0 | \ xargs -0r -n 1 -P 8 script.sh
-bash:  xargs: command not found
find: `T{}/': No such file or directory
[fi1d18@cyan01 FASTQ_Generation_2019-08-03_04_28_02Z-190932857]$ find T{} -type f -name aligned.sorted.bam -print0 | \ xargs -0r -n 1 -P 8 script.sh
-bash:  xargs: command not found
find: `T{}': No such file or directory
[fi1d18@cyan01 FASTQ_Generation_2019-08-03_04_28_02Z-190932857]$ find T -type f -name aligned.sorted.bam -print0 | \ xargs -0r -n 1 -P 8 script.sh
-bash:  xargs: command not found
find: `T': No such file or directory
[fi1d18@cyan01 FASTQ_Generation_2019-08-03_04_28_02Z-190932857]$

    [fi1d18@cyan01 FASTQ_Generation_2019-08-03_04_28_02Z-190932857]$ ls
    305N  305T  310N  310T  324T  327T  335T  337N  337T  338T  344T  346T  349T  353B  353N  353T  script.sh
    [fi1d18@cyan01 FASTQ_Generation_2019-08-03_04_28_02Z-190932857]$

I removed \

[fi1d18@cyan01 FASTQ_Generation_2019-08-03_04_28_02Z-190932857]$ find T -type f -name aligned.sorted.bam -print0 | xargs -0r -n 1 -P 8 script.sh
find: `T': No such file or directory
[fi1d18@cyan01 FASTQ_Generation_2019-08-03_04_28_02Z-190932857]$ ls
305N  305T  310N  310T  324T  327T  335T  337N  337T  338T  344T  346T  349T  353B  353N  353T  script.sh
[fi1d18@cyan01 FASTQ_Generation_2019-08-03_04_28_02Z-190932857]$

Based on the @cas's latest commends

[fi1d18@cyan01 FASTQ_Generation_2019-08-03_04_28_02Z-190932857]$ find . -type f -name aligned.sorted.bam -print0 | xargs -0r -n 1 -P 8 script.sh
xargs: script.sh: Text file busy
xargs: script.shxargs: : Text file busyscript.sh
: Text file busy
xargs: script.sh: Text file busy
xargs: script.sh: Text file busy
xargs: script.shxargs: : Text file busyscript.sh
: Text file busy
[fi1d18@cyan01 FASTQ_Generation_2019-08-03_04_28_02Z-190932857]$
3

As a rough guess, something like the following might work:

find ./A{1..16}/ -type f -name aligned.sorted.bam -print0 | 
  xargs -0r -n 1 -P 8 ./myscript.sh

This will run multiple instances of myscript.sh in parallel (-P 8 runs 8 at once. adjust if you have more or less CPU cores/threads - on my threadripper, I'd use -P 32. or use -P 0 to run as many as possible), giving one filename argument to each instance of the script.

myscript.sh would be something like:

#!/bin/sh

cd "$(dirname "$1")"

module load samtools/1.3.2
samtools view -h -o aligned.sam "$1"

module load htseq/0.6.1
htseq-count --stranded=no -q aligned.sam /local/software/DropSeq/STAR_Genomes/STAR_hg38_Genome/metadata/Homo_sapiens.GRCh38.dna.primary_assembly.gtf > counts.txt

This has to be made executable with chmod +x myscript.sh. The xargs command above assumes it is in the current directory. You can put it in your PATH somewhere (~/bin/ is a good place for your own scripts, just add that to $PATH in your ~/.bashrc) and just run myscript.sh rather than ./myscript.sh.

I have no idea what module load ... does, or samtools or htseq-count, so it's entirely possible that i've got this part of the answer wrong. I assume you know what you want to run, so you can fix the script as required.


ps: I recommend testing with a very simple myscript.sh that doesn't write any files.

e.g. something like this:

#!/bin/sh

cd "$(dirname "$1")"

echo process-id $$ is in $(pwd), processing file "$1"
  • Sorry, the name of folders in my path are different for example 353N, 353T, 353B, 327T, etc so I did not figure out how to tell the path for different folders – Feresh Teh Sep 10 at 14:41
  • 1
    I used ./A{1..16}/ because you said I have 16 separate folders named A1 to A16. Anyway, it's easy enough to change the directory argument(s) to find. – cas Sep 10 at 15:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.