I have a block of code that works as I want it to but is really inefficient.

for f in *.GeneSet; do
#Figure out the cell type by removing "Genes.GeneSet" from the end of the filename

  # Process the file
  for i in `seq 1 22`; do
    python make_annot.py \
    --gene-set-file "$f" \
    --gene-coord-file  ENSG_coord.txt \
    --bimfile 1000G_EUR_Phase3_plink/1000G.EUR.QC.${i}.bim \
    --annot-file ${cell_type}.${i}.annot.gz

This works well. Except that because its a nested for-loop and very inefficient. I would like to use $SGE_TASK_ID to run each iteration independently. How can I achieve this?

  • I don't see "SGE_TASK_ID" in the existing code...?
    – Jeff Schaller
    Mar 25, 2019 at 17:45
  • yes, thats because I haven't implemented it. I am wondering how it can be used to make my code more efficient. Did I write the question incorrectly?
    – Workhorse
    Mar 25, 2019 at 17:50
  • Googling that name comes up with references to Sun GridEngine; are you trying to use that to run your code?
    – Jeff Schaller
    Mar 25, 2019 at 17:53
  • Mhat values may $SGE_TASK_ID have? If they are between 1 and 22, then you could simply use that instead of $i in your for loop. Also, aren't you caring about the X, Y and MT chromosomes?
    – Kusalananda
    Mar 25, 2019 at 21:42


You must log in to answer this question.

Browse other questions tagged .