0

I wanted to run 4 python codes each using 2 processors. (Parallelisation is done inside the MKL.)

#!/bin/bash
#SBATCH -N 1
#SBATCH --ntasks-per-node=4
#SBATCH --cpus-per-task=2

export OMP_NUM_THREADS=2

srun -n 2 python doSVD 1 &
srun -n 2 python doSVD 2 &
srun -n 2 python doSVD 3 &
srun -n 2 python doSVD 4 &

wait

When I try this two copies of each code runs instead of single code that uses 2 cores. What is the right way to spawn 4 codes (each one using 2 processors).

The following works. But it runs each code one after the other.

#!/bin/bash
#SBATCH -N 1
#SBATCH --ntasks-per-node=1
#SBATCH --cpus-per-task=2

export OMP_NUM_THREADS=2

srun python doSVD 1 
srun python doSVD 2 
srun python doSVD 3 
srun python doSVD 4 

wait

1 Answer 1

1

This answer is a bit late but this could still serve others:

The problem here I think is that you are asking for 4 tasks with --ntasks-per-node=4 and ask each python process to run 2 tasks (srun -n 2). You want instead each python process to be 1 task running 2 threads.

You should try with:

#!/bin/bash
#SBATCH -N 1
#SBATCH --ntasks-per-node=4
#SBATCH --cpus-per-task=2

export OMP_NUM_THREADS=2

srun -n 1 -c 2 python doSVD 1 &
srun -n 1 -c 2 python doSVD 2 &
srun -n 1 -c 2 python doSVD 3 &
srun -n 1 -c 2 python doSVD 4 &

wait

I think -n 1 is the default but it doesn't hurt to write it. If this still does not work we could try with job arrays.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.