SLURM: How do I submit multilple OpenMP parallel codes in a job script

Question

I wanted to run 4 python codes each using 2 processors. (Parallelisation is done inside the MKL.)

#!/bin/bash
#SBATCH -N 1
#SBATCH --ntasks-per-node=4
#SBATCH --cpus-per-task=2

export OMP_NUM_THREADS=2

srun -n 2 python doSVD 1 &
srun -n 2 python doSVD 2 &
srun -n 2 python doSVD 3 &
srun -n 2 python doSVD 4 &

wait

When I try this two copies of each code runs instead of single code that uses 2 cores. What is the right way to spawn 4 codes (each one using 2 processors).

The following works. But it runs each code one after the other.

#!/bin/bash
#SBATCH -N 1
#SBATCH --ntasks-per-node=1
#SBATCH --cpus-per-task=2

export OMP_NUM_THREADS=2

srun python doSVD 1 
srun python doSVD 2 
srun python doSVD 3 
srun python doSVD 4 

wait

Answer 1

This answer is a bit late but this could still serve others:

The problem here I think is that you are asking for 4 tasks with --ntasks-per-node=4 and ask each python process to run 2 tasks (srun -n 2). You want instead each python process to be 1 task running 2 threads.

You should try with:

#!/bin/bash
#SBATCH -N 1
#SBATCH --ntasks-per-node=4
#SBATCH --cpus-per-task=2

export OMP_NUM_THREADS=2

srun -n 1 -c 2 python doSVD 1 &
srun -n 1 -c 2 python doSVD 2 &
srun -n 1 -c 2 python doSVD 3 &
srun -n 1 -c 2 python doSVD 4 &

wait

I think -n 1 is the default but it doesn't hurt to write it. If this still does not work we could try with job arrays.