1

Suppose I have two Python scripts: test1.py and test2.py

How do I write a SLURM script to run these files on two different nodes simultaneously?

Note that: test1.py and test2.py are independent of each other.

1 Answer 1

0

In a slum script, you can use an array and define two or more scripts as an array element in the Slurm script. An example can be like following:

#!/bin/bash
#
#SBATCH --ntasks=1
#SBATCH --partition xxxx
#SBATCH --time=6:00:00
#SBATCH --array=1-300

# Job array size will be number of lines in file divided by 
# number of lines chosen below

START=$SLURM_ARRAY_TASK_ID
NUMLINES=100
STOP=$((SLURM_ARRAY_TASK_ID*NUMLINES))
START="$(($STOP - $(($NUMLINES - 1))))"

echo "START=$START"
echo "STOP=$STOP"

for (( N = $START; N <= $STOP; N++ ))
do
    LINE=$(sed -n "$N"p File.txt)
    call-program-name-here $LINE
done

In this case, you should define resources carefully to run those files on different nodes, or you should have the correct scheduling configuration on slurm.conf

https://slurm.schedmd.com/faq.html#multi_job

If you want to allocate whole nodes to jobs, configure configure SelectType=select/linear

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.