We've just switched to using SLURM and I would like to submit a series of jobs using a loop and sbatch. Previously, I could use a variable as part of the output file names. I've been trying to do this in sbatch using --export to pass in the variable but can't get the variable to be interpolated for the std error/output file names. I think it's working for the job name (-J) and --wrap parts though.

for i in *fastq.gz; do sbatch \
--export=i=$i --error='$i.eo%j' --output='$i.eo%j'  \
-J trim_`basename ${i}` \
--wrap="fastq_trim_single.sh ${i}" \
; done

Produces output files with $i in the name: $i.eo68065. Is it possible to do what I want to do?

Note: ideally, I would like to use basename $i rather than $i for the output file name.


Not sure if this fits your use-case, but have you considered using a job array? When using a job array, the variable SLURM_ARRAY_TASK_ID will be assigned to the current index of the job array, which you could use in the --error and --output commands.

Here are some examples that can be useful: https://slurm.schedmd.com/job_array.html

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.