I have a submit script that does several things in a sequential manner (at least that is how I tried to write it). but when I run my script the final operation behaves according to the completion order of the previous operation. attached is a part of the script.

#SBATCH --job-name=cff_cg                                                                                                                                                                      
#SBATCH --nodes=1 --ntasks=20                                                                                                                                                                  
#SBATCH --time=0-06:00:00                                                                                                                                                                      
#SBATCH --mem-per-cpu=4gb                                                                                                                                                                      
#SBATCH --constraint=avx512                                                                                                                                                                    
#SBATCH --partition=sixhour                                                                                                                                                                    
#SBATCH --output=1_%a.out                                                                                                                                                                      
#SBATCH --array=1-100
if [[ -f $main_path/surf_$fold/data.lammps ]] ; then
    echo 'EXISTS'
    check=$(awk 'FNR == 21 {print $2}' $main_path/surf_$fold/data.lammps)
    echo "$check"
    if [[ $check == '1' ]] ; then
        awk 'f{print $3; f=0} /Energy/{f=1}' $main_path/surf_$fold/log.lammps > $main_path/surf_$fold/min_energy.dat
        awk '/Energy/{ if (secondline==0) { secondline=1; next; } else { getline; print $3; } }' $main_path/surf_$fold/log.lammps > $main_path/surf_$fold/min_energy_$fold.dat


cat $main_path/surf_$fold/min_energy_$fold.dat >> $main_path/energy_final_new.dat

exit 0

the way I should be getting the final output file "energy_final_new.dat" is according to the order of the array, i.e, values in the order of 001,002,003,004 but instead I get values with a mixed order (once i got like 002,004,003,001 etc.) I believe this happened in the order of each job finishes but I need them in the order of the array_id. can someone help me with this? P.S - SBATCH makes the script run in parallel.

  • The script you've given us runs sequentially. Perhaps this SBATCH thing runs multiple instances of the script in parallel? If so, you need to describe that. Notice also that $main_path and $fold are not defined in your script. Dec 6, 2021 at 21:01
  • This is only a part of the script, i defined the $main_path and $fold in my main script. yes SBATCH make it run in parallel. how would that affect the order of the output? can ypu please explain?
    – Xdrake
    Dec 6, 2021 at 21:06
  • If SBATCH makes multiple instances of your script run in parallel then you yourself need to identify ways to control its parallelisation. It's not a standard command Dec 6, 2021 at 21:16
  • This looks like a slurm script. If you need this to run in order, you are using slurm wrong. It would be better to save each instance of the array output in a separate file, and then have something external combine them in order.
    – user10489
    Dec 7, 2021 at 0:15
  • I figured it out. thanks for all the help
    – Xdrake
    Dec 8, 2021 at 3:47


You must log in to answer this question.

Browse other questions tagged .