slow down of mpi by Torque

Question

I'm running Torque with Open MPI on a cluster with 30 nodes and 360 cores. I have found that the wall time of mpirun -np N ~./myjob and qsub -l nodes=1:ppn=N mpirun -np N ~./myjob differs many times. For small jobs it grows from 1.2 s to 20 s, from 2 s to 37 s and so on.
For larger jobs the difference becomes important. How to overcome it?

Answer 1

Apparently, it depends on the stuffing of ./myjob. If this script includes the creation and reading of files then they are created N times and their reading causes interference. The slow down is absent if ./myjob is just binary, then the execution time is more or less the same.