I have an account on a compute cluster which uses the SLURM scheduler. I have some jobs in the queue and I'm using the "watch" command to see their status:

watch squeue -u myUserName

Does constantly running this command put any significant stress on the scheduler?


Whenever you're just watching other jobs, and don't want to get in the way, like with top or iostat or systat or whatever, it's always nice to use nice:

nice watch squeue -u myUserName

That runs your process at a reduced priority (compared to the default priority level). Assuming that the cluster jobs are running at normal priority, nice-ing your process tells the scheduler that you are willing to accept only whatever spare CPU time is left over after all the jobs above you have gotten their CPU time.

If you're doing low-priority stuff at all on a heavily-loaded machine, it's always considerate to:

$ nice bash -l

so that everything you do runs at a lower priority than the "real jobs" the cluster is already running.

| improve this answer | |
  • It may be the Slurm scheduler that the user refers to, not the kernel's scheduler. If the squeue command is demanding on the Slurm scheduler, it may interfere with the scheduling of jobs on the Slurm cluster. – Kusalananda May 26 '19 at 7:57

Each squeue command fires off a RPC to the Slurm Control Daemon (slurmctld). This will put a load on the control daemon, but without knowing how much power your controller has or knowing how loaded your system is, it is difficult to tell how much a constant squeue will affect it.

If you have an organization of 500+ users constantly running a watch "squeue -u $USER_NAME", then that will put significant load on the system vs just you running it on a powerful system.

A better use of time is if you were to properly time out your work so that if you know after x amount of min you should be in the completing state you can run something like a sleep x; squeue -u $USER_NAME. Or even go one step further and look into how to incorporate e-mail status messages within your jobs that will ping you when a certain task is complete or your job has reached a certain stage. The watch command is also configurable for the frequency of when it runs using the -n option.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.