54 votes
Accepted

How to submit a job to a specific node using Slurm's sbatch command?

I figured it out. You need to use -w node0xx or --nodelist=node0xx.
Amir's user avatar
  • 1,049
26 votes
Accepted

SLURM: Custom standard output name

Here is my take away from previous answers %j gives job id %x gives job name I don't know how to get the date in the desired format. Job ID kind of serves as unique identifier across runs and file ...
Thamme Gowda's user avatar
18 votes
Accepted

Automatically input "yes" on the bash file

echo yes | your-program yes yes | your-program
Jeff Schaller's user avatar
  • 67.3k
12 votes

How does tcp-keepalive work in ssh?

The TCPKeepAlive option is actually a very different method of keeping connections alive from ClientAlive-like or ServerAlive-like options. Are per BSD SSH manual page, we can read that: The ...
kenorb's user avatar
  • 21k
11 votes

SLURM: Custom standard output name

You can't, at least not the way you want to do it. Those #SBATCH lines are shell comments which happen to be interpreted by the sbatch command, you can't execute shell code in them. Also, sbatch'...
cas's user avatar
  • 78.7k
10 votes

SLURM: Custom standard output name

The documentation for the sbatchcommand provides a comprehensive list of the allowed character replacements: at sbatch man page filename pattern sbatch allows for a filename pattern to contain ...
adjpayot's user avatar
  • 121
8 votes
Accepted

How do commands like fdisk -l find the sector size?

A device’s sector size isn’t stored in the MBR. User space commands such as fdisk use the BLKBSZGET and BLKSSZGET ioctls to retrieve the sector sizes from disks. Those ioctls are handled by drivers ...
Stephen Kitt's user avatar
7 votes
Accepted

Does the size of a file system cluster have to be even bytes?

Technically, it would be possible to implement a file system with clusters of any size; but ultimately, file systems are constructed on top of something else (typically storage, a network …). Disks in ...
Stephen Kitt's user avatar
5 votes

About mem and vmem

Just to remove this from the list of open questions and to give a simplified answer (goldilocks 's comment above and the qstat documentation assume deeper familiarity with systems): The answer depends ...
Joachim Wagner's user avatar
5 votes

SLURM: Custom standard output name

Here's a new answer that's a little interesting. You can't change the name of the log file during the process, and it is not ideal to change it afterward, because it involves wrapping the sbatch ...
joe's user avatar
  • 51
5 votes
Accepted

Simple cp command not working (inside shell script and submitted using sbatch on a cluster)

The ${d%?} construct doesn't work in POSIX shells like sh. Depending on your OS, your sh will either be a simple POSIX shell like dash (e.g. on Debian or Ubuntu) or perhaps bash running in POSIX mode. ...
terdon's user avatar
  • 242k
4 votes

How to ensure that I've entered the server and how to create a directory there?

it creates a directory in my home folder not in the server. Actually, assuming you entered mkdir directoryname on the command line after providing your password to the ssh -X [email protected] ...
thecarpy's user avatar
  • 3,945
4 votes
Accepted

How to ensure that I've entered the server and how to create a directory there?

The IP address you have provided to ssh is the IP address of your own client machine, not of the cluster you want to reach. The net result is that you're actually logging on to your own machine. ...
Chris Davies's user avatar
4 votes
Accepted

Memory Per Core

If you search for your question via Google like this - 'Compute Canada memory per core' you'll be directed to the glossary of terms for Compute Canada. On that page they define it like this: Memory ...
slm's user avatar
  • 370k
4 votes

Expanding only certain variables inside a heredoc

That's somewhat hard to do, since unlike with quotes, you can't just stop and restart expansion in an here-doc. But we could post-process the here-doc with sed: #!/bin/bash sourcedir=/some/path sed -...
ilkkachu's user avatar
  • 139k
4 votes
Accepted

Expanding only certain variables inside a heredoc

You can most straightforwardly do this for your usage just by breaking your heredoc in two parts: cat > run_array_job.sh<<'EOF' #!/bin/bash -l ... EOF printf 'BASEDIR="%s"\n" "$sourcedir" &...
Michael Homer's user avatar
3 votes
Accepted

Submitting list of jobs to slurm

grep -l 'pattern' script_folder/* | xargs -n 1 sbatch xargs will by default read as much as can fit on one command line before executing the given utility with all the things that it has read. With -...
Kusalananda's user avatar
  • 335k
3 votes

How to set the path of a batch environment to find the location of the "qsub" command

Okay! I have made progress! I tried submitting my job with qsub -V run.sh As I learned that this exports all the variables in qsub's command environment to the batch environment. This worked great, ...
KR_Henninger's user avatar
3 votes

NFS high availability

You need a shared storage device, but do you really need NFS? You could try things like OCFS2 (a shared-disk clustering file system, which expects that the same block device is opened on multiple ...
Wouter Verhelst's user avatar
3 votes
Accepted

pacemaker hangs while stopping

If your goal is to just stop the resource from running on any nodes in a cluster then you'd want to disable that resource using: pcs resource disable ClusterIP-01 Your command sudo pcs cluster stop -...
grephappiness's user avatar
3 votes

Monitoring a GPU cluster

You can use the ganglia monitoring software (free of charge, open source). It has number of user-contributed Gmond Python DSO metric modules, including a GPU Nvidia module (/ganglia/...
Franck Dernoncourt's user avatar
3 votes
Accepted

How to connect to the internet through another PC?

If @Richard Neumann's answer did not work, there is a possibility of it being a DNS issue. echo "nameserver 8.8.8.8" >> /etc/resolv.conf Set up IP Forwarding and Masquerading. iptables -P ...
Hunter.S.Thompson's user avatar
3 votes

Pacemaker: stopping pacemaker doesn't always move resources to other node

I cannot say why the first try had the resources stopped altogether without logs or more information. The second exercise behaved how I would have expected given the location constraints below. ...
Dok's user avatar
  • 396
3 votes
Accepted

Slurm on desktop computers, how to prioritize the owner

After a couple of days I managed to answer my own question. In hindsight it was quite simple. Responsiveness: The slurmd daemon can be started with command line arguments, list them with slurmd -h. ...
DavidAce's user avatar
  • 161
2 votes

Zookeeper DNS name problems with leader elections when migrating from Windows to Debian

Probably this issue caused by setting node hostname to 127.0.0.1 in /etc/hosts. In this case ZK will bind leader|election ports to 127.0.0.1 address. Config param quorumListenOnAllIPs=true should ...
Tom Lime's user avatar
  • 121
2 votes

Monitoring a GPU cluster

munin has at least one plugin for monitoring nvidia GPUs (which uses the nvidia-smi utility to gather its data). You could setup a munin server (perhaps on one of the GPU servers, or on the head node ...
cas's user avatar
  • 78.7k
2 votes
Accepted

MySQL DRBD Resource failing to start PaceMaker + Corosync

I don't see mysql defined in the Pacemaker; are you starting it manually after the cluster starts or does it start at boot? If it's starting at boot, it could be starting before Pacemaker, and ...
Matt Kereczman's user avatar
2 votes

Remotely execute commands but still have control of the host

distcc can be used to compile code on multiple machines. From the debian distcc package info: Package: distcc Description-en: simple distributed compiler client and server distcc is a ...
cas's user avatar
  • 78.7k
2 votes

How to monitor the Pacemaker cluster using a script?

Instead of trying to modify the Dummy RA to execute arbitrary scripts, you could instead look at using the anything resource-agent. # pcs resource describe ocf:heartbeat:anything ocf:heartbeat:...
Matt Kereczman's user avatar
2 votes
Accepted

distribute the JVM across a cluster of machines

Elaborating on my comment and assuming that you aren't thinking of something very esoteric... Perhaps you're looking at that diagram and interpreting it as if the users are using an application that ...
B Layer's user avatar
  • 5,171

Only top scored, non community-wiki answers of a minimum length are eligible