▷ the Bright blog

This script requests that the job be run on three nodes with 2 tasks per node. STDOUT and STDERR will be written to the file "hello.out". Note that OpenMPI is tightly integrated with Slurm. Open MPI automatically obtains both the list of hosts and how many processes to start on each host from Slurm so we don't need to specify the --hostfile, --host, or -np options to mpirun.

$ cat hello.sh

#!/bin/sh
#SBATCH -o hello.out
#SBATCH --nodes=3
#SBATCH --ntasks-per-node=2
mpirun hello

Submit the job

$ sbatch hello.sh
Submitted batch job 35

Check job status

$ squeue
  JOBID PARTITION     NAME     USER  ST       TIME  NODES NODELIST(REASON)
     35      defq hello.sh  rstober   R       0:04      3 atom[01-03]

Get detailed job status

$ scontrol show job 35
JobId=35 Name=hello.sh
   UserId=rstober(1001) GroupId=rstober(1001)
   Priority=4294901758 Account=(null) QOS=normal
   JobState=COMPLETED Reason=None Dependency=(null)
   Requeue=1 Restarts=0 BatchFlag=1 ExitCode=0:0
   RunTime=00:00:02 TimeLimit=UNLIMITED TimeMin=N/A
   SubmitTime=2012-07-17T07:41:39 EligibleTime=2012-07-17T07:41:39
   StartTime=2012-07-17T07:41:39 EndTime=2012-07-17T07:41:41
   PreemptTime=None SuspendTime=None SecsPreSuspend=0
   Partition=defq AllocNode:Sid=atom-head1:24006
   ReqNodeList=(null) ExcNodeList=(null)
   NodeList=atom[01-03]
   BatchHost=atom01
   NumNodes=3 NumCPUs=10 CPUs/Task=1 ReqS:C:T=*:*:*
   MinCPUsNode=2 MinMemoryNode=0 MinTmpDiskNode=0
   Features=(null) Gres=(null) Reservation=(null)
   Shared=0 Contiguous=0 Licenses=(null) Network=(null)
   Command=/home/rstober/slurmhello.sh
   WorkDir=/home/rstober

The job has completed

$ squeue
  JOBID PARTITION     NAME     USER  ST       TIME  NODES NODELIST(REASON)

Here's the job output

$ more hello.out
Hello world from process 001 out of 006, processor name atom01
Hello world from process 002 out of 006, processor name atom02
Hello world from process 003 out of 006, processor name atom02
Hello world from process 000 out of 006, processor name atom01
Hello world from process 004 out of 006, processor name atom03
Hello world from process 005 out of 006, processor name atom03