By Robert Stober | July 30, 2012 | HPC Cluster, Linux Cluster, cluster testing, Linpack
It's easy to run Linpack across a Linux cluster with Bright Cluster Manager. This article shows how, using the Bright CMSH.
A. Run these commands as the root user
1. Set up your build environment
# cd /tmp
# module purge
# module add shared openmpi/gcc gotoblas
2. Build the RPM, this creates the patched sources and modified Makefile
# rpmbuild -bb /usr/src/redhat/SPECS/hpl.spec
B. Now log in (or su) as an ordinary user and perform the rest of the commands as this user.
1. Prepare the source
# su - rstober
$ cp -a /usr/src/redhat/BUILD/hpl-2.0 .
$ cd hpl-2.0
$ mv Make.Linux_OPTERON_GOTOBLAS64 Make.`arch`
2. Choose your environment
Note: In the near future you will want to install and use the Intel compiler as you will get a signifcantly better Linpack result.
$ module purge
$ module add openmpi/gcc
Change the ARCH and LINKER parameters (if necessary) to the values shown below
$ vi Make.x86_64
ARCH = x86_64
LINKER = mpicc
3. Use GotoBLAS
Note: gotoblas has optimization for both intel and amd cpu's make sure you choose the correct module as this can improve your results with 50%
$ module add gotoblas/penryn
Change the LAdir and LAlib parameters (if necessary) to the values shown below.
$ vi Make.x86_64
LAdir = $(BLASDIR)
LAlib = -L$(LAdir) -l$(BLASLIB)
4. Build the xhpl binary
$ TOPdir=`pwd` make arch=x86_64
Now copy the input file and binary to your running directory
$ cd ~
$ cp hpl-2.0/bin/x86_64/HPL.dat .
$ cp hpl-2.0/bin/x86_64/xhpl .
5. Run HPL across your cluster
Create a machines file
$ cat machines
atom01
atom02
atom03
Add the required modules to your environment. This ensures that the execution environment is correct in the remote shells that mpirun spawns.
$ module initadd openmpi/gcc
$ module initadd gotoblas
Run xhpl across the cluster
$ mpirun -np 12 -machinefile machines ./xhpl | tee HPL.out
C. Next Steps
1. Install the Intel compiler
2. Build xhpl using the Intel compiler
3. Tune the HPL.dat file to maximize performance