Multiple Workload Management Systems - And Bright


By Martijn de Vries | June 17, 2020 |



Bright Cluster Manager has always integrated with workload management systems. And, system administrators control many aspects of their workload management system from Bright Cluster Manager’s management interface, Bright View. 

When building a new cluster, the workload management system is already up and running when the head node boots for the first time, because Bright packages the workload management system and configures it based on parameters that were provided by the administrator during installation.

The advantage of having this seamless integration between Bright and your workload manager is that when you add, remove or change nodes, the workload management system configuration will automatically change as well. For example, there is no need to add nodes to workload management queues as you add nodes to your cluster.

Also, Bright can run health checks on nodes right before jobs start (e.g. to verify that parallel filesystems are mounted, and that interconnect is up and running with the correct port speed), so that your jobs will always run on healthy nodes.

I’m pleased to say that in Bright Cluster Manager 9.0, it is possible to run multiple workload management system instances on the same cluster, whereas previously you could just have a single (e.g. Slurm) setup on a single cluster. If you wanted more in the past, you would have had to install separate Bright clusters

The main reason that we implemented this feature is to make it possible to create edge setups where each edge location has its own workload management system setup. Bright Edge is a feature of Bright Cluster Manager that allows organizations to deploy and centrally manage computing resources in distributed locations as a single clustered infrastructure, from a single interface. 

Another use case for having multiple workload management systems on a single cluster, is to partition the cluster for different classes of workload (e.g. classified and non-classified). If you want these workloads to be isolated as much as possible - while still running them on the same cluster - you could set up two (or more) instances of your favorite workload management system.

The development project to enable Bright Cluster Manager to support multiple workload management systems was no small undertaking, as the workload management system ties into so many things. However, we are delighted that this enhancement has been made, as we believe it will be a popular new feature for our customers.

For more information about Bright Cluster Manager 9.0 and workload management systems, please contact us.