Bright Cluster Manager 9.0 - Job Data, Jobs, and Project Managers

    

In recent weeks, I’ve been blogging about some of the cool new features in Bright Cluster Manager 9.0, to share an insight into the depth and breadth of the latest version of our technology.

In this blog post, I am going to take a look at the job monitoring features in Bright Cluster Manager 9.0 that give administrators an insight into the runtime behavior of compute jobs - for example to show GPU/CPU usage and memory consumption as a graph over the runtime of the job. 

Users can see this information for their own jobs, and it can be very helpful for users to see how their jobs are dealing with the resources that they allocate. For example, they may learn that GPU is only utilized during the first half of the job, in which case it would make sense to split their job into two interdependent jobs so that the GPU (i.e., the costly resource) can be freed up for other jobs.

Using Bright Cluster Manager accounting and reporting features, it is also possible to aggregate job monitoring data for a certain period of time and group it by user, project, or application. This would be useful in many situations; for example, to figure out which users are underusing or wasting the resources that they allocate for their jobs.

In Bright 8.2, administrators have access to all monitoring data, and users have access to just their own data. In Bright 9.0, we introduced the concept of “project managers” to allow managers to see monitoring data for jobs run by a group of users (i.e., the users that they are responsible for).

In Bright’s user management interface, it is now possible to declare a user to be a project manager for other users, but also projects/accounts. When submitting a job to the workload management system, a user can tag the job as belonging to a certain project (PBS Pro terminology) or account (Slurm/LSF/UGE terminology). 

A user can be configured to be a project manager for one or more projects/accounts, so that they can see job monitoring information for all jobs submitted by users with this tag. Conveniently, you can restrict it to certain users within that project

For clusters that are not using Bright’s built-in LDAP server for authentication (and therefore Bright’s built-in user management), it is possible to provide the information on who should be seeing job monitoring information for which users and/or projects, through a JSON structure.  

    [ { "alice": {"users": ["bob", "charlie"],

                  "accounts": ["projectx", "projecty"],

                  "operator" : "and" } } ]

For more information about Bright Cluster Manager 9.0, please click here