Bright Cloud Utilization
Bright Cluster Manager unleashes the unlimited power of the cloud, while managing the complexity for you.
With Bright, you can easily create new clusters in the cloud, or add cloud-based resources on-the-fly to your existing clusters.
The same powerful cluster provisioning,
monitoring, scheduling and management capabilities that Bright Cluster Manager
provides to onsite clusters extend into the cloud, ensuring effective and
efficient use of the virtual cloud resources.
Cluster Manager, you can extend into public clouds, such as Amazon EC2, with
only a few mouse clicks, without the need for expert knowledge of Linux or
Every cluster that runs Bright Cluster Manager is automatically cloud-enabled.
Bright Cluster Manager's cloud utilization offers the following benefits:
- Ease of Use: Intuitive GUI virtually eliminates user learning curve; no need to understand Linux or EC2 to manage systems. Alternatively, cluster management shell provides powerful scripting capabilities to automate tasks.
- Complete management solution: Installation/initialization, provisioning, monitoring, scheduling and management in one integrated environment.
- Integrated workload managers: Wide selection of workload managers included and automatically configured with local, cloud and mixed queues.
- Single pane of glass; complete visibility and control: Cloud compute nodes managed as elements of the onsite cluster; visible from a single console with drill-downs and historic data.
- Efficient data management via data-aware scheduling: Automatically ensures data is in position at start of computation; delivers results back when complete.
- Secure, automatic gateway: Mirrored LDAP and DNS services inside Amazon VPC, connecting local and cloud-based nodes for secure communication over VPN.
- Cost savings: More efficient use of cloud resources; support for spot instances, minimal user intervention.
Two cloud scenarios
Bright Cluster Manager supports two cloud utilization scenarios: "Cluster-on-Demand" and "Cluster Extension".
Scenario 1: Cluster-on-Demand
The Cluster-on-Demand scenario is ideal if you do not have a cluster onsite, for example because you lack the space, power, cooling or budget. However, if you need compute power, with Bright Cluster Manager, you can instantly create a complete cluster in the public cloud, for any duration of time.
Bright Cluster Manager enables users to create a new cluster on the fly in the cloud with just a few mouse clicks. The Bright GUI provides complete visibility and control. The full capabilities of Bright Cluster Manager are available to this cluster:
cluster health management
Scenario 2: Cluster Extension
The Cluster Extension scenario is ideal if you have a cluster onsite but the cluster does not have enough compute power, including GPUs. With Bright Cluster Manager, you can instantly add compute power from a public cloud to your onsite cluster, for any duration of time.
For clusters already running Bright Cluster Manager, utilizing the cloud is as easy as adding nodes to an onsite cluster. There are only a few additional, one-time steps after providing the public cloud account information to Bright Cluster Manager. The Bright approach to managing and monitoring a cluster in the cloud provides complete uniformity, as cloud nodes are managed and monitored the same way as local nodes, including GPUs.
- Same method of load-balanced, hierarchical provisioning
- Same software image management
- Same integrated workload management
- Same management interfaces — GUI, cluster management shell, user portal
- Same monitoring and health checking
- Same failover for high availability
- Same compilers, debuggers, MPI libraries, mathematical libraries and environment modules
The system administrator has the same visibility and control in the cloud as for the onsite system(s).
Bright Cluster Manager also provides additional features that are unique to the cloud.
Amazon spot instance support
Bright Cluster Manager enables users to take advantage of the cost savings offered by Amazon's spot instances. Users can specify the use of spot instances, and Bright will automatically schedule as available, reducing the cost to compute without the need to monitor spot prices and manually schedule.
Bright supports Amazon VPC setups which allows compute nodes in EC2 to be placed in an isolated network, thereby separating them from the outside world. It is even possible to route part of a local corporate IP network to a VPC subnet in EC2, so that local nodes and nodes in EC2 can communicate easily.
Hardware Virtual Machine (HVM) virtualization
Bright Cluster Manager automatically initializes all Amazon instance types, including Cluster Compute and Cluster GPU instances that rely on HVM virtualization.
Bright Cluster Manager is the only cloud utilization solution to include data-aware scheduling. Data-aware scheduling ensures that input data is transferred to the cloud and made accessible just prior to the job starting, and that the output data is transferred back. There is no need to wait (and monitor) for the data to load prior to submitting jobs (delaying entry into the job queue), nor any risk of starting the job before the data transfer is complete (crashing the job). Users submit their jobs, and Bright's data-aware scheduling does the rest.
The workload manager can also be used for scheduling workload to the cloud resources.
Bright Cluster Manager automatically installs and integrates a customer-selected workload manager. All common workload managers are available. The workload manager can also be used for scheduling workload to the cloud resources. For example, the following types of job queues can be defined:
- Local: for jobs to be run only on the onsite cluster.
- Cloud: for jobs to be run only in the cloud.
- Mixed: for jobs that may be run either locally, or in the cloud, depending upon workload manager scheduling rules.
All workload managers can be configured, viewed and queried through the Bright GUI or scripted through the cluster management shell.
The cloud is only a few mouse clicks away
Regardless of which scenario is chosen, Bright Cluster Manager's cloud wizard intuitively guides the user into the cloud via a few mouse clicks. Alternatively, the powerful Bright cluster management shell can be used to automate the process.
These screenshots show how easy it is to add sixteen cloud nodes to an existing cluster.
- Step 1: Enter the cloud account details into Bright Cluster Manager. In this example, information for EC2 accounts is being entered. For new clusters created in the cloud from laptops or workstations, the user accesses the Amazon EC2 site and is guided to create a Bright head node.
- Step 2: Add cloud nodes. Bright Cluster Manager enables the user to add any number of nodes at once. In this example, using the Bright wizard the range, node category, Amazon EC2 node type, storage type and storage size are all defined. Clicking .Next. creates all the cloud nodes within Bright, and sets them up for creation inside Amazon EC2. (screenshot)
- Step 3: Review and fine-tune as necessary.
- Step 4: Execute. The Tasks tab enables the user to execute most of the same tasks as the physical nodes, and automatically contact the cloud provider to switch on the nodes.
Bright Cluster Manager then automatically sets up nodes inside Amazon EC2 to accept the jobs for execution.
Bright Cluster Manager provides the choice: To "Cloud, or Not to Cloud"
Not all workloads are suitable for the cloud. The ideal situation for most organizations is to have the ability to choose between onsite clusters, for large data sets, high I/O or low latency requirements, sensitive data or advanced storage needs; and the cloud, for additional, low-cost capacity suitable for other types of jobs. Bright Cluster Manager delivers the best of both worlds: a powerful management solution for local clusters, with the ability to easily extend into the cloud without compromising provisioning, monitoring or managing the virtual resources.
Coming soon: support for other clouds
Future releases of Bright Cluster Manager will support utilization of other public clouds and private clouds.