Bright Cloud Bursting

Bright Cluster Manager unleashes the unlimited power of the cloud, while managing the complexity for you. With Bright, you can easily create new clusters in the cloud, or add cloud-based resources on-the-fly to your existing clusters.

The same powerful cluster provisioning, monitoring, scheduling and management capabilities that Bright Cluster Manager provides to onsite clusters extend into the cloud, ensuring effective and efficient use of the virtual cloud resources.

Using Bright Cluster Manager, you can burst into public clouds, such as Amazon EC2, with only a few mouse clicks, without the need for expert knowledge of Linux or cloud computing.

Every cluster that runs Bright Cluster Manager is automatically cloud-enabled.

Benefits

Bright Cluster Manager's cloud bursting offers the following benefits:

1. Ease of Use: Intuitive GUI virtually eliminates user learning curve; no need to understand Linux or EC2 to manage system. Alternatively, cluster management shell provides powerful scripting capabilities to automate tasks.
2. Complete management solution: Installation/initialization, provisioning, monitoring, scheduling and management in one integrated environment.
3. Integrated workload managers: Wide selection of workload managers included and automatically configured with local, cloud and mixed queues.
4. Single system view; complete visibility and control: Cloud compute nodes managed as elements of the on-site cluster; visible from a single console with drill-downs and historic data.
5. Efficient data management via data-aware scheduling: Automatically ensures data is in position at start of computation; delivers results back when complete.
6. Secure, automatic gateway: Mirrored LDAP and DNS services over automatically-created VPN connects local and cloud-based nodes for secure communication.
7. Cost savings: More efficient use of cloud resources; support for spot instances, minimal user intervention.

Two cloud bursting scenarios

Bright Cluster Manager supports two different scenarios: "Cluster-on-Demand" and "Cluster Extension".

Cluster-on-Demand	Cluster Extension

Scenario 1: Cluster-on-Demand

The Cluster-on-Demand scenario is ideal if you do not have a cluster onsite, for example because you lack the space, power, cooling or budget. However, if you need compute power, with Bright Cluster Manager, you can instantly create a complete cluster in the public cloud, for any duration of time.

Bright Cluster Manager enables users to create a new cluster on the fly in the cloud with just a few mouse clicks. The Bright GUI provides complete visibility and control. The full capabilities of Bright Cluster Manager are available to this cluster: powerful provisioning, monitoring, workload management, GPU management, cluster health management and others.

Scenario 2: Cluster Extension

The Cluster Extension scenario is ideal if you have a cluster onsite but the cluster does not have enough compute power. With Bright Cluster Manager, you can instantly add compute power from a public cloud to your onsite cluster, for any duration of time.

For clusters already running Bright Cluster Manager, bursting into the cloud is as adding nodes to an onsite cluster. There are only a few additional steps after providing the public cloud account information to Bright Cluster Manager. The Bright approach to managing and monitoring a cluster in the cloud provides complete uniformity, as cloud nodes are managed and monitored the same way as local nodes, including GPUs.

• Same method of load-balanced provisioning
• Same software image management
• Same integrated workload management
• Same management interfaces — GUI, cluster management shell, user portal
• Same monitoring and health checking
• Same compilers, debuggers, MPI libraries, mathematical libraries and environment modules

The system administrator has the same visibility and control in the cloud as for the onsite system(s).

Amazon spot instance support

Bright Cluster Manager enables users to take advantage of the cost savings offered by Amazon's spot instances. Users can specify the use of spot instances, and Bright will automatically schedule as available, reducing the cost to compute without the need to monitor spot prices and manually schedule.

Built-In VPN, routing and network mapping

In the Cluster Extension scenario, Bright Cluster Manager provides a gateway between local and cloud nodes, and it also mirrors network services such as LDAP and DNS. It also enables everything to securely communicate with everything else, using built-in VPN, routing and network mapping. The VPN set-up is automated and in most cases does not require changes to the firewall configuration — only outgoing access to 1194/udp is needed. A single DNS namespace is used to find the fastest link between nodes, either locally or in the cloud.

Hardware Virtual Machine (HVM) virtualization

Bright Cluster Manager automatically initializes all Amazon instance types, including Cluster Compute and Cluster GPU instances that rely on HVM virtualization.

Data-aware scheduling

Bright Cluster Manager is the only cloud bursting solution to include data-aware scheduling. Data-aware scheduling ensures that input data is transferred to the cloud and made accessible just prior to the job starting, and that the output data is transferred back. There is no need to wait (and monitor) for the data to load prior to submitting jobs (delaying entry into the job queue), nor any risk of starting the job before the data transfer is complete (crashing the job). Users submit their jobs, and Bright's data-aware scheduling does the rest.

Workload Management

The workload manager can also be used for scheduling workload to the cloud resources. For example, the following types of job queues can be defined:

1. Local: jobs to be run only on the onsite cluster
2. Cloud: for jobs to be run only in the cloud
3. Mixed: jobs that may be run either locally, or in the cloud, depending upon workload manager scheduling rules.

The workload manager can also be used for scheduling workload to the cloud resources.

Bright Cluster Manager automatically installs and integrates a customer-selected workload manager. All common workload managers are available. The workload manager can also be used for scheduling workload to the cloud resources. For example, the following types of job queues can be defined:

• Local: for jobs to be run only on the onsite cluster.
• Cloud: for jobs to be run only in the cloud.
• Mixed: for jobs that may be run either locally, or in the cloud, depending upon workload manager scheduling rules.

All workload managers can be configured, viewed and queried through the Bright GUI or scripted through the cluster management shell.

The cloud is only a few mouse clicks away

Regardless of what scenario is chosen, Bright Cluster Manager's cloud wizard intuitively guides the user into the cloud via a few mouse clicks. Alternatively, the powerful Bright cluster management shell can be used to automate the process.

These screenshots show how easy it is to add sixteen cloud nodes to an existing cluster.

• Step 1: Enter the cloud account details into Bright Cluster Manager. In this example, information for EC2 accounts is being entered. For new clusters created in the cloud from laptop or workstations, the user accesses the Amazon EC2 site and is guided to create a Bright head node.
• Step 2: Add cloud nodes. Bright Cluster Manager enables the user to add any number of nodes at once. In this example, using the Bright wizard the range, node category, Amazon EC2 node type, storage type and storage size are all defined. Clicking .Next. creates all the cloud nodes within Bright, and sets them up for creation inside Amazon EC2. (screenshot)
• Step 3: Review and fine-tune as necessary.
• Step 4: Execute. The Tasks tab enables the user to execute most of the same tasks as the physical nodes, and automatically contact the cloud provider to switch on the nodes.

Bright Cluster Manager then automatically sets up nodes inside Amazon EC2 to accept the jobs for execution.

Bright Cluster Manager provides the choice: To "Cloud, or Not to Cloud"

Not all workloads are suitable for the cloud. The ideal situation for most organizations is to have the ability to choose between onsite clusters, for large data sets, high I/O or low latency requirements, sensitive data or advanced storage needs; and the cloud, for additional, low-cost capacity suitable for other types of jobs. Bright Cluster Manager delivers the best of both worlds: a powerful management solution for local clusters, with the ability to easily extend into the cloud without compromising provisioning, monitoring or managing the virtual resources.

Coming soon: support for other clouds

Future releases of Bright Cluster Manager support cloud bursting to other public clouds and private clouds.