Spinning up Hadoop with Bright


By Mark Sargeant | March 21, 2015 | Cluster Management, Hadoop, cluster health



In recent weeks, a growing number of clients have shown interest in Bright’s infrastructure management solution for Hadoop, so I thought it would be useful to record a video to demonstrate just how easy it is to spin up a brand new instance of Hadoop by using Bright. You can view the video here.

My colleague, Michele Lamarca, was driving the technology during the demo, and it took him only five minutes to install the Hadoop distribution—barely enough time for me to get warmed up!hadoop-elephant

Most of those reading this blog post already will be familiar with the many benefits that our HPC infrastructure management solution brings; deploying from bare metal; configure, provision, manage, monitor, and health check your clusters; and radically simplifying day-to-day administration from a single pane-of-glass presentation.hadoop-elephant

What customers were interested in seeing was how Bright logically extends to support Hadoop distributions, taking care of all of those critical elements just mentioned (configuration, provisioning, management, monitoring, health-checking etc.) and having Hadoop up and running in minutes. 

It’s important to remember that managing Hadoop clusters is no different from other types of clusters; without proper infrastructure, Hadoop will not run and the cluster will not be usable for data processing. Another point to keep in mind is that it doesn’t matter who your Hadoop vendor of choice is because Bright supports Apache, Cloudera, HortonWorks, and Pivotal from EMC.

Customers approach Bright at different stages of their Hadoop adoption lifecycle:

Some are in the Hadoop adoption phase and need help to get set up, removing the need of bringing costly Hadoop experts in house. Bright enables you to quickly build and deploy a Hadoop cluster to run that important project.

Some customers are unsure about taking the Hadoop plunge, and wish to build a POC cluster to test-drive Hadoop first. Bright helps with that, and even allows you to run multiple instances of different distributions at the same time during your evaluation. 

Some customers want to build a hybrid HPC / Hadoop cluster with Bright management from a single user interface.

Some have Hadoop in place, but are experiencing challenges, such as:

  • Monitoring issues
  • Decommissioning / re-commission nodes or node failure
  • Capacity issues  or jobs getting stuck
  • The need to improve the regularity and comprehensiveness of health checks
  • On an administrative level, the need to add/remove users
  • The need for application management or better reporting tools

So, please take a few minutes to watch to my video and to learn about the value we bring to Hadoop infrastructure management:
  • Bright addresses the physical cluster and Hadoop, installing from bare metal through to the Hadoop distro—on almost any Hadoop distro—giving you a single pane-of-glass management interface.
  • Bright is architected specifically for Hadoop, fully managing its services, with customized monitoring and health checks, and scaling to support multiple Hadoop instances.
  • Bright works brilliantly with HPC and Hadoop together, overarching simultaneous, independent instances on dedicated hardware, and time-sliced instances on shared hardware.

And please let us know if you have any feedback!