Apache Spark™ lives up to its tagline of ‘Lightning Fast Cluster Computing,’ according to Bright Computing’s Ian Lumb, who discussed this ‘hot’ cluster computing framework in our newly released vodcast. The vodcast, titled ‘The Rise in Popularity of Apache Spark,’ includes Ian’s take on the meteoric rise in popularity of Spark, comparisons between Spark and Hadoop, the importance of Spark’s excellent handling of converged analytics, and the exciting potential intersection of Spark and HPC. Take a listen to learn more about the unique solution that has ignited so much interest in the world of Big Data and beyond.Read More >
The 2015 Hadoop Summit is taking place this week, Wednesday and Thursday in Brussels, Belgium. Of course, Bright Computing will be at the Summit, highlighting our own solution for managing Hadoop clusters, as well as taking part in the conversations about the current state of Hadoop and what to expect for the future.Read More >
Spark'ing Scope Creep
I had the fortunate opportunity to present in the disruptive-technology track at the 2015 Rice Oil and Gas HPC Workshop during the first week of March. What I presented during my two-minute drill in this session ended up being much more disruptive than I anticipated.Read More >
I always wanted to be a seismologist.
Scratch that: I always wanted to be an astronaut. How could I help it? I grew up in suburban London (UK, not Ontario) watching James Burke cover the Apollo missions. (Guess I’m also revealing my age here!)
Although I never gave my childhood dream of becoming an astronaut more than a fleeting consideration, I did pursue a career in science.Read More >
The Summit features 6 tracks. By tapping the expertise of our Hadoop braintrust, we’ve submitted 4 ideas to 3 different tracks. Because we’d like to earn your votes, please allow us elaborate.Read More >
Imagine turning off half of your servers and still getting all of your computing done. Sounds impossible, right? But that’s exactly what one of our customers did. They cut the number of servers they needed in half, but all run at 90% utilization -- way better than the 30% utilization levels they were getting before. Their results are pretty impressive -- just think of the aggregated efficiencies in terms of power, cooling and space utilization they’re seeing.Read More >
The NVIDIA® Tesla® K80 Dual-GPU Accelerator was announced at 6 am PST today at SC14 in New Orleans. Bright Computing announced support for the K80 an hour later. How did we deliver 0-day support for the K80? Easy: People. Product. Process. We’ll briefly cover The 3Ps here, so you’ll understand why your K80s will be ready for production use upon arrival.Read More >
From the components that comprise a node (e.g., CPU cores plus accelerators and coprocessors, memory, disks), to the operating system and HPC software stack (e.g., client system for executing workload), device-specific metrics abound. As individual nodes communicate via interconnect fabrics (e.g., Ethernet, InfiniBand), metrics mushroom to reveal cluster-wide perspectives.
Everyone involved in running HPC clusters needs metrics. Hybrid-architecture, distributed systems, however, are a challenge to monitor, especially as systems scale out. Given the importance and inherent difficulties, it is not surprising that monitoring is once again topical in the HPC community. In fact, recent discussions are calling for an extreme makeover, by expressing the need to modernize monitoring for HPC clusters.Read More >
Last week we released support for Apache Hadoop 2.5.0. I thought it would be a good time to revisit all the updates we’ve made over the last 6 months - in case you missed them.
Our latest updates include:
- Apache Hadoop 2.5.1 Typo? Not exactly. We introduced support for Apache Hadoop 2.5.0 last week. Since Version 2.5.1 (released on September 13, 2014) is a minor release that builds upon the stable 2.4.1 release, you can now expect support via YUM updates to Bright. Note that vanilla Apache Hadoop includes HDFS (the Hadoop Distributed File System), ZooKeeper (the coordination service) as well as YARN (the workload manager).
- Cloudera CDH 5.1.0 We introduced support for CDH 5.1.0 in mid-July.
- Hortonworks HDP 2.1 We introduced support for Hortonworks Data Platform (HDP) 2.1 in mid-May.
Bottom line: Bright Cluster Manager maintains support for all major distributions of Apache Hadoop. Because we’re obsessed with keeping you current, you don’t have to be. We free you up to focus on the analysis of Big Data - and wasn’t that why you got interested in Hadoop in the first place?Read More >