Big Data Takes a Big Step to Hadoop 2.0


By Bright Staff | February 18, 2014 | Hadoop, Hadoop Cluster Management, Hadoop 2.0



 nasa-climate-simulation-mapHadoop 2.0 is taking a giant step forward in its move to produce an open source project that is easier to use and more stable. A recent Tweet Report from the research firm IDC showed that many companies were cohabitating Hadoop with other solutions to enhance their data. The statistics showed that 32 percent already had deployed Hadoop, 31 percent intended to deploy Hadoop in the next 12 months, and 36 percent are saying that they are considering using a Hadoop deployment in more than a year.

This study highlights that some businesses are combining Hadoop 2.0 with such NoSQL databases as HBase, Cassandra, and MongoDB. Others are combining it with Greenplum and Vertica. While not as common, some are using it to work in conjunction with SQL technologies. The biggest benefit from the study is that it shows the number of different ways that businesses are using this latest version of Hadoop.

hadoop deployment

One of the biggest components that Hadoop 2.0 has is YARN (also called MapReduce 2.0). This upgrade to YARN has the advantage that management of the engine is excluded from the actual algorithm. This means that the user can use MapReduce as a plug-in instead of as an interactive process. This is a major milestone in the development of Hadoop, taking it from being a simple tool to a complete operating system for big data. This new organization, powered by YARN, enables the simultaneous execution of multiple applications on HDFS, the distributed file system, while providing better monitoring of data through the whole lifecycle.

The number of companies using Hadoop is growing. Yahoo, Goole, Amazon, and eBay have been pioneers, but now others are joining the Hadoop club using Apache Hadoop. For example, NASA is using Hadoop to cope with the large amount of data in their climate simulation projects, so one could say that Hadoop 2.0 is now helping us get a better look at our planet's future.

Do you have a Hadoop deployment in your future? Let us know if we can help. We specialize in cluster management software that can make your job easier.