Quickly Deploy a Well-Engineered Apache Hadoop Solution to a Production Environment

August 7, 2011
237 Views

As a former enterprise CTO and current technology watcher, I was struck at the incredible brilliance of yesterday’s announcement by Dell

As a former enterprise CTO and current technology watcher, I was struck at the incredible brilliance of yesterday’s announcement by Dell and Cloudera. In a move that will help enterprises of all sizes serve a very wide range of missions, those two organizations have announced a new relationship that will enable something that has never really existed before. Enterprises can now buy a Hadoop-centric solution that has been validated from end-to-end and is available from a single vendor. And they can do this on systems that have hardware designed for performing Big Data analytics.

Since their formation, Cloudera has been working to bring clarity and unity and support to the entire stack of software around Apache Hadoop. A key benefit of the Cloudera Distribution Including Apache Hadoop (CDH) is that it makes enterprise support in production environments possible. Every enterprise CIO wants supported software and CDH is the way to do that with the Cloudera supported Hadoop stack. You can think of this next move as taking that approach one huge step further. Now enterprises will not only get a fully supported software stack of all relevant Big Data analytics tools and management tools, but they will also get that in a specially designed hardware system. And the same vendor will provide training, technology support, professional services and other support. In every sense of the word this system has been validated end to end.

Benefits for organizations and their missions include:

  • Much lower risk
  • Faster deployment
  • Increased efficiency/Higher performance
  • Total supportability
  • Much more agility and ability to support the organization’s mission.
For more see the press release below:

Dell and Cloudera Collaborate to Enable Large Scale Data Analysis and Modeling through Open Source Solution

Enterprises looking to process massive application datasets can now utilize the industry’s first complete Apache Hadoop solution

ROUND ROCK, Texas and PALO ALTO, Calif. – August 4, 2011 –Enterprises looking to process massive application datasets can now utilize the industry’s first complete Apache Hadoop solution, resulting from a relationship between Dell and Cloudera Inc. The Dell | Cloudera solution for Apache Hadoop combines Dell servers and networking components with Cloudera’s Distribution Including Apache Hadoop (CDH), as well as management tools, training, technology support and professional services, to give customers a single source to deploy, manage, and scale a comprehensive Apache Hadoop-based stack.

Aimed at financial services, energy, utility and telecom companies, research institutions, retail businesses, and Internet/media groups, The Dell | Cloudera solution for Apache Hadoop can reduce the complexity of deploying, configuring, and managing Hadoop systems. By combining the leading open source Apache Hadoop-based software platform and purpose-built hardware, the integrated Dell | Cloudera solution for Hadoop enables organizations to analyze large amounts of complex, dynamic data at a lower total cost of ownership (TCO) than proprietary solutions.

The Dell PowerEdge C line is designed and optimized to work in large scale-out computing environments which highlight the company’s approach to producing best of breed solutions. This offering brings simplicity and power helping enable more organizations to process big data sets in order to spot business trends and buying patterns, conduct risk modeling, direct customer profiling and retention programs, determine buying patterns or simulate physical, biological, and environmental processes. Enabling the analysis of large amounts of data gives customers the power to derive greater and faster insights that lead to a tangible business advantage.

“Many companies today are recognizing the inherent value in analyzing the vast amount of structured and unstructured data currently being stored across their organization. Apache Hadoop enables companies to perform complex analyses, including detailed special-purpose computation across large collections of mixed data. However, deployment and management complexity, as well as requirements for highly specialized knowledge, have been barriers to adoption,” said Merv Adrian, Research VP, Information Management at Gartner. “Providing complete end-to-end solutions for Apache Hadoop deployments will be key to significantly speeding its adoption within mainstream enterprise environments.”

The Dell | Cloudera solution for Apache Hadoop consists of CDH, Dell Crowbar software, and Cloudera Enterprise combined with a Dell PowerEdge C2100 server (other models to be added shortly) and PowerConnect 6248 48-port Gigabit Ethernet Layer 3 switch. Joint service and support and a deployment guide are also included. Available for purchase directly from Dell within the next 30 days, The Dell | Cloudera solution for Apache Hadoop is designed to offer customers:

  • Streamlined Deployment: Cloudera Enterprise and Dell Crowbar enable enterprises to manage the complete operational lifecycle of their Apache Hadoop systems. The combination simplifies the deployment and management of Apache Hadoop services, including the ability to install and configure Apache Hadoop in minutes from a central dashboard and continue to make configuration updates while the system is running.
  • Increased Efficiency: Designed with inspiration from Dell’s Data Center Solutions (DCS) business, the PowerEdge C series is a high performance data analytics, cloud compute platform and cloud storage server line-up. Feature- and power-optimized to provide lower total cost of ownership in addition to saving on space and energy, it is an ideal platform for high performance computing (HPC), Web 2.0, gaming, social networking, Software-as-a-Service (SaaS) and public and private cloud infrastructures.
  • Low-Risk: Built on CDH, the leading distribution of Apache Hadoop in commercial and non-commercial environments, and Dell’s performance-based PowerEdge C hardware, The Dell | Cloudera solution for Apache Hadoop is tested, validated, and supported by Dell, one of the industry’s most experienced providers to hyperscale computing environments.
  • Complete Support: Led by Dell, customers will have access to comprehensive and collaborative service and support for the entire solution, from installation, configuration, and deployment to optimization and tuning with existing IT infrastructures.

This solution also leverages Dell’s popular Crowbar software, which Dell has released to the community as open source. Crowbar manages the Apache Hadoop deployment from the initial server boot to the configuration of the primary Apache Hadoop components allowing users to complete bare metal deployment of multi-node Hadoop environments in a matter of hours, as opposed to days. Once the initial deployment is complete, Crowbar can be used to maintain, expand, and architect a complete data analytics solution, including BIOS configuration, network discovery, status monitoring, performance data gathering, and alerting. Crowbar provides the necessary tools and automation to manage the complete lifecycle of Hadoop environments.

Dell is committed to providing our customers with the best solutions for expanding IT capabilities without creating additional management requirements. Our PowerEdge C servers are purpose-built for performing Big Data analytics. Through our relationship with Cloudera, companies can now take advantage of the leading distribution of Apache Hadoop along with Dell servers and networking capabilities to easily and quickly deploy an end-to-end solution for Hadoop,” said John Igoe, executive director of Cloud Solutions at Dell. “Organizations can begin solving business challenges by analyzing structured and unstructured data while knowing that they have the backing of Dell service and support and access to Cloudera training and management tools, all via a single point of contact.”

Our relationship with Dell is a natural fit as we continue to expand the footprint of Apache Hadoop across mainstream commercial environments. Enterprises can now quickly and easily deploy an entire solution for Apache Hadoop that is optimized for production level environments,” said Ed Albanese, head of business development at Cloudera. “Cloudera customers have spoken clearly. They want a Hadoop configuration that has been validated from end–to-end and is available from a single vendor. We are excited that Dell is investing to meet this demand.”