Hadoop pushes, pulls Big Data analytics into mainstream (Part Two)

Did you miss out? It’s ok. We’ll fill you in. Part one of this two-part exploration of Hadoop’s affect on Big Data analytics outlined the that Hadoop, like Big Data itself, is about much more than just data volume – variety and velocity are equally as important. Oh, and that Yellowfin now connects to Hadoop.

Did you miss out? It’s ok. We’ll fill you in. Part one of this two-part exploration of Hadoop’s affect on Big Data analytics outlined the that Hadoop, like Big Data itself, is about much more than just data volume – variety and velocity are equally as important. Oh, and that Yellowfin now connects to Hadoop. For all the details, Go Here >

Now, on with the (big) show…

Key drivers: What’s pushing adoption and interest in Big Data solutions?

Despite Big Data rating highly on Gartner’s latest Hype Cycle – it’s not all puffery.

Big Data will continue to receive significant marketplace attention in 2012, as vendors and analysts scramble to meet and dissect emerging market demand. But, why has Big Data become a hot topic? The answer seems to be two-fold – and fairly simple.

New technology has made capturing, analyzing and understand more data possible
Firstly, new affordably priced products and services have emerged to give organizations the capability and capacity to capture more information than ever before.

TDWI’s aforementioned Big Data Analytics Best Practices study backed this view.

“User organizations have been collecting Web data for years,” noted report author and director of TDWI Research Philip Russom.

“For most organizations, it’s been a kind of hoarding. We’ve seen similar untapped big data collected and hoarded, such as RFID data from supply chain applications, text data from call center applications, semi-structured data from various business-to-business processes, and geospatial data in logistics.

“Far more users are now analyzing big data instead of merely hoarding it, thanks to new analytical database technologies such as Hadoop,” wrote Russom.

A recent TDWI article – 5 Macro Trends Shaping Next-generation BI and Analytics – provides an extremely relevant example of how such developments have changed corporate data collation approaches and mindsets: “suppose you run a manufacturing company that produces baby bottles. In 2001, you probably focused on just two data points: how many bottles your plants produced and how much money you made from that production. Today, however, you’re likely to track how many bottles you manufacture and how much money you make as well as how satisfied your customers are, what percentage are repeat customers, and what people are saying about your bottles on social networks.”

Rapid proliferation of new and existing data types and sources
This has been simultaneously met with the expansion of existing data types and sources, and the creation of new ones. The emergence, and continuing proliferation, of a multitude of notable social media platforms has created vast amounts of potential customer data. Data that is just waiting to be mined and explored! For example, Facebook now has over 800 million users whose details, preferences and interactions are available for analysis.

Gartner has put the propagation of available data into perspective, stating that the volume of data generated in 2009 alone was greater than in the preceding 5,000 years combined. Further, the IDC predicts that digital content volume will balloon by 2.7 zettabytes in 2012 – a 48 percent boost from 2011 – and drive the next wave of investments in business analytics.

“To cope with the challenges ‘Big Data’ poses, organizations must begin looking at deploying not only the applications traditionally used for Business Analytics, but also the supporting architecture in order to scale efficiently,” said IDC in a statement.

“This approach should also allow for the deployment of new technologies and frameworks such as Hadoop to assist with the analysis of large pools of disparate, unstructured data.”

According to Gartner’s Pattern-Based Strategy: Getting Value from Big Data report, worldwide information volume is growing at a minimum rate of 59 percent per year. Gartner expects global enterprise data assets to grow by around an additional 650 percent by the end of 2014.

Aberdeen’s two cents…
A recent benchmark report by Aberdeen Group – Future Integration Needs: Embracing Complex Data– revealed the biggest drivers for companies attempting to leverage value from Big Data (respondent organizations with data sets from 500 gigabytes to over 20 terabytes) as: