Big Data Success Stories: Take Them with a Grain of Salt

March 8, 2013
41 Views

big data success storyFor every successful “Big Data”  case study listed in Harvard Business Review, Fortune or the like, there are thousands of many failures.

big data success storyFor every successful “Big Data”  case study listed in Harvard Business Review, Fortune or the like, there are thousands of many failures. It’s a problem of cherry-picking “success stories,” or assuming that most companies are harvesting extreme insights from Big Data Analytics projects, when in fact there is a figurative graveyard of big data failures that we never see.

“Big Data” is a hot topic. There are blogs, articles, analyst briefs and practitioner guides on how to do “Big Data Analytics” correctly. And case studies produced by academics and vendors alike seem to portray that everyone is having success with Big Data analytics (i.e. uncovering insights and making lots of money).

The truth is that some companies are having wild success reporting, analyzing, and predicting on terabytes and in some cases petabytes of Big Data. But for every eBay, Google, or Amazon or Razorfish there are thousands of companies stumbling, bumbling and fumbling through the process of Big Data analytics with little to show for it.

One recent story detailed a certain CIO who ordered his staff to acquire hundreds of servers with the most capacity available. He wanted to proclaim to the world – and on his resume – that his company built the largest Hadoop cluster on the planet.  Despite staff complaints of “Where’s the business case?” the procurement and installation proceeded as planned until the company could claim Hadoop “success.” And as suspected, within 24 months the CIO moved on to greener pastures, leaving the company with a mass of hardware, no business case, and certainly just a fraction of “Big Data” business value.

In an Edge.org article, author and trader Nassim Taleb highlights the problem of observation bias or cherry-picking success stories while ignoring the “graveyard” of failures. It’s easy to pick out the attributes of so-called “winners,” while ignoring that failures likely shared similar traits.

In terms of charting Big Data success, common wisdom says it’s necessary to have a business case, an executive sponsor, funding, the right people with the right skills and more. There are thousands of articles that speak to “How to win” in the marketplace with Big Data. And to be sure, these attributes and cases should be studied and not ignored.

But as Dr. Taleb says, “This (observation) bias makes us miscompute the odds and wrongly ascribe skills” when in fact in some cases chance played a major factor. And we must also realize that companies successfully gaining value from Big Data analytics may not have divulged all their secrets to the press and media just yet.

The purpose of this article isn’t to dissuade you from starting your “Big Data” analytics project. And it shouldn’t cause you to discount the good advice and cases available from experts like Tom Davenport, Bill Franks, Merv Adrian and others.

It’s simply counsel that for every James Simons—who makes billions of dollars finding signals in the noise—there are thousands of failed attempts to duplicate his success.

So read up “Big Data” success stories in HBR, McKinsey and the like, but be wary that these cases probably don’t map exactly to your particular circumstances. What worked for them, may not work for you.

Proceed with prudence and purpose (and tongue in cheek, pray for some divine guidance and/or luck) to avoid the cemetery of “Big Data” analytics projects that never delivered.

You may be interested

Big Data is the Key to the Future of Multi-Device Marketing
Big Data
0 shares122 views
Big Data
0 shares122 views

Big Data is the Key to the Future of Multi-Device Marketing

Ryan Kh - May 26, 2017

Digital marketers must reach customers across multiple devices. According to Criteo Mobile eCommerce Report, 40% of all online transactions involve…

Empowering Partners and Customers with Data Insights: A Win-Win for Everyone
Analytics
0 shares237 views
Analytics
0 shares237 views

Empowering Partners and Customers with Data Insights: A Win-Win for Everyone

Guy Greenberg - May 26, 2017

All businesses in the digital age rely on analytics for various activities: Product managers rely on analytics to gain insights…

The State of US Cyber Security
IT
0 shares277 views1
IT
0 shares277 views1

The State of US Cyber Security

bcornell - May 25, 2017

During the first week of May 2017 President Donald Trump signed a cyber security executive order focusing on upgrading government…