Big Data and Data Science: Is This Hype?

Big Data is one of those heavily abused technology terms today.  It’s become a catch-all phrase used to fit any data related problem into.  When the pundits defined the traits of Big Data and articulated the 3Vs, Volume, Velocity and Variety, little did we realize that the lack of clarity on whether it had to be all the 3Vs provided the perfect fertile grounds for term abuse.

Big Data is one of those heavily abused technology terms today.  It’s become a catch-all phrase used to fit any data related problem into.  When the pundits defined the traits of Big Data and articulated the 3Vs, Volume, Velocity and Variety, little did we realize that the lack of clarity on whether it had to be all the 3Vs provided the perfect fertile grounds for term abuse.  To add fuel to the fire, the lack of objective measures of what minimum Volume or Velocity would qualify as big data has led to the “beauty is in the eye of the beholder” syndrome, with everyone coming up with their own qualifying criteria.  In fact there is a broad tendency that when it comes to the first 2 Vs, Volume and Velocity, it’s almost anything that is more than what they are currently working with that has become the criteria for justifying something as Big Data. In my opinion this was intentional (the lack of clear qualifying criteria).

Over the years it’s clear that Information Technology, being one of the most “self-sustaining” entities that has ever manifested, has continuously founded and promoted something new every few years.  Each had a great window of opportunity with billions of dollars being spent and invested helping sustain the IT industry.  For any such phenomenon to have a golden run required, it must go thru a hype phase with some level of grounded reality until it has a critical mass of followers/adopters.  The golden run clock would start only if this resulted in exciting enough executives about the potential that it begins to influence their decision making.  It begins with getting executive support for allocation of budgets to experiment and explore leading to much bigger investments for larger scale deployments and projects.

Big Data has clearly entered its golden run.  This is definitely on the minds of many executives of companies all over the world.  Interestingly for many, the non tech-savvy folks do not even have a good understanding of what this is yet Big Data has garnered the status of a “competitive differentiator.” This is the pinnacle status for any phenomenon to achieve, and only a few such phenomenons have ever achieved this.  Big Data’s incredibly broad applicability in industries as diverse as one can imagine makes it a great candidate to achieve this status.

Big Data may have some hype associated with it, but the applications and value that is becoming apparent very quickly removes the veil of doubt and speculation.

The Big Data era and its golden flight run has already taken off.  Like other similar technologies there will be many roller coaster moments, but its going to be a great fun ride.  So, buckle up, sit forward (identify the top possible big data project candidates), don’t relax (begin working on one of these projects) and enjoy the ride (you should, and trust me, you will!).