The Humbling Effect of Big Data

December 10, 2012
55 Views

Many of us walk around with the knowledge that we generally understand how the world works. When people ask us for our professional opinions, we’re often more than happy to oblige. Some of us are even flattered when someone wants to know what we should do when faced with uncertainty.

Many of us walk around with the knowledge that we generally understand how the world works. When people ask us for our professional opinions, we’re often more than happy to oblige. Some of us are even flattered when someone wants to know what we should do when faced with uncertainty.

But do we know as much as we think we do? I have my doubts, especially after finishing Everything Is Obvious: How Common Sense Fails Us by Duncan J. Watts. It’s a thought-provoking book because it challenges readers to ask themselves, “What do we really know?” The answer, unfortunately, is not as much as we think.

Simple vs. Complex Systems

As Watts points out, part of the problem stems from the distinction between simple and complex systems. Let’s take a simple situation: the game of blackjack. I’ve been known to play a few hands, and only a fool would hit on 18 with the dealer showing a six. It’s just bad strategy. In such a simple environment, the best decision is clear.

The problem with simple systems is that they aren’t terribly representative. In complex systems like, say, the economy, myriad forces are at play, the vast majority of which are not under the control of any one person, group, department, or organization. Even the US government could not completely solve the financial crisis with a $900 billion stimulus. Bottom line: there’s only so much that any of us can do in a complex system.

And, as I turned the final pages on what is easily the best book I’ve read this year, I started to think about that big question in the context of Big Data. If we truly embrace Big Data, then we will have to question many long-held assumptions about how many things work: our jobs, our departments, our industries, and our environments. Looking at data with open eyes means that we may not like what we see, nor what that data will tell us. And that makes many of us uncomfortable. How many of us want to constantly question what we think we know?

Simon Says

To me, Big Data is not all about technology. Far from it. I’d argue that there’s a human element in all new technologies, and Big Data is no exception to this rule. There’s an organizational readiness component to it, as well as a personal one. Will people and organizations unaccustomed to consulting the data suddenly change their behavior? Will they be open-minded? Or will they act as if they know how things have worked, work now, and will work in the future.

It’s a big question. Big Data may prove that you don’t know as much as we think you do. What will you do then?

Feedback

What say you?

You may be interested

IEEE Big Data Conference 2017 to Highlight Challenges, Opportunities
Big Data
65 shares687 views
Big Data
65 shares687 views

IEEE Big Data Conference 2017 to Highlight Challenges, Opportunities

Ryan Kade - June 23, 2017

Since 2013, the Institute of Electrical and Electronics Engineers has held annual big data conferences to highlight changes and opportunities…

10 of the Top Marketing BI Software Options
Business Intelligence
117 shares1,053 views
Business Intelligence
117 shares1,053 views

10 of the Top Marketing BI Software Options

Hayden B. - June 23, 2017

Business can be complicated sometimes. It’s not always easy to keep track of all the data and information we deal…

The Race for 5G Is the Race for Data Dominance
Big Data
80 shares918 views
Big Data
80 shares918 views

The Race for 5G Is the Race for Data Dominance

Daniel Matthews - June 22, 2017

Have you noticed how often the phrase “by the year 2020” comes up? In the tech sphere, many are heralding…