Big Data, Good and Evil

As I get involved more and more in the world of Big Data, I find myself reflecting upon wh

As I get involved more and more in the world of Big Data, I find myself reflecting upon where it all will go.  Big Data could help us live better lives by solving crimes, predicting scientific outcomes, detecting  fraud and, of course, optimize our marketing so that we don’t bother people who don’t want our products and target them when we think they do. While the ‘goodness’ of some of those items are decidedly debatable, that’s the bright side. Big Data does represent a paradigm shift for our society, but since it’s still young, we’re just not sure exactly how big Big Data is yet.

When I write about Big Data, I’m talking about leveraging new sources of data like social media, transaction data, sensor data, networked devices and more.  These data sources tend to be… well, big. Mashing them up with your traditional CRM data or supply chain data can tell you some fascinating things.  They even tell you some interesting things all by themselves. It can give you information that wasn’t possible to attain, until recently, when we achieved the technology nd ability to handle Big Data in a meaningful way. We are already starting to see amazing case studies from Big Data.

On the other hand, there is potential folly. Despite the absolute evolutionary power that Big Data can bring to us, it’s also human nature for some to abuse.  When technological evolution brought us snail-mail, many abused it with junk mail.  When technology brought us e-mail, a few abused it by spamming us. Abuse is my biggest concern. The potential abuse with Big Data is that corporations completely figure out what makes us tick thereby giving them unprecedented power over our buying decisions. It could lead our social issues, too.  For example, if Big Data says that people who eat cheeseburgers after 9 PM are more likely to get a heart attack, do we justify outlawing cheeseburgers after 9?  I’d rather make my own decisions.

The movie “The minority report” starring Tom Cruise has come to mind.  As truth imitates fiction, I can help but think of the mall scene from the movie which overall painted a fairly grim picture of marketing in the future. Now, I see it as prophetic.

This type of marketing already exists within some free online e-mail systems.  For example, if I’m e-mailing my friends about a trip to Vegas or gambling, or even when I post this blog that mentions Vegas, it’s no mistake when ads for Caesars Palace appear.  It’s cool, but yet I am uneasy. Will future employers use big data to help decide if I am worthy of work. Will my e-mail conversations about Las Vegas lead them to believe I am a compulsive gambler thus giving the edge to someone else?  If so, what is my recourse to set the record straight?

Government has reportedly been getting in on big data, too.  A recent Wired magazine story talked about a huge government facility outside in Utah. While there is clearly a “good” aspect to this big data, namely the catching of bad guys, the most troubling aspect of this might be that the citizens have no control of their own data. Oversight on what can and cannot be done with the wealth of information at this facility is unclear.

That said, I generally have an overall positive view of the good that Big Data will bring to society, and the positive influence it will have on data management professionals. We have a society today that is more open and more willing to post private information to the public. Society is therefore more tolerant today and will be even more so in the future.

Ultimately, when and if Big Data becomes abusive to privacy, overzealous capitalism, social issues, et al, expect capitalism to also solve it. Look for companies who set up online e-mail and promote the fact that they don’t track conversations. Look for utilities to overwhelm any negative information about you in the Big Data universe with positive information. We could be looking at a cottage industry of managing  and protecting your Big Data image.

Covering the world of data integration, data governance, and data quality from the perspective of an industry insider.