What’s the Big Deal with Big Data?

I recently debated big data with a worthy opponent in Marc Alley at the Corporate Research Conference.  He stood firm in his belief that big data is the best type of data whereas I stood firm in my position that traditional research is the only way to go.

I recently debated big data with a worthy opponent in Marc Alley at the Corporate Research Conference.  He stood firm in his belief that big data is the best type of data whereas I stood firm in my position that traditional research is the only way to go. You can read a summary of the debate written by Jeffrey Henning here.

The interesting thing is that, outside of the debate, Marc and I seemed to agree on most points. Neither of us think that big data is the be all and end all. Neither of us think that market research answers every problem. But both of us were determined to present our side as if it was the only side.  

In reality, the best type of data is ALL data. If you can access survey data and big data, you will be better off and have an improved understanding of thoughts, opinions, emotions, attitudes AND validated actions.   If you can also access eye tracking data or focus group data or behavioural data, you will be far better off and have data that can speak to reliability or validity.  Each data type will present you with a different view and a different perspective on reality. You might even see what looks like completely different results.  

Different is not wrong.  It’s not misleading. It’s not frustrating.  Different results are enlightening, and they are indeed valid.  Why do people do different than what they say? Why do people present contradictory data? That’s what is so fascinating about people. There is no one reality. People are complex and have many contradictory motivations.  No single dataset can describe the reality of people.  

There is no debate about whether big data has anything to offer.  Though Marc and I did our best to bring you to our dark side, we must remember that every dataset, regardless of the source, has fascinating insights ready for you to discover. Grab as much data as you can.