Big Data and Business Intuition Work Together

Author: Cliff Cate, SVP of Customer Support at GoodData.

Author: Cliff Cate, SVP of Customer Support at GoodData.

Should you rely on your intuition? Or should you trust the facts and forget about your gut? In business, it’s an age-old debate. Steve Jobs was famous for ignoring marketing analysis, stating things like “People don’t know what they want until you show it to them.” On the other hand, Time Warner Cable uses data analysis throughout its organization, successfully deciding everything from how many routers to use to which ads to deliver.

Who’s right and who’s wrong? With terabytes of data at the fingertips of every company, the question of gut vs. data more relevant today than ever before.

Take the Middle Road

It turns out the best option may just be the happy medium. One Economist report found that executives who consider facts and data in their decision-making “dramatically increase their chances of success.” A different survey found that the best decisionmakers “balance judgment and analysis, possess strong analytic skills, and listen to others’ opinions but are willing to dissent.” The trick is to know how to combine intuition with data analysis, and to which degree.

Intuition works spontaneously, generating that gut-level sensation of ‘knowing’ something. Yet it isn’t necessarily a rational operation. Research has shown that if we don’t examine our instant assumptions, we fall into a range of bad assumptions. These include mistaking confidence for credibility and seeing only what we want to see, to ignoring a gorilla beating its chest in the background of a basketball game.

Intuition can be seen as a data model that comes from the brain’s store of learning and experience, including unstructured data from our five senses. As such, it is perhaps best emphasized for strategic decisions, where our knowledge of all the factors involved is incomplete. There’s guesswork involved (reasoning that takes place on our subconscious back-end) that machine data can’t completely address.

Still, that’s no reason to ignore data completely. “Don’t think of gut-based and fact-based decision making as competing concepts because they actually complement each other,” said management expert Dr. Chongqi Wu in this interview. He added: “Data just makes executives better gamblers.”

Ask, and the Data Will Answer

When is it better to rely more heavily on machine data than on human hunches?  Wu hinted that intuition doesn’t help much when it comes to correcting course, for example if an intuitive generated a poor result. Hard data enables decision makers to rationally understand the nature of their problem and correct course in a less reactive way. Wu also said that operational decisions, especially those that can be automated, are where data analysis shines.

Analyst Ray Wang writes that the way you interact with data analysis is crucial to making it work for you. Data analysis plays an important role in pinpointing inefficiencies in operations, as well as compliance, brand presence, revenue growth and market differentiation.  The trick is to interact with the data in the right way—by asking the right questions of it. Only then will the data give you a good answer.

After you get your answer, you can extrapolate more data to drive your decisions. What does the data tell you about your competitors, the health of your business? What’s the next smart action, if any, that you can take? What do you need to do to mitigate risk? Which use cases are the results most applicable to?

In this way, your brain is working with the data to infer reasonable conclusions. If, however, you assume that the data will give you a certain result and then try to arrange the technology to make it happen, you’ll end up confused and stressed out. Perhaps the worst-case scenario is that you’re both confused by your technology and trying to use intuition to instantly dissipate your problems.

What We Know for Sure

When used well, data gives you more confidence in the decision you’re making—even if it started on a hunch. Now that big data is freely available, businesses are working with it to make better decisions. It’s the management style of the 21st century, plain and simple.

How do you use data to inform your decision-making?