Not so long ago, it took ten years of data processing to sequence a single human genome; now the task can be accomplished in less than a week. What’s more, the cost of this feat has been divided by a factor of 10,000 since 2001, according to the National Human Genome Research Institute—100 times the rate predicted by Moore’s Law.
Data is easier to collect, simpler to store, more convenient to share, faster to transmit and less expensive to manage than at any time in human history. Gaining true insight from data, however, is still a work in progress. It’s one thing to review a set of data—even terabytes of data—but quite another to gain the deep understanding from it necessary to solve a particular problem.
Some would say that only human intelligence can transform data into insight. While it certainly takes intelligence to interpret data, such a statement begs the question as to how to make the connections necessary to reach actionable conclusions from these massive repositories of information. In other words, how do we turn Big Data into Smart Data?
Context, it turns out, is key. When we examine the surrounding environment and identify the types of data that influence a given situation, we can cross-analyze those sets to provide illumination.
Take, for example, quarterly sales data for a mid-sized clothing retailer. Using a sophisticated financial database tool we can measure the impact of promotions, new apparel, pricing changes and so on. But to truly understand the nature of revenue performance we must add in other factors that influence sales including weather, competitive activity, local employment changes, even recent news and events.
One famous example of the unexpected insights Smart Data can deliver was detailed in “The Power of Habit” by Charles Duhigg. Target, the mass retailer, wanted to identify which customers in its database could be predicted to be pregnant. Pregnancy, perhaps not surprisingly, is one of the most consequential life events for both new parents and for retailers, since it generates a massive amount of purchase activity — not to mention first-time brand engagement.
Looking at purchases of traditional baby items such as nursery furniture, baby clothes and diapers were all poor indicators, since those items might be bought for expectant friends or family members. It was only when Target’s statisticians brought in data from moms-to-be who had signed up for Target’s baby registry program, that other buying habits were identified.
Second trimester moms, for example, buy unusually large quantities of unscented lotion—and later, they buy vitamins, scent-free soap, cotton balls, hand sanitizer and washcloths in huge amounts. Combining this information produced context for someone who is expecting a baby. Soon, Target was able to use this information to effectively cater to millions of soon-to-be mothers.
Fortunately, analytics tools have evolved in lockstep with the rise in Big Data. New cloud-based solutions can mash together disparate data stores on an as-needed basis to create the “ah-ha” moment. Especially useful are online data from PayPal, Facebook, Twitter, Google Adwords, Google Analytics, Mailchimp and others. These sources can take businesses closer to their customers to gauge consumer sentiment, purchase indicators, conversion rates and so on.
In recent years, technology has advanced to the point where small and mid-sized businesses have the tools and information at their fingertips to level the playing field. For the first time, any business can connect multiple sources of data in their search for new, transformative conclusions. This new era in connected data is more than an opportunity, however; it’s a mandate. For any company to succeed it must see data as more than a series of silos—it must take the next step and approach data as a patchwork of rich, relevant and interconnected information. Only then can it evolve its data analysis into something truly actionable—and truly smart.
