From Big Data to Smart Data: Supporting Critical Business Decisions

Last week at Lithium’s LiNC event I did a session on big data with Lithium Principal Scientist Dr Michael Wu. It’s always a pleasure working with Michael, the session was well attended and the audience was very engaged. Since then I have been thinking about what we presented and I think there were a few key points that are very important in understanding how businesses can start to make use of data to support business decisions.

First I should bring back out a system model that I’m using to represent the three basic systems that we need in the enterprise today; systems of transaction, systems of decision and systems of relationship. It’s important to understand the three systems, how they’re different and more importantly how they are interrelated.

Systems of transaction are the application foundation for the modern enterprise. They are designed to automate routine business processes and support business transactions. They are also the source for most of the structured enterprise data. Systems of relationship are the new social solutions that enable peer to peer collaboration, content sharing, and community. The systems are used to connect employees, customers, partners and other stakeholders together and to content, data and other enterprise systems.

The third system, decision, represents the evolution of analytics and business intelligence from specialty tools that could only be used by specially trained analysts, to a system that puts analytics into the workflow, in context and as an end user tool. This concept is very important, the move from a system used by a few to look at history, to a system that is integral to the decision process. Systems of decision move information to the “right” person, at the time of need and in a work context.

So how does big data play into this decision system concept? Big data is a hot topic and according to IDC last year alone the world created ~1.8 zettabytes of new data growing by a factor of 9 over the next 5 years. The other part of that story is that much of the data is unstructured, which creates its own set of issues. Now on the face of it, it does seem that with the mountain of new data growing at a staggering pace that businesses must need to “do” something with it. The problem though, is exactly what to do? Just because the data’s out there, and no doubt it’s big, doesn’t make it relevant, consumable or more importantly actionable. Data minus context can’t be turned into usable information. There’s also the problem with redundancy, a lot of data is redundant, and therefore isn’t important even if you can add context and relevance.

Data is useful and actionable only if it’s of interest to me (relevant) at the specific place and time (contextual) of use. The rest of the pile of “big data” is then, not of interest to me at all. The usefulness of big data is individual to a specific place / time, in the context of a specific issue or problem; or in other words, one person’s signal (actionable data) is noise to someone else. Some of the data then is useful and of interest but you must know what questions to ask, for whom, and when and where it will be of use to solve a specific problem or issue, otherwise, big data is just dumb data.

So the real challenge for systems of decision is to turn the dumb data into smart data, or maybe more accurately, information. Systems of decision have to provide relevant, useful, actionable, intuitive, digestible and interactive information to the right person at the right time. The next generation of analytics are systems of decision that can provide the relevant information to every system user, in work context, to make smart business decisions. Here’s a socialytics concept from Michael that fits this smart data / system of decision model: