Dazed and Confused About Big Data

Not every business needs to initiate “big data” analytics, but every company needs to understand more about big data – even if the final decision is that big data doesn’t yet have a role for the business objectives of a particular company. Big data sources can reveal vital information for every aspect of the business. But first, organizations have to gain a better understanding of what “big data” is and why it might matter for business insight.

Research firm Gartner has introduced 12 dimensions to clarify the many aspects of the confusing world of big data. Instead of “big data”, Gartner opts for Extreme Information Management, to reflect the diversity of data formats that can be considered “big data”. The various dimensions underscore the reality that “big data” isn’t just one thing and isn’t just ‘lots of data’. Approaching big data through different dimensions can help companies understand whether or not big data holds any value for their business needs, and where their most valuable big data may reside.

I like the term Extreme Information Management, where Extreme maps to the massive and highly disparate data stores and data classes that defy and overwhelm conventional data management techniques; Information points to intelligence and insight; and Management reminds us that hard work must be done to achieve real results. Big data analytics are only useful if they benefit desired business outcomes, open doors to new possibilities, alert businesses to risks and opportunities, and so on.

Big data accumulates from many sources – a scan through the different categories of data helps an organization identify data sources that matter to the business:

• Multi-structured or unstructured data with highly variable formats and semantics – examples are social media content, log files, and e-mail. Multi-structured data comes with many difficulties – information can be mined, but for it to have meaning and value, sentiment and context must be derived as well and correlated with master data for customers, products, and so on.
• The fastest growing area of big data is machine-generated data – examples are medical devices, industry sensors, automated machinery and systems, and GPS locations. Current and newly minted enterprises are very interested in this kind of data.

• Big data can be big too – datasets that are scaling from hundreds of terabytes up to petabytes (and more). Often organizations need this data processed very quickly to achieve analytics in real time or right time situations.

More understanding of what can be accomplished from big data analytics is revealed by real world business cases:

• Evolution of Smart Cities: Cities need to ensure the sustainability of resources and services, while making city services responsive to more people. Big data analytics interface with interconnected devices, technologies and digital systems to improve operations and efficiencies.

• Continuous Operational Optimization: People use operational data for planning, improving and expanding business capabilities. Automated systems utilize machine-generated data to self-optimize based on continuous automated analytics.

• Fighting Cyber Threats: Sophisticated cyber criminals don’t ever stop their attacks on organizations. Proactive security processes utilizing big data analytics must be put in place to continuously seek out active and potential threats. 

Eventually each organization must decide if big data analytics are a worthwhile – even game-changing – investment. Much thought and exploration should go into such a significant decision.