Why Companies Are Not Engaging With Their Data

A look at a number of recent reports and surveys shows that while there is an appetite for using data more wisely, and an intention amongst top executives to use data more there are still  many companies not using data as effectively as they could.

According to Hewlett Packard

A look at a number of recent reports and surveys shows that while there is an appetite for using data more wisely, and an intention amongst top executives to use data more there are still  many companies not using data as effectively as they could.

According to Hewlett Packard

• 87% Of enterprises cannot fully exploit their data
• 85% Cannot Analyse data quickly enough
• 92% Do not capitalise on live data ¹

The appetite for data as well as the need for it are well understood so why are around 9 out 10 enterprises in such a poor state as HP would suggest?

A study by HBR finds that ‘Leading analytics users embrace a host of strategies, which evolve into best practices to create an “analytics ecosystem” in their organisations over time.’ and that ‘ In a rapidly changing global business environment, the pressure on organisations to make accurate and timely decisions has never been greater. The ability to identify challenges, spot opportunities, and adapt with agility is not just a competitive advantage but also a requirement for survival.’ it conclude this from ‘A global survey of 646 executives, managers, and professionals across all industries and geographies’ that ‘reveals a significant, albeit subtle, change in decision-making processes and their use of these analytics/BI tools’. ²

CALLOWNESS

A recent PWC report states that ‘Today’s companies have vast amounts of existing data on hand. And, as if this were not complicated enough, they continue adding to the pile by collecting extraordinary amounts of data.’ They go on to say that ‘good data visualisation can communicate the information that drives smart decisions.’ and that ‘In predictive analytics initiatives, companies place a large focus on the entirety of their internal data—and rightly so. However, it is also important to consider that incorporating relevant third-party data into analysis and the decision-making process can offer valuable supporting information that few companies leverage.’. The report notes that as far as predictive analytics is concerned businesses want to do it, but few know where to begin ³ . So despite the appetite for working with data what is preventing more companies starting to do so. 

Most companies are relatively immature or inexperienced in leveraging data, this is not due to any delinquency on their part but stems directly from the difficulties in working with data, data that is more often than not hard to drag out of the silos where it is kept and sometimes hidden. Maturity as it applies to data use within companies is a fairly well understood measure and can be determined by observing practices, processes and degrees of insights. Companies inexperienced in using data often express many reasons for not doing so ranging from a higher degree of trust in intuition and a distrust of the data that they have available. The figures they have do not show the full story is oft quoted (particularly if they are bad). The hint of anything statistical rouses deep suspicion as well, the “the well known fact that that statistics lie” is couple with the “it does not show the reasons behind the data”. Evidence that successful companies have taken a different view is fairly widespread but a big lag exists between the fortunates and those on lower spokes of the wheel.

In their report ‘The Evolution of Decision Making: How Leading Organizations Are Adopting a Data-Driven Culture’ HBR find that “Typically analytics first take hold in a siloed manner, where they are not integrated into company-wide decision making. They are usually a response to a specific challenge in a high-profile department, such as finance or marketing. The siloed and focused nature of the implementation often means that workers do not develop a deep grasp of the power of data-driven decisions and so do not develop the necessary skill set to appreciate or use analytics”. ⁴

It is not too surprising when the other two C’s are brought into the equation, that companies are reluctant to engage full with their data so it is perhaps unfair to lay too much blame on the callowness of smaller enterprises.

COST

Working with data can be a very expensive activity, traditionally it has required expensive expertise, costly infrastructures. A very big budget needs to be set aside to accommodate and purchase the resources and the technology requires to deliver it to the decision makers within a company. Cost always strikes fear into those who have most to fear and finance is traditionally one domain that is reluctant to have its view questioned. Technology is another department that traditionally loathes embracing changes as for them the main cost is acquiring skills and disrupting a platform that may only recently have become stable. Aside from blaming the prudence of the finance department and the luddites in IT alone it fair to say that, until recently, business had a very good case for not embracing ways of using data more strategically. Implementing a system to manage and handle data has traditionally had a very high price tag, more often than not multi millions £. Even at the entry level the cost for data services are substantial. The learning curves required from existing resources can also be very steep and high, hiring new expertise does not come cheep that is for sure.

A survey carried out by Deloitte found that “the areas in which big data and analytics were found to be the most important were those directly related to income production or cost control/reduction. ‘ and that ‘ Financial operations have long been data-driven, but the availability of big data and the growth of data analytics capabilities have further heightened its importance. These are no doubt the reasons that the area most often found to invest in analytics, at 79 percent, is finance.’ they go on to conclude that ‘It stands to reason that if finance is willing to invest in analytics, there is ROI to be had.’ ⁵ 

COMPLEXITY

The data that is most useful for a company is often generated from different sources, it is often structured differently (if it is structured at all) it invariable resides in different places and is, lets face it, a bit of a mess.

As a recent report fromErnst & Young points out ‘Most organisations have complex and fragmented architecture landscapes that make the cohesive collation and dissemination of data difficult. this involves

• Data from multiple source systems is cleansed, normalised and collated
• External feeds can be gathered from the latest research, best practice guidelines,
• Benchmarks and other online repositories
• Use of enhanced visualisation techniques, benchmarking indexes and dashboards can inform management and consumers via smartphones, laptops, tablets, etc., in-house or remotely’ ⁶

Working with data therefore becomes a complex business, it need to be brought together, a way of dealing with different structures needs to be implemented and it needs to be cleaned up.

Market analyst firm IDC tells clients ‘they should be able to double the productivity of their business analysts by providing them with the right tools and decision-making culture. “There’s a genuine movement toward decision making based on data rather than heuristics today,” says Bob Parker, a group vice president at IDC.

Among the big challenges is the growing number of data sources analysts must gain access to. For another, many of the analytical tools on which analysts rely are incapable of processing, or struggle to process. many of the data sources. These tools, including Microsoft’s Excel application, have analysts manually poring over spreadsheets and creating complex formulas from scratch each time out.’  ⁷ 

The capture and sort process can be so time consuming because blending unstructured and semi structured data and then organising is and ordering it require a lot of manual effort or a traditionally a complex series automated processes. Most data analysts spend most of their time cleansing data. “The dirty little secret of big data,” says Tom Davenport, a professor of IT and management at Babson College and the author or co-author of 17 books on those topics, “is that most data analysts spend the vast majority of their time cleaning and integrating data— not actually analysing it.” In his other role as a senior advisor to Deloitte Analytics, Davenport sees many data analysts today manually poring over Excel spreadsheets all day, then generating the most basic cut-and-paste reports to the departments that need business insight. “Some companies with a global reach can have hundreds of data analysts,” he says, “which equates to a huge waste of time.” ⁸

Now there is a new breed of cloud based application that takes the three C’s out of the equation, making data analytics; an easy process to engage with, even for the data immature business; it has low financial and resource overheads and a simple interface removing complexity.

Souces

1. Hewlett Packard: Big Data Solutions
2. Harvard Business Review: The evolution of Decision Making
3. PWC: Putting predictive analytics to work
4. Harvard Business Review: The evolution of Decision Making
5. Deloitte: The analytics advantage
6. Ernst and Young: Big data
7. Harvard Business Review: Alteryx Report August 2015
8. ibid