Big Data Accountability (Part 1)

Accountancy within organizations has been around practically since the existence of organizations itself. As long as we can recall, we have kept track of what happened within an organization, or in the old days while farming or herding. The earliest records of accounting data go back to 7,000 years ago, from the ruins of Babylon, where the growth of crops and herds were recorded. Since then, accountancy has evolved into an entire industry that helps organizations understand their resources and determine the correct value of their assets and liabilities.

With the new era of big data, a new era of accountancy is required: data accountancy or data accountability. One that is able of handling high volumes of a variety of data that has to be checked and controlled whether it is correct or not, with extensive and smart algorithms that perform incredible analyses within a fraction of a second and that provide predictions and visualizations that are used to define the course of an organization that will affect many stakeholders.

With so much data within an organization it is extremely important for organizations that the data they generate, store and analyze is 100% correct. With information-centric organizations that base their decisions on algorithms, it is fundamental that the algorithms and their (predictive) analyses are accurate. But who is qualified of checking and controlling thousands of Petabytes of data or extensive and really complex algorithms that improve over time? How do we ensure that consumer data is kept secure, private and is not abused? How do we ensure that the predictions made are based on the right variables? How do we know that green is really green and not perhaps red?

The era of big data will oblige a new form of auditing and control, of checks-and-balances and perhaps as well of quality labels for organizations. An ISO for big data? It could result in a completely new industry being developed next to and part of the global big data industry that is being formed in the coming years. Especially when organizations start to place big data on the balance sheet after they have determined the ROI, auditing or regulating organizations will pay very close attention to how the data is stored, collected, analyzed and visualized as it could make or break an organization.

There are three pillars belonging to big data accountability: the data itself, the algorithms and the auditors responsible for the checks and balances. In the next post, I will discuss these in more detail and give insight in how organizations have to deal with big data accountability.