Why We Must Declare War on Data Obesity

We probably all know that obesity is becoming a big (pardon the pun) problem in the developed world and just becoming bigger. While we obviously need food to survive, Westerners in particular have access to so much food now that it has become unhealthy—and we are dying early as a result. It’s a mindset that more is always better — more food, more choices — but that’s not always the case. This is a similar phenomenon to the one I see with businesses and governments that are amassing data (big data) without thought to what they’re actually collecting or why — they are becoming data obese. The cost of data obesity Food is a necessity to life and I think an argument can be made that data is a necessity to a modern business; we need data in business to gain insights and make better decisions. But just as we must choose the right foods in the right amounts to keep us healthy, businesses must be judicious about what data they collect. We are collecting so much in many cases that it is becoming unhealthy and costly. First is the monetary cost. For most companies, the vast quantities of information they gather is stored in warehoused data solutions that cost $10s–$100s of millions to operate and maintain. Even with advances in storage technology, which could reduce the costs to closer to $1 million per petabyte of storage, the costs are still extraordinarily high. And the more you store, the more you need to spend to store it. Second, these big data centers use valuable natural resources in the large amounts of energy used up by these server farms. The servers themselves suck energy, but they must also be kept cool to function properly, requiring vast air conditioned warehouses that burn vast quantities of fuel. The environmental cost of storing more and more data should not be underestimated. And finally, having more and more data only makes it harder to find the valuable bits of information. The data points available to collect are practically infinite — as unique as the individual customers you serve. But how much of that data is (or ever will be) actually useful to your bottom line? Going on a data diet. I completely disagree with the notion that you can never have too much data, as some big data pundits claim. Similarly to our food intake, where we have to select the good, nutritious things that will give a balanced and healthy diet, in business we have to focus on collecting the data sets that will help us solve our most important problems and help us answer our most important unanswered questions. In the same way we see obesity spreading, I see more and more organisations that hoard data. Also important to remember is that data goes out of date. Simply hoarding it is not clever. With big data, more is not always better. Most organisations could probably do with going on a data diet and understanding that they need less data overall, but more specific data that helps them solve their most important problems. We don’t need more data, we need the right data.