Big Data and the End of Civilization as We Know It
As you may be aware, the world (or civilization, at least) is due to end in a couple of weeks, as the Mayan calendar counts to the last day of this "Sun". For those of you living beneath a stone, the date / time is 21st December at sunset in the Yucatan... depending on whom you choose to believe.
Big data, conversely, has been heralded by some as the harbinger of a bright, shiny and new world where all things will be will be possible using the vast quantities of data that are becoming available on the Internet. Many contend that the transformation has already begun. We will discuss the more mundane truth of how business is using big data in a joint EMA / 9sight webinar "Big Data Comes of Age" on Thursday, 13th December, 11am PST / 2pm EST / 7pm GMT.
The truth is somewhere in between... as always. And as year-end approaches, it might be a good time to ponder just where big data is leading us as people and as a society.
There's little doubt that big data--in all its meanings and incarnations--is effecting major changes in advertising and marketing. Much of what we see in this area is about increasing the efficiency of targeting and conversion. As Google tracks our searches, Facebook and Twitter our shared opinions, mobile Apps our movements and sellers our purchases, the message we hear is that businesses want to understand us and our needs more clearly, serve us better and ensure that we are increasingly delighted. However, the reality in the vast majority of cases is that businesses are driven simply by the financial bottom line, on a quarterly or even monthly basis as BI reports are produced and earnings statements released. Unfortunately, in my opinion, big data is most widely used as the next spin of the "sell more at higher profit" story, or to put it bluntly, driving consumption.
And yet, the other areas of application of big data offer insights about some of the biggest challenges to humanity, such as climate change, energy efficiency, health quality, economic and financial management, and more. The increasing quantities of data being gathered or available for collection and analysis in all of these areas offer us the opportunity to make a real difference in the lives of humanity, and to avert the catastrophes of which most scientists and philosophers already warn. That even one of the most data-driven of companies, PricewaterhouseCoopers, warns of impending global catastrophe due to the increased rate and scale of warming--as much as 6Â°C--is surely a sign that the writing is on the wall. To quote their report: "The only way to avoid the pessimistic scenarios will be radical transformations in the ways the global economy currently functions: rapid uptake of renewable energy, sharp falls in fossil fuel use or massive deployment of carbon capture and storage, removal of industrial emissions and halting deforestation... business-as-usual is not an option."
The PwC report does not, unfortunately, make the explicit link between ever increasing consumption of energy and raw materials on which the global economy currently functions and the seeming impossibility of reducing carbon emissions at the required rate to avert the worst possible scenarios. But big data analysis across both sides of this simple equation could show how to tackle the problem. Big data is about bringing data from widely disparate areas together and discovering new possibilities. How to consume less but improve living standards. How to prevent the type of financial behavior that paralyzes international economies. Of course, all this assumes the business and political will to do so.
Those of us who understand big data technology and promote its use must surely begin to advocate the more responsible and sustainable uses of this powerful technology. A New Year resolution, perhaps...?
Dr. Barry Devlin is a founder of the data warehousing industry and among the foremost authorities worldwide on business intelligence (BI) and beyond. He is a widely respected consultant, lecturer and author of “Data Warehouse—from Architecture to Implementation”. Barry has 30 years of experience in the IT industry, previously with IBM, as an architect, consultant, manager and software ...