Big Data Will Make IT the New Intel Inside

Tim O’Reilly famously declared in 2005: “Data is the next Intel Inside.” It well may be that big data—the organizational skill of using data as the key driver of performance—will make the IT function the new Intel Inside, the most strategic component of any organization.

Based on the success of Google, Amazon, and eBay at the time, O’Reilly correctly asserted that database management was a core competency of Web 2.0 companies and that “control over the database has led to market control and outsized financial returns.” The upcoming—and outsized—Facebook IPO is a testament to O’Reilly’s foresight, saying in 2005 that “data is the Intel Inside of [Web 2.0] applications, a sole source component in systems whose software infrastructure is largely open source or otherwise commodified.”

The linking of documents by web 1.0 and the linking of people by web 2.0 created what we now call big data. The explosion in the volume, variety, and velocity of data and the technologies invented to manage it occurred outside enterprises and, so far, had limited impact on enterprise IT.

But companies—and their IT departments—cannot ignore big data anymore. While Gartner predicted at the end of last year that “through 2015, more than 85 percent of Fortune 500 organizations will fail to effectively exploit big data for competitive advantage,” leading-edge organizations are moving fast to update their business intelligence, data mining, and business analytics practices with the new tools and skills set of big data.

Prime example is P&G, whose CEO Bob McDonald “has staked out a mission to ‘digitize’ the company’s processes from end to end,” and whose CIO, Filippo Passerini, said recently in an interview published in The Wall Street Journal: “I believe in our industry the differentiator in three to five years will be the ability to predict what is going to happen in the business. It is very possible to detect those key indicators early on, or roll out a predictive model that will allow us to be more accurate in the way we go to market with a new product. It is so critical to run the business in real-time. The ability to analyze big data—massive quantities of data—is important.”

As I see it, CIOs have three options right now:

Let the business units acquire and experiment with big data tools and hire the data scientists that can use them.
Start working with the business units and functions to identify their big data needs, help them hire data scientists, and experiment with the new technologies on a project-by-project basis.
Lead the business in its transformation to a big data-driven business, making IT its core competency.

The potential for business transformation is huge. McKinsey declared last year that “We are on the cusp of a tremendous wave of innovation, productivity, and growth, as well as new modes of competition and value capture—all driven by big data as consumers, companies, and economic sectors exploit its potential.” Research conducted by MIT’s Eric Brynjolfsson shows that companies that use “data driven decision making” are about 5% more productive and profitable than their competitors.

In a few years, we may see an even bigger impact of big data on business performance, acting as a key differentiator between winners and losers. The winners will make big data their own unique organizational skill, a “sole source component” (O’Reilly’s phrase) that determines whether a business can stay on top of its rapidly-changing environment and way ahead of its competitors.

IT can play a leading role in this transformation and become the next Intel Inside, this special organizational skill. It may be a unique moment in IT’s history, an opportunity to finally close the age-old “business-IT gap,” and lead the business in a once-in-a-century re-definition of the nature of business competition. The business of the future will be based on data and data is IT’s business.