Big Intelligence: Measuring the ROI of Your Hadoop Big Data Project


ImageBig Data projects, like any other business investment, need to prove their worth. Most organizations define worth as ROI, which presents a problem for Big Data. 

ImageBig Data projects, like any other business investment, need to prove their worth. Most organizations define worth as ROI, which presents a problem for Big Data. 

Measuring the return on investment is a process developed by the manufacturing industry—one that quantifies tangible results, such as: How did our investment in R&D impact our bottom line? While no one in the manufacturing industry would say ROI is easy to calculate, the process becomes far more difficult when you are trying to measure a return on an investment that is expected to make people smarter, accurately predict the future, and drive better decision-making.

Additionally, many of the most significant benefits of Big Data are not immediately measurable. You want to look for sustainable uplift in business growth, customer acquisition, and brand value, among other things.


It is, however, possible to gauge ROI—albeit on a more limited scale—on the annual reporting cycle that most enterprises prefer. Look to other business use cases to determine where you can expect to see growth, and then develop your own metrics to gauge Big Data’s value to your company.

What the Overall Numbers Show

The Industrial Insights Report for 2015 found that 89% of those surveyed believe that Big Data analytics is an essential factor in remaining competitive and increasing market share, with 75% saying that growth is the key value of analytics. Another benefit, cited in the 2015 Internet Trends Report shows that Big Data projects also reduce operational expenditures: Computing costs dropped an average of 33% annually, cut the cost of storage by 38%, and slashed bandwidth costs by 27%. Impressive!

That said, companies are still struggling to extract actionable insights from Big Data. A recent report from IDG Connect found that 42% of those surveyed said that using data is their biggest challenge. IDG suggested that this difficulty is rooted in a “lack of investment in a defined data management process that includes ongoing, consistent data migration, data maintenance, quality control, and governance. Too often data is held and managed in multiple organizational silos. This results in inconsistency, duplication, gaps, and errors.”


Eighty-seven percent of organizations polled by IDG stated that the key goal for their Big Data projects is Better Decision-Making. The top three capabilities associated with “better decision-making” included Better Marketing Intelligence (19%), Better Data Analytics (16%), and Predictive Analytics Capability (13%). Your organization may have different goals, but it is helpful to know what other enterprises have determined are Big Data’s key growth generators.

Measuring Your Big Data ROI

Calculating ROI for most IT initiatives tends to be a straightforward task. Upgrading software, adding physical assets, or contracting managed services produces a measurable result. Big Data’s benefits are harder to quantify. Certainly you would expect Big Data to enable better decision-making, which should then drive revenue, but it can be difficult to quantify all of the value delivered by data analytics. The benefits of business intelligence should be pervasive, and while a company may know that data-driven decisions have made everyone smarter and more capable, that knowledge is difficult to measure.

You can still, however, measure some of the benefits that Big Data has brought to your business by selecting the right metrics for assessment. Doing this will require planning and development of supportive processes—but it will be far more effective than attempting to gather use case statistics just prior to the next board meeting.


While every business will have unique outcome anticipations for their Big Data investment, in general ROI expectations always center on growth. Thus, your ROI calculations to-do list should include:

1: Pinpointing Data Pain Points. Build consensus on the key priorities for analytics—the areas that will drive growth across the enterprise. Focus data initiatives on areas that deliver the most impact addressing these points. Avoid duplication of efforts across business divisions, as this will skew your measurements.

2: Identifying Short Term Growth Opportunities. Big Data’s business benefits should be expected to accrue significantly over the long-term, but CFOs and other stakeholders often want to see more immediate—or at least interim—results. Devise metrics that focus on the primary immediate drivers of data-related initiatives. A safe bet for most companies is to look at the sales and marketing departments. You may be able to look at customer relationship management and social media results as well.

3: Defining Your Metrics. This isn’t news, but it is such a critical aspect of measuring Big Data ROI that it is well worth repeating: Ensure that your metrics are tied to business goals. Your goals may be customer engagement, sales conversion, new business acquisition, market expansion, or something else. Determine how Big Data will drive those goals, and then measure the results on a team-by-team basis and across the organization as a whole. Ensure that team members understand how their KPIs fit into the overall business strategy.


4: Developing or Refining Your Data Strategy. Know what data business teams need to drive results. Prioritize availability of necessary data resources based on your business goal-driven metrics. As an example, if key teams need real-time access to semi-structured or unstructured data (digital media, website click-throughs, and social media), ensure that the Big Data ecosystem supports their needs. And encourage need-driven data exploration by providing self-service tools such as Apache Drill.


5: Determining How to Measure Key Goal Progress. How will you gather the resulting data that shows the growth of your metrics? You may need to have teams report on their KPI progress, or have regular meetings with team project leaders. You may be able to gather necessary data via internal and competitive analysis analytics. Develop a plan for collecting and working with metric data that suits your organization’s structure and your own preferred method of working with data.