While there are myriad reasons why a specific big data project fails, it’s the brave director that knows exactly when to pull the plug and examine what happened in terms of failure to learn for future endeavors. The challenge, however, is that some big data projects get more money pumped into them with the hope that someday they’ll drive financial value. Sadly, according to this analyst firm, for 60% of big data projects, that “eventually” never comes.
At some point, good money needs to stop chasing bad projects, but it takes leadership to make the right decision. The Financial Times’ Andrew Hill and Tim Harford agree: “Abandonment is a rare, difficult management skill. The natural instinct of most human beings is to persist. When the project is a collective commitment…it becomes even harder to hit the red “stop” button.”
When it comes to a failing big data project there are certainly good reasons “fold ‘em” and try a different approach. Common mistakes include:
- Objective – The overarching plan for a big data roll-out is too broad (boiling the ocean), instead of taking an approach that starts with prioritized use cases.
- Architecture – Don’t “rip and replace” haphazardly. When it comes to architecture, “leave and layer” is usually a better choice.
- Team – Too many big data initiatives end up solely sponsored by IT and fail to gain business buy-in.
- Experience – With millions of dollars potentially invested in a big data project, “learning on the job” won’t cut it.
Of course, no IT executive likes to fail at big data or any other technology project for that matter. But these warning signals are a harbinger of tougher times ahead:
Low Project Cash Flows
Let’s suppose after the first year you’ve been asked to by the CFO to revisit the initial business value calculations for your big data project. Imagine that upon examination of the numbers, your big data project was projected to bring in $2 million dollars of incremental net cash per year but now the project is trending at half that amount.
One choice is to wait it out and see if things get better. However, if break-even isn’t looking like it will happen until year three or four, it might be time to pull the plug on your big data project. Keep in mind, however, that if your project is a strategic one you may end up keeping the project going regardless of how it’s financially trending.
More Risk than Return
Every IT project carries risk. Open source projects, considering how fast the market changes (the rise of Apache Spark and the cooling off of MapReduce comes to mind), should invite even more scrutiny. Clearly, significant cost rises in terms of big data salaries, vendor contracts, procurement of hard to find skills and more could throw off your business value calculations. Consider a staged approach to big data as a potential panacea to reassess risk along the way and help prevent major financial disasters.
************************
One thing’s for sure, if you decide to pull the plug on a specific big data initiative, it’s important to take your licks and learn from the experience. By doing so, you will be that much smarter and better prepared the second time around. And because big data has the opportunity to provide so much value to your firm, there certainly will be another chance to get it right.
