Why Are Mid-Market Companies Waiting to Embrace Big Data?

Though many companies endorse the potential benefits of Big Data, very few are actually giving this development some serious thought. Particularly, many mid-market companies view this trend with much skepticism and closely seek confirmation from their risk affine industry peers. Among the numerous factors that hamper Big Data adoption in this sector, the most dominant is the Penguin Effect. In this post, I shall address some of the underpinnings of this effect and benefits that mid-market companies could aspire to accrue from being the first to adopt this technology.

Though many companies endorse the potential benefits of Big Data, very few are actually giving this development some serious thought. Particularly, many mid-market companies view this trend with much skepticism and closely seek confirmation from their risk affine industry peers. Among the numerous factors that hamper Big Data adoption in this sector, the most dominant is the Penguin Effect. In this post, I shall address some of the underpinnings of this effect and benefits that mid-market companies could aspire to accrue from being the first to adopt this technology.

The sunk cost fallacy

Most companies, including the mid-market companies, have already invested in particular technologies and are often unwilling to deploy or even test a superior new technology, thus becoming victims to the sunk cost fallacy. They are more willing to throw more good money after bad in extending their current technology or in customizing functionalities to deliver the intended business benefit. After all, what has worked so well for so long should work well in the future, isn’t it?

However, I argue that the reality is much different.

Software solutions that addresses customers’ needs pertain to problems they face “today”, whilst the “ideal solution” that requires conception and development will, however, only be delivered “tomorrow”. Given today’s dynamic business environments, the solution that meets customer expectations is already obsolete by the time it’s ready for deployment. As a result, software maintenance projects are perennial activities that deliver little IT or business value, forcing both businesses and IT service providers to continually play the “catch up” game.

Hype or reality?

Even though many mid-market companies realise and appreciate the potential benefits from Big Data technology adoption, they are reticent in introducing an unproven technology into their current landscape, not to mention the complete process overhaul that’s required in shifting to a data-driven mindset. Many managers in such companies consider decisions based only on insights derived from data inferior to those derived from experience accumulated over a course of several decades and the entire enterprise juvenile. Though one may argue that a decision forged from a combination of data-derived insights and experience is superior to either on its own, such a notion is either unconceivable or foreign to many senior managers in mid-market companies. These managers are not necessarily in the wrong.

Often times, such disruptive technologies as Big Data are accompanied by much media buzz and fanfare that it gets immensely difficult to distinguish claims about a real technological benefit from hype. Despite several use cases and success stories widely available all over the Internet, the situation with Big Data is no different. Many organizations have assiduously studied the hype cycles from Gartner and many CIOs would rather wait for the glorified Plateau of Productivity, where the technology has proven itself, many vendors and alternatives are available, benefits are predictable and the technology is “safe” to be deployed in the internal ecosystem.

Hence, many of these companies are still playing the waiting game, hoping that their competitor or industry peer makes the first move and embraces Big Data. The behavior is what social psychologists call, The Penguin Effect. For these mid-market organisations, many first-mover advantages result from adopting or testing Big Data solutions, which proffers several competitive advantages much sooner than their competitors and industrial counterparts. These vanguard companies may further benefit from exclusivity in using the new technology for a specific time period, competitive pricing, increased media coverage due to prototypicality effect etc.

Conclusion

Getting the mid-market companies to embrace Big Data requires a concerted approach to value confirmation. If the past is any reasonable predictor of the future, mid-market companies will either fall under the late majority or laggard region of the technology adoption lifecycle without a compelling choice architecture. A comprehensive business case that clearly demonstrates value creation, co-existence with the current IT landscape and business continuity will induce such companies to consider pilot projects that generate outcomes, which demonstrate that the advertised benefits of Big Data are indeed, attainable. Such small wins are indispensable in catalysing the transition of Big Data technologies into mainstream operations as strategic assets. Companies that build data-centric capabilities are well poised to win in the long run. The managers in mid-market companies understand this. They also understand that they should demonstrate value creation in the short-term for their very existence. Winning these stakeholders to advocate Big Data technology adoption is a delicate act of balancing short- and long-term value creation strategies. For Big Data technology companies, Delivery excellence, in addition to technological excellence, are key to showcasing sustained value creation that mid-markets companies value and to garnering managerial support in such organisations.