3 Big Data Potholes to Avoid

Potholes—depending on the size—can really damage your automobile tires, rims and alignment. Even worse, sometimes drivers don’t see them lurking, until after an incident occurs. Similarly, a bumpy ride may be in store for companies building out their big data infrastructure, as there are often hidden and sometimes unexpected hazards ahead.

Potholes—depending on the size—can really damage your automobile tires, rims and alignment. Even worse, sometimes drivers don’t see them lurking, until after an incident occurs. Similarly, a bumpy ride may be in store for companies building out their big data infrastructure, as there are often hidden and sometimes unexpected hazards ahead.

Potholes, or road divots, are a problem in just about every American city. The problem is so acute in San Diego, CA that the city has paid out nearly $1m in damages to automobile owners in the past ten years. And in New York City, a research group noted only 20% of the roads are in adequate condition.

Without a display of orange caution cones to clearly mark a dangerous gap in the road, most drivers unknowingly drive over potholes and thus damage their automobiles. In the same way, without an understanding of three big data potholes listed below, big data projects are prone to either outright failure, or delays in business outcomes.

Big Data Failures Abound

Caution, caution, there’s a cemetery ahead. Gartner cites a disturbing statistic that should give everyone considering a big data project caution: Through 2017, “60% of big data projects will fail to go beyond piloting and experimentation and will be abandoned.” This estimation may not necessarily be a troubling statistic as it’s often better to fail fast and quit an IT project than keep feeding money to a bad idea.

However, this Gartner prediction also speaks to a potential lack of big data strategy supported with quality use cases. Circumvent this gap in the road by making sure your big data use cases are clearly outlined and sequenced. Without a solid plan to go beyond pilot or proof of concept, it might be hard to move ahead and justify additional monies.

There’s Little Forgiveness for Error

Here’s an ugly truth: IT really has just one opportunity to get ‘big data right’, or business users will go around IT and move straight to the cloud.

With the rise of cloud computing, companies like Amazon Web Services have made it easy for business users to “swipe” a credit card and gain access to compute power, storage and applications. Impatient business users no longer have to wait for IT and their dreaded waterfall project schedules to deliver new business capabilities. This means that the stakes are high for IT—either get the big data project right—or face a long bout of fighting shadow IT initiatives. Avoid this pothole by involving business users in a big data project from the get-go, and make sure they have immediate access to analytic sandboxes for theory testing and experimentation.

Big Data Adoption Isn’t a Given

If you build your big data system, chances are that business users won’t come. Why? Let’s be honest—people hate change. That’s why there are consulting practices solely dedicated to the theory and practice of change management.

Big data adoption isn’t a given. It’s possible to spend 6-12 months building out a big data system in the cloud or on premise, giving users their login and passcode/s, and then seeing close to zero usage. That’s because without a high level executive directing the program and mandating  change, people will mostly resist new technologies and processes. Avoid this pothole by gaining executive sponsorship from the start, investing in change management which includes training and a process for moving the organization through various transitional phases. Big data insights aren’t assured, especially if no one uses the new systems and applications.

There’s significant opportunity with big data, but there are few guarantees for success. And there are surely more than three big data potholes to avoid. What obstacles have you encountered? I’d love to hear your thoughts!