Big Data’s Journey: From Big and Clumsy to Small and Cost-Effective

Author: Chanu Damarla, Senior Director of Product Management at GoodData (Twitter: @cd2k)

We’ve evolved from a world of pre-packaged business process solutions to democratized data. About a decade ago, the only way to get your hands on enterprise data analysis was to buy an enterprise business intelligence package from a provider like Microsoft and IBM. Millions of dollars later, you’d have a solution that could spit out monthly reports.

Author: Chanu Damarla, Senior Director of Product Management at GoodData (Twitter: @cd2k)

We’ve evolved from a world of pre-packaged business process solutions to democratized data. About a decade ago, the only way to get your hands on enterprise data analysis was to buy an enterprise business intelligence package from a provider like Microsoft and IBM. Millions of dollars later, you’d have a solution that could spit out monthly reports.

Meanwhile, the amount of data in the world was increasing. Primarily Yahoo! and other players confronted the problem by creating the big data technologies you hear about today. These include Apache Hadoop, Apache Hawk and NoSQL. Even these technologies, however, don’t come easily. If you want to architect them in-house, you need a team who knows how to build and manage big data solutions, and you need a data scientist to interpret the results for you. It ends up being another pricey proposition.

Now the big data model is changing again. It’s gravitating away from in-house systems and towards simple, SaaS-based platforms (software-as-a-service) that anyone can use to make better decisions — anytime. Services like Coupa, Zendesk and GoodData let you harness and analyze massive amounts of data with all the infrastructure already built in. You get the data insights you need without having to think about technical concerns, like how Hadoop is behaving. No data scientist or IT experts are needed. This is the true democratization of big data.

How to Choose the Best Data Platform

So if big data is becoming accessible to everyone through SaaS platforms, how do you decide which solution is right for you? Here are some baseline tips that we wish every vendor would adopt (sadly, they don’t):

• Provide an easy, intuitive way to parse data into a useful context. Data is meaningless unless it’s translated into something I can understand. Charts, graphs and visuals should be the foundation of any big data SaaS.

• Deliver information at the right time and on the right device. If I’m two blocks away from a potential new prospect now, don’t tell me that in a spreadsheet three weeks later. Tell me now, on my smartphone.

• Combine a consumer-friendly interface with enterprise-class security and data collection. Bring your own device (BYOD), the trend in which companies adopt consumer technologies because of widespread use by employees, has proven that people use technologies that are simple, yet powerful and intuitive. A big data SaaS provider should enable employees to use their our devices but be all business on the back-end, with the best security, data analysis and data collection technologies.

Finally, look for companies that provide support that you can trust, continually innovate new solutions and offer services that you can customize to your needs. With big data becoming a resource for everyone, the ball is in your court to choose the right provider.