Data Scientists Not Required: Big Data Is About Business Users

Big data doesn’t sound like a provocative field. In a recent panel at the Global Big Data Conference, however, I said that the hype that has been building around the data scientist is over. Judging by the reaction, you’d think I’d have said that Abraham Lincoln is alive and living in Ohio.

Big data doesn’t sound like a provocative field. In a recent panel at the Global Big Data Conference, however, I said that the hype that has been building around the data scientist is over. Judging by the reaction, you’d think I’d have said that Abraham Lincoln is alive and living in Ohio.

The issue was that I was talking about the new school of big data, the one in which businesses use intuitive, interactive UIs to derive value from big data and avoid the dependency on data scientists. The people taking issue with this didn’t see beyond big data infrastructure — and the data scientists who interpret what comes out of those systems.

The panel I attended was called “Big Data Analytics: What’s the Next Big Thing for Data, Both Big and Small?” It featured one data scientist, three database infrastructure experts, a senior infrastructure architect and myself.

I stuck out like a sore thumb. As the Senior VP of Customer Success for GoodData, I was the only panelist focused on customer-facing activities. The tech guys were mainly focused on infrastructure issues, like speed, scalability and latency. I’m close to customers who are actually using the technology. It’s my job to be aware of how customers use big data to gain business value. So I see the world of big data in a different way.

I made three points that were seen as particularly controversial:

1. Companies need to provide solutions that a business user can understand.

Business users are hungry for expertise on how to use the massive amounts of data at their disposal. They often don’t know where to start, however, or who to turn to. Big Data providers need to educate business by providing best practices on how to derive value from their data. That involves metrics and visualizations that measure success, layered over the high-performing back-end solutions that everyone’s still talking about today. In other words, you need to know how to measure your marketing funnel, track customer engagement, benchmark yourself against the competition and maintain the health of your user base, among other functions. These are all business functions, and they must be presented in a way that anyone can understand.

2. Data in spreadsheets isn’t as powerful as a graph or a chart.

Spreadsheets are functional, but they don’t encourage deep interaction. To derive the greatest value from big data, users need to view and engage with their data on an ongoing basis. A spreadsheet doesn’t offer any motivation to do that. An intuitive UI, on the other hand, promotes regular useage, enabling companies to befriend their big data. Those visuals, in turn, encourage companies to make smarter, data-driven decisions. You just don’t get as much out of a quarterly report as you do from daily engagement.

3. The data scientist concept will die.

This is the one that really got people. Companies need solutions that enable them to use and customize their data easily, because it is the whole team, not just the individual analyst, that knows the business best. By offering business users intuitive data solutions, we bypass the need for the data scientist, who works in isolation. In fact, most data scientists are associated with the old school of business intelligence, where systems were so complicated that they needed someone with a data science background to run and get value from them. The new generation of solutions, on the other hand, is making it easy for business users to engage big data. An interdisciplinary team will see and use the visuals provided, and collaborate on the best decisions on a regular basis.

For me, speaking on the panel triggered a larger perspective of the market for big data. A new generation of data solutions is emerging. They are making big data accessible to the masses. That’s a hard fact for the old school to accept.

It’s time to move beyond infrastructure. We need to deliver applications that everyone can use, and turn big data into a democracy.

Cliff Cate, GoodData