Big Data Scientists Are Bridge Builders

Kathryn Kelly recently put up an excellent post titled Data Scientists Not Required: Big Data Is About Business Users. I commented on it, but my comment ended up being so long and involved, it seemed like a blog post in itself, so, here’s my view on the same subject.

I completely agree with Kathryn on her other two main points:

Kathryn Kelly recently put up an excellent post titled Data Scientists Not Required: Big Data Is About Business Users. I commented on it, but my comment ended up being so long and involved, it seemed like a blog post in itself, so, here’s my view on the same subject.

I completely agree with Kathryn on her other two main points:

1. Companies need to provide solutions that a business user can understand.

2. Data in spreadsheets isn’t as powerful as a graph or a chart.

But I disagree with her third point:

The data scientist concept will die.

I don’t think that the data scientist/data analyst/statistician, whatever they’ve been called over the years, will disappear. The name “data scientist” may be new, but the job isn’t. And there’s a good reason for that.

Analytics, over time, becomes more and more accessible to the business person or average end consumer of that information, yes. The goal of most analytics projects and analytics software is to bridge the gap between the person who asks questions and the person who can find answers. Data scientists and the tools they use are the bridge for a time, then as the technology matures, they become the designers and builders and maintainers of the bridge. But once the bridge is built, anyone can walk across. When the person who asks the question and the person who can find the answer are the same person, the bridge is built, that area of questioning is mature.

We, as curious human beings, once we’ve reached the point where we can find all the answers to the questions we’ve been asking, simply find tougher questions to ask. Then, you’re right back where you were, needing a person with greater knowledge of analytics and analytic technology to dive in and find them, and be our bridge to that information.

I don’t see that tendency ever going away as long as businesses are run by human beings. How long ago was it that you needed a specialist to tell you how the stock market was doing, or what the weather would be like next weekend, or what the population of Uruguay was? Now, I can find all that information out on my phone.

Somewhere, there is still a statistical expert making the predictions on what the weather will be like next weekend, but I don’t have to go through that person to get to the information. Does that mean there are no more meteorologists? Of course not. Someone is still designing, building and maintaining those bridges. I just no longer need them to hold my hand and walk me across.

So, now I ask tougher questions.

How will the weather next weekend affect the stock market prices in Uruguay?

I can’t figure that out on my own. There’s no bridge built. I would need a data scientist to design, test, and build it for me.

As long as human beings continue to ask questions, to push the envelope, and to be hungry for more and more information, ie as long as they continue to be human, data scientists will have a job.