The Demise of the Data Scientist: Heresy or Fact?

As an avid practitioner of data science, I purposely try to read articles that provide new learning and knowledge in this discipline or which at least tend to be controversial from my perspective. One article from an IT leader of a well-respected U.S. organization hypothesized that data scientists will in the future become like switchboard operators:obsolete.

The primary reason for this declining demand according to the author was that increased automation and operationalization of business processes will not require the technical skills of the data scientist. Obviously, my initial reaction was emotional with my first reaction being to completely ignore this opinion but the more rational part of my nature aroused my curiosity in not what was being said but why it was being said. The “what is being said” does present an opinion which is based on his perspective. But it is this perspective that must be understood if we are to obtain a better sense of this viewpoint.

This perspective can be better understood by knowledge of this individual’s background and experience. For the writer in question, the background is IT or computer science/systems development and not data science in the truest sense. The IT practitioner, here, does have some understanding of data science but not as a true practitioner of the discipline. Both IT and the data scientists do develop business solutions. But the types of solutions are different. IT individuals steeped in the more traditional computer science discipline are trained and developed to focus their skills on developing solution that streamline business processes. In most cases, this is about operationalizing or automating a given business process.

Meanwhile, the data scientist focuses their efforts on developing analytics solutions that solve a specific and unique business problem. In most cases, these data science solutions when initially built, are customized to solve the problem at hand . The need for automation or operationalization is not paramount at least initially in the data scientist’s mind.

With Big Data and big data analytics, the need for analytics and more customized type solutions is experiencing exponential growth. Methods and approaches in employing analytics need to be quicker and more flexible which require I/T support for more operationalization and automation . But this does not replace the data scientist. In fact, big data analytics reinforces the need for more and not less data scientists. The data science “work” is really about building the right template for a given business solution. In specific terms, this could be a predictive model or a pivot table to conduct a given business analysis. Meanwhile, IT needs to create the necessary technical and systems architecture towards a more automated infrastructure. In this environment, the data scientist has the capacity to create the more automated templates for many different types of business problems. Some may question that this is the role of IT.

The answer is no as these templates are uniquely designed for each business problem, but automation or operationalization of these solutions is facilitated through the infrastructure that has been set up by IT. Architecting the data to solve the business problem is the role of the data scientist. Creating a technology environment which allows access to data and the potential for increased automation is the role of IT. This kind of scenario is simply going to increase the demand for both data scientists and IT in our Big Data world regardless of certain opinions on the declining importance of data science.