EMC Study: Data Scientists in Short Supply

Author: Linda Rosencrance
Spotfire Blogging Team

We mentioned yesterday that data scientists continue to be a trending topic on twitter. And if we’ve said it once, we’ve said it a thousand times (OK, a thousand may be an outlier in this case)—data scientist jobs are so hot, they’re sizzlin’.

And a study released this week by EMC Corp. (@EMCcorp) bears this out.

According to the EMC Data Science Study, there’s a “looming” shortage of data science professionals. In fact, 65% of the nearly 500 data scientist surveyed say they believe demand for data science talent will outpace the supply over the next five years. And most of them think companies are going to have to look to recent college grads to fill their data scientist jobs.

The fact is there’s been an explosion of digital data created by mobile sensors, social media, surveillance, medical imaging, smart grids and other technologies. There’s also been an increase in new tools to analyze all that data.

That means companies have incredible opportunities to use that data to make better business decisions. That also means the business demand for data scientists will necessarily increase.

Organizations that best use their vast amounts of data are the most effective, according to Andreas Weigend (@aweigend), Head of the Social Data Lab at Stanford University, former Chief Scientist Amazon.com. To do that, organizations must have the tools, the skills and “more importantly, the mindset to see data as the new ‘oil: fueling a company,’” he says.

The problem: the technology has evolved faster than the workforce skills to make sense of it, he says.

Here are some other findings from the study that reinforce the need for companies to employ data scientists and/or for you to think about pursuing data science as a career or maybe even going back to school and getting the advanced degrees necessary to make a career change:

83% of respondents believe that new tools and emerging technology will increase the need for data scientists;
Data scientists are three times as likely as business intelligence (BI) professionals to have a master’s or doctoral degree;
Only 12% of respondents say today’s BI professionals are the most likely source to meet the increasing demand for data scientists;
Data scientists love the work;
Data scientists are more likely to be involved across the data lifecycle – from acquiring new data sets to making business decisions based on the data – than BI professionals;
Data scientists are more likely than BI professionals to use scripting languages, although Excel followed by SQL are the tools of choice for both data scientists and BI executives;
Data scientists need more business and technical skills than BI pros. According to the study, data scientists are twice as likely to apply advanced algorithms to data, and they’re 37% more likely to make business decisions based on that data.

“Neither tools nor people alone can solve the challenges of big data,” says Michael Driscoll (@medriscoll), Co-Founder and CTO at MetaMarkets (@metamx).

Rather they must work together. The trouble is that although there have been advancements in the software tools, not enough people know how to use the tools, he says.

“Data science is a young field, and its growth will be fueled as much by technology as through the mentorship of new acolytes by leading practitioners.” Driscoll says.