Norman Nie on Two Big Problems with Big Data

Revolution Analytics CEO Norman Nie sat down with Cassimir Medford from Business Agility to talk about the problems business today face with respect to Big Data. The two big problems identified: finding adequately trained personnel and locating the right tools.

Norman traces the problem of finding skilled practitioners to work with Big Data to the US educational system:

The US is currently experiencing an acute shortage of mathematicians and others trained in related fields such as statistics.
The problem, Nie believes, is American students are afforded early opt-out of their majors, and many mathematics majors take advantage of that. They tend to go into other fields that involve less academic work and more money in the labor market, often heading off to the financial market to create more exotic investment derivatives.

Data analytics requires knowledge in multiple fields. For instance, a math major might need some familiarity with social sciences such as sociology, psychology, or biology. And candidates with degrees in the social sciences often lack sufficient math training.

As a result, recruiting to a data science team can be a challenge (but this guide from O’Reilly Radar provides some helpful advice). And once you’ve built such a team, solving the second problem — locating the right tools — becomes doubly important. “The key”, says Norman, “is software parallelization on multiprocessor commodity hardware. That is the future of big data storage and analysis.” That’s why Revolution Analytics created Revolution R Enterprise, to bring Big Data capabilities to the R language, the data analysis software preferred by more than two million data scientists around the world.

Business Agility: Two Big Problems for Big Data