Businesses have been eager to use big data as a way to grow their companies, reach more customers, and make their operations more efficient, but utilizing big data analytics takes having the right talent at hand. Data scientists can be highly valuable members of any workforce, but at the moment, the demand for them is far outpacing their numbers.
Businesses have been eager to use big data as a way to grow their companies, reach more customers, and make their operations more efficient, but utilizing big data analytics takes having the right talent at hand. Data scientists can be highly valuable members of any workforce, but at the moment, the demand for them is far outpacing their numbers. While estimates vary, some experts place the shortfall of data scientists as high as 140,000. Some are even referring to this shortage as a “crisis.” Put simply, businesses want to have data scientists on their teams, but finding them has proven exceptionally challenging. The problem doesn’t appear to be going away either, with more and more companies seeking to hire data scientists each year. With this problem clearly in mind, organizations are looking for alternative approaches to finding the right data talent. One of those ways that has slowly gained more acceptance is the use of crowdsourcing. While it may seem an odd choice for finding data scientists who are talented at machine learning, big data platforms, and engines like Apache Spark, crowdsourcing could prove to be an effective means to discover the people who would make perfect fits for a business.
One of the best examples of finding success with this approach comes from Walmart. While some may think a large company like that shouldn’t have any difficulty finding the talent they want, Walmart executives were finding the data talent shortage a significant barrier. To solve the problem, they turned to Kaggle, a crowdsourced analytics competition platform. Kaggle works by holding competitions that data scientists can enter, solving analytics problems that are put up by actual companies. While their competitions, like many crowdsourced projects, usually featured a financial prize for the determined winner, Walmart had different plans in mind. In this case, the best data minds from the competition would be offered jobs on Walmart’s data analytics team.
The results were a total success for Walmart. Data scientists for various backgrounds were able to demonstrate their skills solving real world problems, giving the company a better idea of what they could accomplish. In one instance, one of the winners was an expert in physics with little formal data analytics background. Had Walmart recruiters gone only on resumes, he probably wouldn’t have been given a second thought. The crowdsourcing was deemed to be such a success, Walmart held the same type of competition the next year.
From these outcomes and others like it, it’s clear that crowdsourcing offers up some unique advantages when it comes to finding high quality big data talent. As seen in the Walmart example, it expands the pool of potential applicants. Recruiters no longer have to depend on words on a resume and can instead focus on the work of people coming with different experiences of their own. Crowdsourced competitions also allow potential employees to show off their skills, not just in big data but in communication. After all, their solutions often have to be explained to corporate executives, many of whom will have little experience with big data analytics. Testing out so many people also opens up the possibility of getting referrals from new employees, once again expanding the number of people a company may target in the future.
While crowdsourcing has definite benefits, it’s not without its challenges as well. Crowdsourced competitions can be difficult to organize and set up, especially when the problem presented is as complex as those posed by big data. Also worth noting is that many enterprise-grade tools used in competitions don’t come with automatic ranking and benchmarking features. Luckily, there are numerous helpful data crowdsourcing platforms to choose from, like the aforementioned Kaggle, InnoCentive, and CrowdANALYTIX. Companies that use these platforms will be in a better position to capitalize on crowdsourced talent.
Crowdsourcing could very well represent the wave of the future, at least when it comes to hiring data scientists. The shortage of data talent is very real, something all too many organizations are experiencing, so it stands to reason that unique ways of finding that talent would come to the forefront. While crowdsourcing has difficulties of its own, it has also shown to be an effective strategy for seeing who would really make a valuable addition to a data team.
