LinkedIn mines data for future job paths

Let’s say you’re a 26-year-old accountant. You’re wondering about your future, and it’s easy to see the three most common paths. You climb up in an accounting firm, look for accounting work on your own or at another company, or you go back to school and get an MBA.

Let’s say you’re a 26-year-old accountant. You’re wondering about your future, and it’s easy to see the three most common paths. You climb up in an accounting firm, look for accounting work on your own or at another company, or you go back to school and get an MBA.

But how about outside those options? I was talking yesterday with Dipchand (Deep) Nishar, LinkedIn’s vice president of products and user experience. He came to LinkedIn from Google 18 months ago (The two headquarters are about a half mile apart) with a mandate to develop new data-centric products and services. And he’s walking me through this hypothetical accountant’s career conundrum. If you think about everyone who was ever a 26-year-old accountant, hundreds of thousands of people have wrestled with these same choices. And what paths did they follow?

That’s where LinkedIn’s data trove comes in. The company has some 70 million members. That’s data on 70 million careers. Conceivably, the company could provide a service showing each one of us the paths that others took when they were in the same position we’re in now. It could diagram where those choices led. “Maybe he ends up deciding to be a high school math teacher,” Nishar says. In that case, he could find current math teachers who have followed that path and debrief them.

Nishar says that this type of service, now under development, will be available by year end. Of course, to message the accountants-turned-math teachers directly, our 26-year-old would have to upgrade to an paid account at LinkedIn. That’s part of the business plan. But if Nishar and his team figure out ways to create these types of services, more of us might be willing to pay for them.

I asked Nishar how much data LinkedIn had. He wouldn’t say, and told me that the quantity of data was irrelevant. “I could have exabytes,” he said. “If I don’t do anything with it, it’s useless. To know the five people you should connect to, you might need only a kilobyte of data.” I suspect that LinkedIn has a relatively small trove of data compared to other social networks, because most of the LinkedIn stash is in words and numbers, not videos or jpegs. But those words and numbers could spell gold.

Nishar also pooh-poohed one current theory in data, espoused by Wired’s Chris Anderson, that Big Data will turn the process of discovery on its head. According to that school, which leans heavily on insights from Google:

Petabytes allow us to say: “Correlation is enough.” We can stop looking for models. We can analyze the data without hypotheses about what it might show. We can throw the numbers into the biggest computing clusters the world has ever seen and let statistical algorithms find patterns where science cannot.

Nishar, who headed up product development for Google in Asia, disagrees. “There are two types of consultants,” he says. “The unsuccessful ones collect lots of data. The successful ones start with a hypothesis.” He also maintains that gifted humans are far better than machines at picking out patterns in data.

This debate will rage for years. But if you learn about new job trends or other insights coming out of the data drove at LinkedIn, chances are it started with a hypothesis from a human on Nishar’s team.