In anticipation of his upcoming conference presentation, A Transaction-Based Approach to Understand Sales Representative Growth, Performance, and Gaming, at Predictive Analytics World for Workforce, we interviewed Pasha Roberts, Co-Founder and Chief Scientist at Talent Analytics Corporation.
In anticipation of his upcoming conference presentation, A Transaction-Based Approach to Understand Sales Representative Growth, Performance, and Gaming, at Predictive Analytics World for Workforce, we interviewed Pasha Roberts, Co-Founder and Chief Scientist at Talent Analytics Corporation. View the Q-and-A below to see how Pasha has incorporated predictive analytics into the workforce of Talent Analytics Corporation. Also, glimpse what’s in store for the new PAW Workforce conference.
Q: In your work with predictive analytics, what specific areas of the workforce are you focused on?
A: We work with clients to solve workforce problems, using a data science approach to predict the ways that an employee will behave and perform. Only rarely are the problems neatly defined, but it usually comes down to estimating tenure or performance factors. We develop and deploy these models in the cloud to inform candidate selection and internal operations.
Q: Do you primarily work inside of HR – or inside of the Line of Business? If Line of Business – which one(s)?
A: Occasionally we find an HR department that is in sync with the vast potential of predictive analytics. These people are wonderful, and are earning their place at the future of their business; several are speaking at PAW for Workforce this year. Most HR departments have not crossed this chasm; they feel that their scope is more limited.
The line-of-business tends to actually feel the actual pain, and is typically more facile with data. We often work with the Sales, Service, or Call Center Lines of Business. These operations managers are on the front line and are accountable.
Q: What workforce outcomes do your models predict?
A: We create models that predict tangible outcomes – such as dollar sales, or calls per day, or hours worked, or error rates, or tenure/survival. The business leaders decide what is important to success in a specific role, and then we build models for those factors. For example, we may model the likelihood of a job candidate to achieve a top sales level within 6 months.
I am not a big believer in working with intermediate variables, such as engagement or job satisfaction. You can’t eat engagement. People may assume that engagement drives business performance, but that link needs to be proven case by case. At that point, you might as well predict the business performance directly.
Q: What is one specific way in which predictive analytics actively drives decisions?
A: We recently deployed a model to reduce bank teller attrition at a large bank. The model predicts the Cox proportional hazard for survival in the role, based on our aptitude tool metrics.
Advisor, our deployment platform, displays the probability of a job candidate to be on the job in one year. Recruiters use this number (actually cutoff levels of this number) to move candidates forward, or not, during hiring.
Q: Can you describe a successful result, such as the predictive lift of one of your models or the ROI of a predictive analytics initiative?
A: The AUC of the attrition model above is quite good, well over 0.70, with lift over 2.0 in the important regions.
The business benefit of this is two-fold – (1) fewer bad fits, which means less wasted training due to early attrition, and (2) more good fits, which means longer employee runs and higher lifetime value. The total benefit of this incremental change will be over half a million dollars.
Q: What is an example of surprising discoveries you have unearthed in your data?
A: Cluster analysis of a recent sales assignment revealed a clear group of sales reps who were succeeding largely by cheating. They were “poaching” sales from regions assigned to other reps, pulling low-hanging fruit away from others. Reps in this cluster barely worked any hours, but had very high performance numbers.
It just goes to show that you can’t use just one KPI. And yes, we had great lift in predicting job candidates who were likely to fall into this poaching cluster.
Q: What area of the workforce do you think has seen (or will see) the greatest advances or ROI from the use of predictive analytics?
A: The best predictive analytics can be done with largest sample sizes and quantified, quality output variables. We see the most of this in high volume roles, such as call centers, sales, retail banking, and insurance.
The costs and benefits of attrition/under-performance drive the ROI. In high volume situations, even a small increment can make millions of dollars of difference.
Q: Why do you think Business Leaders, HR Leaders and Analytics professionals should attend Predictive Analytics World for Workforce?
A: We need business leaders who understand what predictive analytics is, what it can do, and how to leverage the methods. It requires thinking about the world in a slightly different way, in terms of probability. This is not as hard as it may sound, and it is the key to unlocking a whole new level of corporate quality and performance.
Analytics practitioners should come to PAW for Workforce to learn – not only methods and technology, but to the sometimes-harder problem of business application and deployment. It’s the difference between solving equations and solving word problems.
Q: Do you feel any urgency you want to pass along to your fellow HR and Business Executives to implement predictive analytics to help solve employee challenges? Why?
A: I would like practitioners to realize the fact that there is a rare opportunity in employee analytics. Analytics can be so much more effective in hiring, because we directly choose employees, therefore predictions can directly drive results. This is a luxury that most other forms of analytics do not have – we don’t choose our customers, for example, so our ability to drive marketing results is indirect at best.
Q: What is one misunderstanding people have about using predictive analytics to solve employee challenges?
A: We predict patterns that tend to happen over time. We don’t predict what will happen in each specific case at each moment.
For example, if we predict that an employee has a 36% chance of staying on the job for one year; it’s still possible that they will last on the job for years. This doesn’t mean we’re wrong, because other employees with the same pattern may compensate.
As a machine learning geek, I love being “wrong” in this way, and love it more when managers disobey the models, because these variances only make new models stronger on the next iteration.
Q: How involved has the business unit been in the work you’ve done inside of your organization?
A: So far, very involved. The impact of our work boils down to understanding risk and tradeoffs, which is something senior managers often understand better than line managers.
You just have to speak their language, instead of going on about your latest foray into conditional random forest algorithms or how many Hadoop nodes were used.
Q: SNEAK PREVIEW: Please tell us a take-away that you will provide during your presentation at Predictive Analytics World for Workforce.
A: I am working with millions of geo-located sales transactions to discover and predict patterns in how they learn to sell. It is a fascinating dataset that chronicles thousands of reps as they sink or swim, including some who cheat their way to the top. I hope to deliver insights into how employees learn, as well as techniques to analyze large unaggregated data sets.
By: Greta Roberts, Conference Chair, Predictive Analytics World for Workforce 2015
