How HR Can Use Big Data in a Smart Way (Hint: Most Are Not)



HR and people management have been taken over by the data and analytics wave. This part of organisations that has traditionally focused on the softer things such as culture, talent development and employee engagement is now obsessed with data analysis.


HR and people management have been taken over by the data and analytics wave. This part of organisations that has traditionally focused on the softer things such as culture, talent development and employee engagement is now obsessed with data analysis.

And with good reason.  Human resources are one of the most data rich functions) in most businesses. There are tonnes of performance scores, employee demographics, training stats, recruitment data, culture audits, etc.

In the past this data was mostly ignored or at most put into charts and tables and stuck into the corporate performance pack. Now, in the era of big data and analytics, companies want to turn their data into insights, predict when employees will leave, where to recruit the most suitable candidates from, how to identify and retain top talent.

According to the Economist Intelligence Unit, more than half of HR departments have already reported an increase in data analytics since 2010, and that trend is likely only to grow.

Crunching HR Data.

Innovative HR departments have been crunching their data for a while now and many have gained new insights. Consider these successful examples: 

  • One bank assumed their best performing people would be those with Ivy League degrees, but data analytics clearly showed the assumption was wrong. It turned out that candidates from non-prestige universities outperformed the top-university candidates, allowing the business to recruit the right talent for less money.
  • One of my retail clients is now able to analyse social media profiles of job candidates in order to very accurately predict the level of intelligence as well as the emotional stability of potential candidates.
  • Another client of mine wanted to recruit self-driven people that are able to take initiative. By analysing different data sets from the type of people they wanted to recruit and those they wanted to avoid, the company found that candidates who filled out applications with browsers that were not pre-installed on their computers and instead had to be installed separately (such as Firefox or Chrome) tended to be better for that particular job.
  • A retail company I work with can now predict how a specific increase in staff satisfaction will drive a certain increase in revenue by square foot in their shops.
  • Another company found that call centre sales staff with a criminal record performed better than those without a criminal record!

And those are just a few examples of companies crunching the numbers and extracting real meaning and value from their HR data

Crunching The Wrong HR Data.

One big problem I am seeing is that most HR departments simply start crunching the masses of quantitative data they have in the hopes of finding something. This is like looking for the proverbial needle in the haystack.

As with any big data or analytics project, the key is not in the quantity of data points, but in the quality of the questions being asked about the data. 

What HR departments should be doing much more is embracing qualitative and immersive data and analysis – not just simple quants of existing data. Instead of analysing the masses of structured numeric data like accounting and finance departments might do, HR departments should build on their strengths and capture more qualitative data. The less quantitative and more qualitative side is what HR departments have traditionally been good at and where big data offers huge opportunities. 

Instead of counting training hours per employee and crunching the data to death, one of my clients is now using an ethnography-type data collection to gain powerful insights about training effectiveness using qualitative methods. The company is now collecting and analysing data using written training journal logs, video analytics as well as performance impact assessments. The company realized it needed to look beyond the simple existing metrics to answer the real business questions.

How Google got it right.

Google is one of the most successful companies in the world. It has 28,000 employees or ‘Googlers’ across 60 offices in over 30 countries. Unlike most HR departments Google HR’s team states the objective that ‘All people decisions at Google are based on data and analytics’.

At the beginning, Google founders believed that middle management wasn’t important, so they did away with the positions. When it became obvious that this belief was false, they brought the managers back, but the perception that the managers weren’t valuable persisted.

So, Google turned to data to quantify the value of managers. Through data, they were able to go from the opinion: ‘Managers don’t impact performance’ to proving that great managers had a statistically significant impact on team performance, employee engagement, employee churn and productivity. For the analysis Google looked at data from performance reviews as well as qualitative interviews and the submissions for a ‘best manager’ award program.  By extracting the insights from that analysis using qualitative methods such as text analytics, Google was then able to identify and articulate what made a great manager for them and what caused less proficient managers to struggle.

These insights were then embedded into Google culture through twice-yearly performance evaluations of these factors, which acts as an early warning system to detect both great and struggling managers. For those that are struggling, there is access to improved training and support as well as plenty of role models to learn from. And for those who are doing well, there is the Great Manager Award.

And all this was possible because they started with the right question; refining their question until they got to a practical, verifiable analytics.

The possibilities here are practically limitless. But rather than just applying data for data’s sake, HR professionals need to remember that their qualitative skills are still their most valuable assets and to go beyond the numbers to use data to drive HR strategy and action.

As always, thank you very much for reading my posts. You might also be interested in my new big data case study collection, which you can download for free from here: Big Data Case Study Collection: 7 Amazing Companies That Really Get Big Data, and my article How is Big Data Used in Practice? 10 Use Cases Everyone Must Read.

And if you would like to read a free sample chaper of my book: Big Data: Using Smart Big Data, Analytics and Metrics To Make Better Decisions and Improve Performance You can find it here.

Bernard Marr is a best-selling author, keynote speaker, strategic performance consultant and analytics, KPI and Big Data guru.