Practical Data Analytics – When is “close enough” good enough?

Data analytics isn’t always about getting the right answer – it’s often about getting useful answers that help make the best decisions. There are many instances where the right answer doesn’t even exist. An example is if we’re using social data or a predictive model.  So how do we know when “close enough” is good enough?

Let’s say you are using data analytics to help prevent undesirable turnover of high potential employees in your organization. You have a model that predicts which high potential employees will quit next. So, the idea is to alert management so they can intervene.  In this case, there is no right answer – until someone quits, and then it’s too late. The model provides an indication along some continuum of the likeliness to quit.  You will draw a line on that continuum and intervene for every employee who falls above the line.  Hopefully you have the model inputs and an understanding of how the model works.

If the answer from the model is low, medium or high, is that “close enough” to help you decide where to place the line? What if the model produces a whole number from 0 to 10? Can you make a decision about where to intervene?  What if it provides half-steps (8.5, 9.0, 9.5)? What about tenths (8.1, 8.2, 8.3)? Hundredths (8.01`, 8.02, 8.03)? Pretty soon you’ll reach a point where an increase in precision doesn’t really affect your decision, and you’ve found your definition of “good enough”.

If the model doesn’t provide acceptable precision, you can assess the cost of increasing precision against the cost and consequence of intervening unnecessarily. Once you get acceptable precision, where you place the line will be a balance between your tolerance for risk (someone quits without intervention) and the cost and consequence of intervening unnecessarily.

As with any kind of change management, your success will be heavily influenced by the way you communicate this to others. If you think some users won’t find the results “good enough”, you should manage their expectations and either discuss the process you’ll take to improve the results, or discuss the economics of why improving the results will cost more than the problem is worth. In either case, it’s more likely that a bad decision will be due to a lack of understanding than a lack of information.

Steve McDonnell
Spotfire Blogging Team

Image Credit:  thechive.com