Predictive Models are Only as Good as Their Acceptance by Decision-Makers

August 25, 2010
10 Views

I have been reminded in the past couple weeks working with customers that in many applications of data mining and predictive analytics, unless the stakeholders of predictive models understand what the models are doing, they are utterly useless. When rules from a decision tree, no matter how statistically significant, don’t resonate with domain experts, they won’t be believed. Arguments that “the model wouldn’t have picked this rule if it wasn’t really there in the data” makes no difference when the rule doesn’t make sense.

I have been reminded in the past couple weeks working with customers that in many applications of data mining and predictive analytics, unless the stakeholders of predictive models understand what the models are doing, they are utterly useless. When rules from a decision tree, no matter how statistically significant, don’t resonate with domain experts, they won’t be believed. Arguments that “the model wouldn’t have picked this rule if it wasn’t really there in the data” makes no difference when the rule doesn’t make sense.

There is always a tradeoff in these cases between the “best” model (i.e., most accurate by some measure) and the “best understood” model (i.e., the one that gets the “ahhhs” from the domain experts). We can coerce models toward the transparent rather than the statistically significant by removing fields that perform well but don’t contribute to the story the models tell about the data.

I know what some of you are thinking: if the rule or pattern found by the model is that good, we must try to find the reason for its inclusion, make the case for it, find a surrogate meaning, or just demand it be included because it is so good! I trust the algorithms and our ability to assess if the algorithms are finding something “real” compared with those “happenstance” occurrences. But not all stakeholders share our trust, and it is our job to translate the message for them so that their confidence in the models approaches are own.

You may be interested

How Email Automation Is Driving Sales
Marketing
0 shares68 views
Marketing
0 shares68 views

How Email Automation Is Driving Sales

Ryan Kh - May 24, 2017

Email automation is one of the best ways of ensuring that you save time and fulfill your email marketing tasks…

Are Flexible Work Policies Coming to IT?
IT
0 shares126 views
IT
0 shares126 views

Are Flexible Work Policies Coming to IT?

Larry Alton - May 24, 2017

With new remote technologies and shifting expectations, flexible working has become much more realistic over the past couple of years.…

Administration of Spring Boot Applications
Software
0 shares165 views
Software
0 shares165 views

Administration of Spring Boot Applications

Johnnymorgan - May 24, 2017

Technology:  Spring Boot Admin is the application for managing and monitoring based on Spring Boot Applications. The application can be…