Avoid Screwing Up Predictive Analytic Projects

There’s a great article over on Computerworld – 12 predictive analytics screw-ups.

There’s a great article over on Computerworld – 12 predictive analytics screw-ups. They asked some of my favorite data miners (John Elder and Jeff Deal of Elder Research, Eric Siegel of Prediction Impact and Dean Abbott of Abbott Analytics) what they saw as the top ways to screw up predictive analytic projects. The list of 12 is great – every one is worth committing to memory. What I want to do though is point out an effective way to avoid not one, not two but five of these problems – Decision Modeling.

In a decision model like the one shown to the right (based on the emerging standard for Decision Model and Notation and built using our modeling tool, DecisionsFirst Modeler) you specify how a decision will be made precisely by decomposing it into its component decisions (rectangles), you show how this decision making consumes information (ovals) and you show where the knowledge required to make a decision comes from (documents). This knowledge might be policy-based, expertise or analytically derived. Because the purpose of an analytic is to improve decision making, any analytic project can develop such a model to describe which decision(s) will be improved and where the analytic fits in this. There’s more on how to do this in our white paper.

Using Decision Modeling to describe the business requirements for a predictive analytic project helps address 5 of the 12 screw ups. Starting in best Letterman style with the end of the list:

12. Don’t define clearly and precisely within the business context what the models are supposed to be doing.
As Dean Abbott said “Too many people are just trying to build good models but have no idea how the model actually will be used.” A decision model clearly defines the business context for the model – the business decision making that the model is designed to influence along with all the other influences on the decision-making.

10. If you build it they will come: Don’t worry about how to serve it up.
Decision Models make it clear how the model affects business decision making but they can also be linked to the business processes, business events and information systems that need to make those decisions. We find that once someone has a decision model it is much easier to see where that decision (and thus the analytic) will be used. From this an effective deployment strategy can be developed.

8. Ignore the subject matter experts when building your model.
Decision models help with this by modeling how expertise (and regulations, policies) matters to decision-making so it is clear what other influences there will be. By breaking down the decision-making formally into a model you can also manage the organizational relationships involved. In DecisionsFirst Modeler, for instance, we let you record which organizational unit owns each part of the decision-making, which ones make each decision and which other ones might care. This let’s you see clearly who cares about which model and why.

3. Don’t proceed until your data is the best it can be.
I often find companies that tell me they can’t do any analytics because their data is not perfectly ready. However the quality of data you need is driven in large part by the decision you are trying to influence. If the people making the decision are currently guessing then your data only has to be good enough to support a model that gives you a 60/40 prediction to be useful. If on the other hand you are already making a very precise decision then your data will have to be proportionally better.

1. Begin without the end in mind.
Or as I like to say “Begin with the decision in mind.” The value of predictive analytics comes from improving decision-making. Begin by focusing on the decision you wish to improve and use that to drive your predictive analytic projects.

Decision modeling has a lot to offer analytic projects so why not read our white paper on Decision Requirements Modeling for Analytic Projects or contact us to learn about Decision Management for Predictive Analytics Projects.

P.S. It works for BI projects too….