Copyright © 2009 James Taylor. Visit the original article at New Challenges for creating predictive analytic models.
Syndicated from Smart Data Collective
Khosrow Hassibi of KXEN and Will Tangalos of Wells Fargo presented together on the challenges of predictive analytics in the real world and on Wells Fargo as an example of how some of these challenges can be met. Khosrow began with an overview of the basic predictive analytic tasks:
- Understand the business problem
- Data access, understanding and preparation
- Model creation
- Assess and present the business value so the model gets accepted
- Scoring, deployment and monitoring to deliver the value
These have challenges but the challenges are being addressed. In optimized environments (modeling “factories”) the turn around for a
new predictive analytic model from problem identification to solution
deployed is 1-2 weeks. This requires lots of automated support but is achievable.
In more recent years, the role of analytics has expanded as success has been achieved in CRM, Marketing, sales and risk. This has resulted in higher expectations and increased demand and this has combined with new opportunities, like online media, has led to pressure on the model development process – must develop and deploy more models and do so faster. The challenges then:
- Managing large numbers of models
Companies want more models and want them to be more focused. For instance, a telco that insisted on a model for each product for each region that changed monthly – 1,600+ models a year! They got great results using these models – 260% increase in response but this required automation and model management. - Data volume, variety, velocity and validity has improved
More and more data sources are available and more events and attributes are being stored. Time series, interactions and text data are all contributing to an exponential growth in data. For instance a credit card issuer has several thousand attributes about a customer and wants to select the 10-20 best variables. Using automated attribute detection they managed this in a staff day rather than 2-3 weeks. - In-warehouse modeling/scoring
Getting models in a timely fashion there is a need to do the modeling without having to move data around. This requires execution in the warehouse by generating code that can be executed on the warehouse system. - Relationship or social network data
Not just social media but email trails or call detail records – all ways to establish a web of relationships. Making this information availble to models is a challenge but it provides a new perspective. Lots of new potential attributes for modeling. Deriving these atttributes from call records, for instance, provided lift of between 9 and 13% but the data volumes and complexity are high. - Use of under-utilized data
Web, unstructured and other under-utilized data can be used to develop behaviorial models. This needs lots of pre-processing and other changes.
Will came next to discuss Wells Fargo’s experience in email modeling. Wells has groups focused on both online sales and marketing and email marketing – what to display when someone is online and what to proactively send them. Email has to coordinate with lines of business and does not have a long history of email targeting. Changing this, and using models, was a challenge because the data infrastructure needed to be created and organizational buy-in for the use of analytics had to be managed.
They have created a data infrastructure to bring data to KXEN and can now turn around models in less than a week from problem statement to scored data. They still have to select from all the possible models they could develop because there are so many products that could be targeted. They also have to manage the organizational buy in as channel and product managers no longer have as much control. A model, for instance, to reactivate online banking was delayed for a month while everyone was brought on board and agreed to a test. Even doing this required senior management support but, now it’s done, there is enthusiasm and demand for models amongst these folks.
Lessons learned:
- Create organizational buy in – communication, education, upfront approval
- Tools and infrastructure to improve modeling value – automation to reduce time, ease of use to bring in less technical analysts, in-warehouse scoring for quick operationalization
- Results drive success
