Data Mining in the New Economy and How to Get Started

October 1, 2010
120 Views

We are all, to some extent, experiencing the drastic transformations that are underway within our current economy.  Spurred by the sub-prime mortgage debacle, the world’s economy seems to have folded like a house of cards with the result being that significant job losses are occurring throughout the world. Within certain sectors such as manufacturing, many of these jobs are gone forever. This major economic transformation is not new as economic historians can recite numerous time periods where the economy underwent major changes.

We are all, to some extent, experiencing the drastic transformations that are underway within our current economy.  Spurred by the sub-prime mortgage debacle, the world’s economy seems to have folded like a house of cards with the result being that significant job losses are occurring throughout the world. Within certain sectors such as manufacturing, many of these jobs are gone forever. This major economic transformation is not new as economic historians can recite numerous time periods where the economy underwent major changes. The most familiar transformation to most people of course is the Industrial Revolution of the 19th century where machines began to perform tasks that were manually done by humans.  Most economic pundits have attached the name ‘The Knowledge Economy’ in attempting to put some description towards the transformations occurring in today’s economy.  The cause of this transformation is twofold, with the first being  access to information from ever-increasing computer technology and the second being the ability to use this information for meaningful insight and intelligence.

 

Besides this economic transformation and the requirement for corporations to become more knowledge-based, we are also witnessing social transformations.  In particular, tremendous emphasis is now focused around the environment and how we as a society can be responsible stewards of its resources.  For marketers, the challenges are the execution of marketing program activities but with significantly less resources.  Direct mail campaigns that are untargeted are now an extinct business practice.

 

Alongside the need to become more effective at using environmental resources, the new economy has also resulted in less corporate resources being available to do a given task.  Less people are available to do the same task.  Once again, the notion of ‘doing more with less’ is being reinforced.

 

These economic and societal changes are resulting in demands for skills that focus on information and the ability to mine that information into meaningful business intelligence. Clearly jobs such as those related to data mining and analytics that provide this capability will represent areas of growth in the new economy.  

 

 

ROI is a critical output from data mining and analytics 

 

Even in today’s economic downturn, discussions with data mining counterparts have indicated virtually no decrease in their work activities and in many  cases an actual increase in their workload. None of this should be all that surprising if we understand that one of the demands from the new economy is accountability.  Within the marketing area, accountability is about measurement, with the marketing mantra being “What gets measured gets done”.  Establishing an ROI on every marketing initiative is now a common discipline for most marketers with data miners and analysts representing vital components of the marketing team. The data miner’s ability to understand data along with the marketing business needs enables organizations to create ROI templates for a variety of marketing initiatives.

 

 


Data mining solutions to optimize ROI

 

In addition to providing a level of accountability through measurement, another demand of the new economy is the ability to create solutions that actually maximize ROI, as opposed to revenue maximization that was the traditional marketing objective through most of the 20th century.  The data mining industry itself had traditionally been focused on the development of predictive models seeking to maximize revenue (e.g. response).   For many organizations, data mining is still a daunting term and represents a formidable challenge in becoming embedded as a common business practice.  How can organizations change this?  Like most business roadblocks, the key is to identify quick wins that demonstrate significant positive benefit within a very short timeframe.

 

 

Customer Value as a quick win

 

Let’s look at some examples of quick wins. The first and often most common quick win is for an organization to identify its best customers. Using the principle of the 80/20 rule, we attempt to see how company revenues are distributed amongst its customers. Using purchase revenue (which for most organizations is more than adequate in identifying customer value), we can create a measure of a customer’s value by summing-up each customer’s purchase revenue over the last 12 months.  Customers are then sorted into deciles (10 equal size groups of customers), with decile 1 being the highest value group and decile 10 being the lowest value group.   

The production of this type of report (often referred-to as a “gains table” or decile report) might reveal that 60% of the company’s revenue is being delivered by 20% (deciles 1 and 2) of customers (not quite the 80/20 rule in this example).  Customers in the top two deciles would represent our highest value customers.  Medium value customers might  be represented in deciles 3 and 4 where they account for 25% of all revenue.  Low value customers are found in the remainder of the file (deciles 5-10) and account for only 15% of all revenue. This information is a critical enabler for organizations to effectively deploy customer management strategies and tactics.  The basic ability to prioritize marketing activities against different customer groups based on customer value will yield significant performance returns.  For example a greater proportion of focus and resources should be devoted to customer segments that deliver the highest returns.

 

Change as a quick win

 

Another quick win looks at the notion of changes in customer behaviour.  Marketers historically have been interested in being able to identify changes for a customer and in particular, lifecycle type changes.  The key of course is to be able to identify these lifecycle changes based on what we can observe from customer behaviour and demographics.  Knowledge of a given customer that is a university student and is about to graduate would represent key information to a marketer in being able to design the appropriate programs that might appeal to this person. Suppose we observe that a person has just turned 65 and we notice that certain types of expenditures have significantly changed.  Expenditures related to gas purchases have decreased while purchases related to travel have increased. One could certainly impute that this customer may have become a retiree.   In other examples, we might see a married person significantly increase purchases on categories related to furniture.  This might suggest that this person is perhaps buying a house.  With this same married person, we might observe that purchase activity has now significantly increased in areas such as baby clothing, implying that this same person has now become a parent.  In all these change examples, data is required in order to make inferences like those above.  Inferences can be made with any data but the type of available data will dictate how confident we are in our inferences and how we manage the expectations of those using the insights.   For example, a change in an investment portfolio whereby a customer aged 30 has gone from a more risky portfolio to a less risky portfolio might suggest a number of events; the customer might be getting married, starting a family, or simply just changing their investment strategy.  In any case, the marketer would develop strategies and tactics that could leverage these insights.

 

 

Engagement as a quick win

 

Change and value both represent good quick win areas to focus on from a data mining perspective, but another area relates to customer engagement. Although engagement can in many cases be similar to value, it is not always the case.  A high value customer at a bank may have a large loan and large mortgage, yet make all my payments through automatic withdrawal.  If this is the way the customer interacts with the bank, the customer’s engagement level may be very low.   We need to consider other types of information in order to obtain a true reflection of customer engagement.  With the growth of digital marketing, the extent of customer engagement becomes much broader as we can now look at customer-initiated activities like email communications, web site browsing activity, in-bound telephone calls and other means of customer-initiated communication with the company.  This communication-initiated activity complemented with purchase activity and  response activity across all channels can be integrated to create a more complete measure of customer engagement.

 

 

Conclusion

 

In today’s new economy, data mining and analytics is quickly becoming a common business discipline.  Given that this discipline is new, most people tend to consider data mining and analytics as a more advanced statistical technique.   It is this perception that has created the perception of complexity regarding the use of data mining and analytics as a regular business discipline. I hope that this discussion has shed some insight that simple techniques can be used to yield significant wins. More importantly, it should reinforce the impetus to create a data mining and analytics culture in the organization that will ultimately yield to the more significant advantages available when using advanced statistical techniques.

 

—Richard Boire

Partner, Boire Filler Group