In Analytics, It’s the Actions that Matter

May 1, 2009
173 Views

In this note, let’s define analytics as the analysis of data in order to take actions. (This is a narrow definition of analytics, but one that is useful here.) If you don’t have day to day work experience with analytics, it is easy to have the mistaken impression that analytics is only about data and statistical models.

Although understanding data and developing statistical models is certainly an important component of an analytic project, this is just one aspect of analytics. This aspect includes cleaning data, enriching data, exploring data, developing features, building models, validating models, and iterating the process. From a broad perspective, this is a process in which the input is data and the output is a statistical model. When most people think of modeling, this is what they think of. For many analytic projects, this is just a small part of what is required for a successful engagement.

The second aspect of analytics is what I am concerned with in this note. This is the aspect of analytics concerned with:

  • developing an appropriate score for a statistical model;
  • using the score to define useful actions;
  • determining which measures are best for evaluating the effectiv

In this note, let’s define analytics as the analysis of data in order to take actions. (This is a narrow definition of analytics, but one that is useful here.) If you don’t have day to day work experience with analytics, it is easy to have the mistaken impression that analytics is only about data and statistical models.

Although understanding data and developing statistical models is certainly an important component of an analytic project, this is just one aspect of analytics. This aspect includes cleaning data, enriching data, exploring data, developing features, building models, validating models, and iterating the process. From a broad perspective, this is a process in which the input is data and the output is a statistical model. When most people think of modeling, this is what they think of. For many analytic projects, this is just a small part of what is required for a successful engagement.

The second aspect of analytics is what I am concerned with in this note. This is the aspect of analytics concerned with:

  • developing an appropriate score for a statistical model;
  • using the score to define useful actions;
  • determining which measures are best for evaluating the effectiveness of these actions;
  • tracking these measures (often with a dashboard) and making sure that that they advance the strategic objectives of the company or organization.

One way to remember this is using the mnemonic SAMS for Scores, Actions, Measures and Strategies.

For example, with a response model, often a threshold is used. If the score from the response model is above the threshold, an offer is made (this is the action); if not, no offer is made.

Here are some examples of SAMS:

ModelScoreActionMeasureStrategy
on-line response modellikelihood to respond to an offerdisplay the offer to the visitor that has the highest likelihood of response and available inventoryrevenue per day generated by the web siteincrease revenue from a website by improving targeting of offers
fraud modellikelihood that a transaction is fraudulentapprove, decline, or obtain more informationdetection and false positive ratesreduce costs and improve customer experience by lowering fraud rates
data quality modellikelihood that a data source has data quality problemsif the score is above a threshold, manually investigate the data to check whether there is in fact a data quality problemdetection and false positive ratesimprove operational efficiencies by detecting data quality problems more quickly

A successful analytics projects requires a careful study of what actions are possible; of the possible actions, which can be deployed into operational systems; and, how the systems can be instrumented so that the data required to compute the required measures is available.

For more information, see: blog.rgrossman.com