Defining The Analytic Process

I was asked the other day to explain what I meant by the terms Analytics and ‘Analytic Process’. My first answer was that “analytics is reporting with an intent to inform a decision that you want to make.” As I write this, it sounds a little clunky – and to be honest I probably just said at the time that “analytics is reporting with intent.” Remember – my world is commerce. I haven’t given much thought to other areas like academia and medicine where analytics is also used extensively. With that in mind, I then went on to explain what I thought an analytic process was and I thought that I would share this with you. To me, it is the process you go through to analyse a problem and make a decision. Here are what I think are the common steps involved: Define the Problem. Example: “We need to improve by 20% the success of our Product X marketing campaigns.” If you don’t know what the problem is, then…

I was asked the other day to explain what I meant by the terms Analytics and ‘Analytic Process’. My first answer was that

“analytics is reporting with an intent to inform a decision that you want to make.”

As I write this, it sounds a little clunky – and to be honest I probably just said at the time that

“analytics is reporting with intent.”

Remember – my world is commerce. I haven’t given much thought to other areas like academia and medicine where analytics is also used extensively. With that in mind, I then went on to explain what I thought an analytic process was and I thought that I would share this with you.

To me, it is the process you go through to analyze a problem and make a decision. Here are what I think are the common steps involved:

Define the Problem. Example: “We need to improve by 20% the success of our Product X marketing campaigns.” If you don’t know what the problem is, then 99 times out of 100 you should stop the process and go and do something else that is useful!
Generate Hypotheses. Use your experience and knowledge to guess ways in which you can use data to solve your problem. Often you will have several candidate hypotheses. These are ‘educated guesses’. Example: “We can better predict which existing customers will buy more products if we can understand better what existing Product X customers have in common.”
Determine what information you need & gather it. Traditionally this information came from a data warehouse or some other sort of database or electronic file. In recent years unstructured textual data, multimedia and other sorts of data stores are being used.
Test sources. Run some basic reports to see if the data can support one (or more) of your hypotheses. If you are lucky, you have access to a Business Intelligence tool like Cognos, Business Objects, QlikView, etc. Often you don’t and in these cases you can use SQL to extract data from their sources and import it into Microsoft Excel instead. Then test how good the source data is – or more precisely – determine how suitable the data is to our analytic purpose.
Test Hypotheses. Run further reports that are built to show how successful (or not) your hypothesis actually is. It is important to question your results and there are many way to do this. Here are a couple of the most common ways to do this:
- Find Root Causes. Look extra hard at the results that are the exceptions that do not support your hypothesis. Apply root cause analysis – this is analysis done in order to understand why the exceptions exist. Exceptions are often the data that do not fit your hypothesis. Example: “70% of the existing customers analyzed had purchased a car in the 2 weeks before they purchased a tow bar [US: tow hitch]. But for 30% the towbar was their first purchase – but they only work in cars we sell”. Concentrate on understanding the 30%. Example answer: “The tow bar purchaser for the exception 30% of customers, was 90% of the time either the spouse of the car purchaser or their employer”.
- Experiment. Identify a set of customers with something in common but that haven’t purchased Product X. Example: “30 – 40 year old women with 2 or more children under the age of 18”. Also, randomly select a second group of customers (i.e. those with nothing in common) to be your control group. Then run the same test campaign using both groups and compare the results for both sets of customers.
Review the results with your peers to check that the result really is valuable. Is it fit for purpose? Is the effort worth the benefit gained? Can you identify enough customers meeting the criteria to meet the uplift target?
Package the process. If you regularly need to analyse the same or similar problems then you need to make the process repeatable by putting it into ‘production’ and packaging it with additional information about how it works, any known issues, its purpose, etc.
Get user feedbackand evaluate the results by either tracking the decisions made based on the analysis or more informally. If you do the former, then congratulations – you are operating Decision Management. This is a holy grail for my work. More often you have to settle for informal feedback from the decision makers (or their colleagues). Example: “Analyst to Campaign Manager: So whadya think?”
Learn. Use this knowledge to improve the hypotheses, sources, data and process.
Repeat the process again until you get the result you want – or give it up as an unattainable goal.

Well that’s how I see the process – at a very high level. I went looking online to see what other people thought and there was very little to find in the commercial context. Even wikipedia let me down although it does have interesting things to say about Analytic Hierarchy Processes and the Intelligence Analysis Process used by NSA and the like. There is an old discussion thread on LinkedIn that’s relevant. I do remember reading a blog or article about the analytic process but I can’t find it at the moment.

So the questions I would like help answering are: