The Importance of Data Experimentation

The McKinsey Quarterly had an article today on Ten Tech-enabled business trends to watch and number 5 caught my eye -Experimentation and big data. As the authors say

What if you could analyze every transaction, capture insights from every customer interaction, and didn’t have to wait for months to get data from the field?

The McKinsey Quarterly had an article today on Ten Tech-enabled business trends to watch and number 5 caught my eye -Experimentation and big data. As the authors say

What if you could analyze every transaction, capture insights from every customer interaction, and didn’t have to wait for months to get data from the field?

If you could, and of course you can, then you could constantly experiment to see what would work best for your business. The article talks about using experimentation to make strategic choices about

new products, business models, and innovations in customer experience

and goes on to give some great, if well known, examples.

What is interesting to me, however, is that all the examples are really about experimenting with operational or micro decisions – transactional, customer-specific, small scale decisions – that are taken in huge numbers such as what to display to a customer, what promotion to offer to a customer, what credit line to extend. The management or strategic decisions involved are those about picking between alternative approaches – the experiments being conducted. These strategic choices are only possible if the micro decisions are being managed and experimented on. This has two consequences:

1. You cannot conduct the kind of experimentation being discussed unless you have control of these micro decisions – unless you have adopted a decision management mindset in other words.
2. When you implement systems to automate these micro decisions – when you build decision services – you need to build in the capability for experimentation

This is a challenge because, frankly, IT departments don’t like experimentation and adaptive control or champion/challenger testing. Most folks in IT think there should be a “right” answer and that the code should implement it. Persuading them that multiple approaches should be allowed so that other approaches can be tried, many of which will be worse than the default, can be very difficult. Nevertheless, this capability should be built in to your decision services and the decision analysis reporting that you develop.

Similarly many business people don’t like the idea that some customers will be “tortured” with an experimental approach that is likely to be worse than the default. They want to treat every customer the best they can and have a hard time letting go of the short term to build understanding for the long term. They too may resist experimentation.

But experimentation is going to be more and more important. Companies that can use their data to push the envelope, to get better and better, will out perform those that stick to the tried and true. Making test and learn part of your normal approach to business will be important, though not easy. As the article says:

Using experimentation and big data as essential components of management decision making requires new capabilities, as well as organizational and cultural change.

And one of those changes is making operational decision making a corporate asset to parallel your strategic decision making process.

 

Copyright © 2010 http://jtonedm.com James Taylor

Syndicated from ebizQ