Is Campaign Measurement slowly becoming a farce?

I along with friends in other Analytics companies, work with a number of well known, huge retail chains. And campaign (especially Direct Mailers) design and measurement is one of the most regular type of projects we do. There is a common theme across all these retailers – they will ask for something quick (make it extra, unreasonably quick!) when it comes to the target audience selection, and when the time for measurement comes up they will ask us to do a number of things that will eventually lead to the numbers or results they want to see.

I along with friends in other Analytics companies, work with a number of well known, huge retail chains. And campaign (especially Direct Mailers) design and measurement is one of the most regular type of projects we do. There is a common theme across all these retailers – they will ask for something quick (make it extra, unreasonably quick!) when it comes to the target audience selection, and when the time for measurement comes up they will ask us to do a number of things that will eventually lead to the numbers or results they want to see.

Below are a few of their requests or their logic for doing things in a certain way:

1. Build a response model on a previous similar campaign (using logictic regression or decision tree) within a week.

2. Select customers based on a criteria. From this population, you randomly sample and assign customers in 2 groups – Test and Control. Now, why in the world would want do a 1 sample test on this? What is the logic behind using numbers from the Control group as the population numbers?

3. Why should someone treat outliers (read as very high positive RFM values) in the Control group only? The client’s logic is very simple – when customers who haven’t been contacted shop like crazy, they are clearly outliers. But when customers in the test group do the same thing, it’s all because of the message or campaign!

We all do our best to educate them on the techniques or question any flaws in their logic but not all clients are the same. Not everyone accepts mistakes or appreciates feedback. And a few of them are so stuck with their qualifications and designations that questioning their request or logic is simply impossible.

How do you talk to these clients? How do you say that something is simply wrong or inapprorpiate?