We use cookies, including third-party cookies from Google to serve personalized ads through AdSense, to operate this site and understand how it is used. By continuing to browse, you accept this use. See our Privacy Policy and Terms of Use for details, including how to opt out of personalized advertising.
Accept
SmartData CollectiveSmartData Collective
  • Analytics
    AnalyticsShow More
    Flat editorial illustration: The article is about joining public web signals with internal business metrics so BI teams can inter
    Alternative Data in BI: Bringing Public Web Data Into Your Dashboards
    23 Min Read
    Flat editorial illustration: The article is about automating the collection, preparation, and delivery of marketing data across p
    From Data to Dashboards: 5 Marketing Reporting Automation Software Picks
    12 Min Read
    Flat editorial illustration: The article contrasts passive follower totals with a behavioral path from attention and trust to cli
    Creator Analytics Beyond Follower Count: Seven Metrics That Measure Audience Behavior
    11 Min Read
    Flat editorial illustration: The article connects AI and marketing analytics with managerial judgment grounded in consumer behavi
    How an Online MBA in Marketing Connects AI and Data Analytics
    11 Min Read
    What Kind of Problem-Solving Distinguishes Data Analysts From Software Engineers -- AI-generated illustration
    What Kind of Problem-Solving Distinguishes Data Analysts From Software Engineers
    7 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-25 SmartData Collective. All Rights Reserved.
Reading: A/B Testing and the Need for Clear Business Objectives
Share
Notification
Font ResizerAa
SmartData CollectiveSmartData Collective
Font ResizerAa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Big Data > Data Mining > A/B Testing and the Need for Clear Business Objectives
Data Mining

A/B Testing and the Need for Clear Business Objectives

DeanAbbott
DeanAbbott
3 Min Read
A/B Testing and the Need for Clear Business Objectives
Photo by LyraBelacqua-Sally on Pixabay (https://pixabay.com/photos/nikon-camera-objective-lens-1196113/)
SHARE

The website http://videolectures.net/ contains a wealth of interesting lectures on a wide variety of topics, including data mining. I was reminded of one today by Ronny Kohavi entitled  “Practical Guide to Controlled Experiments on the Web: Listen to Your Customers not to the HiPPO” It’s short (only 23 minutes) and filled with some very good common-sense principles.

First, it is a talk about the importance of A/B testing, or in other words, constructing experiments to learn customer behavior rather than having the experts make a best guess at how people will behave. He gives some good examples from Microsoft and the sometimes non-intuitive results from actual testing. A book he recommends is Breakthrough Business Results With MVT: A Fast, Cost-Free, Secret Weapon for Boosting Sales, Cutting Expenses, and Improving Any Business Process


The second part of the lecture I found particularly interesting is what Kohavi calls the Overall Evaluation Criterion (OEC), or what I usually call business objectives. He included the great Lewis Carroll quote, “If you don’t know where you are going, any road will take you there.” I find this a common problem as well: if we don’t define a business objective that truly measures the impact of the predictive models we build, we have no way of determining if they are effective or not.  This objective must be tied to the business itself. For example, Kohavi argues for using Customer Lifetime Value (CLV) rather than click-through rates as they are more tied to the bottom line.

I would add that it can be useful to have two objectives that are measurable, especially if two objectives better measure the value. For example, in collections risk models, the age of the debt and the amount of the debt are both important components to risk. These are difficult to put into a single number in general, so the two-dimensional risk score can be helpful operationally.

More Read

72% of People Aren’t Familiar with Hosted VoIP
72% of People Aren’t Familiar with Hosted VoIP
Data Mining in the New Economy and How to Get Started
Decision Management and software development I – Agile
PAW: Five Ways to Lower Costs with Predictive Analytics
How Habitat UK *should* have used Twitter
TAGGED:a/b testing
Share This Article
Facebook Pinterest LinkedIn
Share

Follow us on Facebook

Latest News

Flat editorial illustration: The article connects downstream report failures, data-quality gaps, and conflicting metric definitio
Signs Your Data Architecture Needs an Outside Review
Big Data Exclusive
Data-Driven Pultrusion: Best GFRP Rebar Machines & ROI
Data-Driven Pultrusion: Best GFRP Rebar Machines & ROI
Big Data Exclusive
Flat editorial illustration: The article contrasts tools for gathering fresh interviews with tools for analyzing existing qualita
10 AI Market Research Tools That Turn Qualitative Data into Insights
Artificial Intelligence Big Data Exclusive
Flat editorial illustration: The article compares proxy MCP servers by what an AI agent can do through them: manage proxy account
Proxy MCP Servers: 7 Options and What Each Lets an AI Agent Do
Artificial Intelligence Big Data Data Collection

Stay Connected

1.2KFollowersLike
33.7KFollowersFollow
222FollowersPin

You Might also Like

A/B Testing Basics: What You Need to Know
Best PracticesStatistics

A/B Testing Basics: What You Need to Know

5 Min Read
Conversion Rates
Big DataMarketingStatisticsWeb Analytics

6 Steps to Use Big Data to Improve Conversion Rates

8 Min Read
Know Your S#*!: Maximize Web Conversion with A/B Testing
CRMExclusiveInside CompaniesMarketingPredictive Analytics

Know Your S#*!: Maximize Web Conversion with A/B Testing

8 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

AI chatbots
AI Chatbots Can Help Retailers Convert Live Broadcast Viewers into Sales!
Chatbots
Chatbots and SEO: How Can Chatbots Improve Your SEO Ranking?
Chatbots and SEO: How Can Chatbots Improve Your SEO Ranking?
Artificial Intelligence Chatbots Exclusive

Quick Link

  • About
  • Contact
  • Privacy
Follow US
© 2008-26 SmartData Collective. All Rights Reserved.
Welcome Back!

Sign in to your account

Username or Email Address
Password

Lost your password?