We use cookies, including third-party cookies from Google to serve personalized ads through AdSense, to operate this site and understand how it is used. By continuing to browse, you accept this use. See our Privacy Policy and Terms of Use for details, including how to opt out of personalized advertising.
Accept
SmartData CollectiveSmartData Collective
  • Analytics
    AnalyticsShow More
    Flat editorial illustration: The article is about joining public web signals with internal business metrics so BI teams can inter
    Alternative Data in BI: Bringing Public Web Data Into Your Dashboards
    23 Min Read
    Flat editorial illustration: The article is about automating the collection, preparation, and delivery of marketing data across p
    From Data to Dashboards: 5 Marketing Reporting Automation Software Picks
    12 Min Read
    Flat editorial illustration: The article contrasts passive follower totals with a behavioral path from attention and trust to cli
    Creator Analytics Beyond Follower Count: Seven Metrics That Measure Audience Behavior
    11 Min Read
    Flat editorial illustration: The article connects AI and marketing analytics with managerial judgment grounded in consumer behavi
    How an Online MBA in Marketing Connects AI and Data Analytics
    11 Min Read
    What Kind of Problem-Solving Distinguishes Data Analysts From Software Engineers -- AI-generated illustration
    What Kind of Problem-Solving Distinguishes Data Analysts From Software Engineers
    7 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-25 SmartData Collective. All Rights Reserved.
Reading: This Is Why You Shouldn’t Weight Your Survey Data
Share
Notification
Font ResizerAa
SmartData CollectiveSmartData Collective
Font ResizerAa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Uncategorized > This Is Why You Shouldn’t Weight Your Survey Data
Uncategorized

This Is Why You Shouldn’t Weight Your Survey Data

AnniePettit
AnniePettit
4 Min Read
This Is Why You Shouldn’t Weight Your Survey Data
Illustration generated with Qwen Image.
SHARE

For a long time I’ve been a big proponent of weighting data as little and as rarely as possible. You see, I would rather fill cells to achieve the desired sample size rather than fake cells based on data I’ve already gathered – not on data that I failed to gather.

For a long time I’ve been a big proponent of weighting data as little and as rarely as possible. You see, I would rather fill cells to achieve the desired sample size rather than fake cells based on data I’ve already gathered – not on data that I failed to gather. Obviously, I failed to gather that data for a specific reason, perhaps one that is completely unknown and unguessable to me.

Let’s look at an example. I asked Excel to randomly generate a score from 1 to 10 for fifty men and fifty women. Then, I copied those scores four times. In the first copy, I deleted the last ten scores of women. In the second copy, I deleted the last twenty scores of women, and so on. What I was left with was five datasets based on the exact same population, but with various percentages of missing data. Why was the data missing? I don’t know. Maybe the introverts didn’t answer the door/pick up the phone/talk to me at the mall. Maybe the high income people were all away from home and at their cottages that weekend. Maybe my interviewer drove up in a beat-up, rusty Pinto and scared people off. Regardless, data analysis must begin!

In the Perfect Sample, we see the results from 50 men and 50 women wherein the weights for both men and women are 1, and both the unweighted and weighted total scores are 5.5. But notice what happens when data from more and more women become missing, and the weights for women must compensate and become higher and higher. Although we know that the “true” score is 5.5, the Unweighted Total Score eventually becomes 4.8 and is made even worse with the Weighted Total Score which eventually becomes 4.5 in Scenario 4. In other words, although weighting the data ensured that women’s opinions were counted just as much as men’s opinions, it didn’t improve the quality of the data. It simply exaggerated a flawed sampling technique.

 Random SampleScenario 2Scenario 3Scenario 3Scenario 4
Unweighted Total Score5.55.35.14.94.8
Number of men5050505050
Number of women5045403530
Weight for men1.01.01.01.01.0
Weight for women1.01.111.251.431.67
Weighted Total Score5.55.35.04.84.5

The moral of the story is this.

Never use weighting to adjust or correct for missing data.

Fill every cell, and then use weighting sparingly to adjust for extremely minor differences that resulted from oversampling (not undersampling).

This message has been brought to you by the letter α and the number 27644437.

 

Share This Article
Facebook Pinterest LinkedIn
Share

Follow us on Facebook

Latest News

Flat editorial illustration: The article compares proxy MCP servers by what an AI agent can do through them: manage proxy account
Proxy MCP Servers: 7 Options and What Each Lets an AI Agent Do
Artificial Intelligence Big Data Data Collection
Flat editorial illustration: The article is about joining public web signals with internal business metrics so BI teams can inter
Alternative Data in BI: Bringing Public Web Data Into Your Dashboards
Big Data Business Intelligence Exclusive Web Analytics
Flat editorial illustration: A SaaS-editorial scene showing product usage, support patterns, and renewal feedback informing a fle
Using Product Usage Analytics and Support Data to Build a B2B Product Roadmap
Business Intelligence Infographic
From Hiring To Payroll: Managing HR Data Across Business Systems -- AI-generated illustration
From Hiring To Payroll: Managing HR Data Across Business Systems
Big Data Business Intelligence Infographic

Stay Connected

1.2KFollowersLike
33.7KFollowersFollow
222FollowersPin

You Might also Like

eMail is Dead, Long Live Social Networking: Don’t Get Left Behind
Uncategorized

eMail is Dead, Long Live Social Networking: Don’t Get Left Behind

5 Min Read
The Rise of the Intercloud
Uncategorized

The Rise of the Intercloud

5 Min Read
Actian DataFlow, the Little Hadoop Engine That Could, But Probably Won’t
Uncategorized

Actian DataFlow, the Little Hadoop Engine That Could, But Probably Won’t

11 Min Read
Understanding Influence; the Chris Brogan Effect
Uncategorized

Understanding Influence; the Chris Brogan Effect

6 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

Chatbots and SEO: How Can Chatbots Improve Your SEO Ranking?
Chatbots and SEO: How Can Chatbots Improve Your SEO Ranking?
Artificial Intelligence Chatbots Exclusive
ai chatbot
How AI Website Chatbots Improve Customer Support and Lead Generation
Chatbots Exclusive

Quick Link

  • About
  • Contact
  • Privacy
Follow US
© 2008-26 SmartData Collective. All Rights Reserved.
Welcome Back!

Sign in to your account

Username or Email Address
Password

Lost your password?