The Mythbusters and Statistics

September 16, 2009
53 Views

I love the TV show “Mythbusters.” If you’ve never seen the show, it’s great. (A new season starts this week on the Discovery Channel in the US.) Every week, the team describes one or more myths or urban legends (example: a penny dropped from the top of the Empire State Building could kill a pedestrian), and then they attempt to confirm or “bust” the myth with experimentation. (Wind tunnel experiments showed that the terminal velocity of a penny isn’t fast enough to seriously injure someone.)

One thing that has mildly bugged me about the show is the lack of replication in the experiments: typically, they only ever attempt to replicate the myth once or maybe twice, and rarely discuss the variability in their measurements. So I was pleased to see the issue come up in this New Scientist interview:

You often have sample sizes of one or two, but science is all about replication. How do you respond to that criticism?

JH: People simply wouldn’t watch it if we were just repeating things over and over again. We do them as compactly as we can to keep up the energy level and flow. We intend these shows to be thought-provoking, not definitive.

AS: I think the part of the scientific enterprise

I love the TV show “Mythbusters.” If you’ve never seen the show, it’s great. (A new season starts this week on the Discovery Channel in the US.) Every week, the team describes one or more myths or urban legends (example: a penny dropped from the top of the Empire State Building could kill a pedestrian), and then they attempt to confirm or “bust” the myth with experimentation. (Wind tunnel experiments showed that the terminal velocity of a penny isn’t fast enough to seriously injure someone.)

One thing that has mildly bugged me about the show is the lack of replication in the experiments: typically, they only ever attempt to replicate the myth once or maybe twice, and rarely discuss the variability in their measurements. So I was pleased to see the issue come up in this New Scientist interview:

You often have sample sizes of one or two, but science is all about replication. How do you respond to that criticism?

JH: People simply wouldn’t watch it if we were just repeating things over and over again. We do them as compactly as we can to keep up the energy level and flow. We intend these shows to be thought-provoking, not definitive.

AS: I think the part of the scientific enterprise that we do illuminate is that it’s a messy, creative process that changes your whole understanding. We’ll spend half an episode finding that we’re asking the wrong question.

A fair response. I think in general they’re aware of the variability and significance issues, but I agree that more important contribution of the show is the experimental process: encouraging kids in particular to actually measure and compare things to answer questions. Issues like controls, replication, and significance are moot without data, after all. (Co-host Adam Savage talks more about the effect of the show on kids in this video interview from reason.com. Heartwarming stuff.)  

I was also pleased to see that they do seek the input of statisticians from time to time, too:

When you are testing your own reactions, might you bias your results because you have expectations about the outcome?

AS:
That’s a good point and makes me think that we should demonstrate
experimental bias on the show. It was an issue when we investigated
“beer goggles”: whether drinking alcohol can make people seem more
attractive. I spent a long time with a friend of mine who’s a
statistician to try and remove as much of the bias as possible.

New Scientist: MythBusters: ‘Using your head is a lot of fun’ 

Link to original post

You may be interested

How SAP Hana is Driving Big Data Startups
Big Data
298 shares3,039 views
Big Data
298 shares3,039 views

How SAP Hana is Driving Big Data Startups

Ryan Kh - July 20, 2017

The first version of SAP Hana was released in 2010, before Hadoop and other big data extraction tools were introduced.…

Data Erasing Software vs Physical Destruction: Sustainable Way of Data Deletion
Data Management
57 views
Data Management
57 views

Data Erasing Software vs Physical Destruction: Sustainable Way of Data Deletion

Manish Bhickta - July 20, 2017

Physical Data destruction techniques are efficient enough to destroy data, but they can never be considered eco-friendly. On the other…

10 Simple Rules for Creating a Good Data Management Plan
Data Management
69 shares654 views
Data Management
69 shares654 views

10 Simple Rules for Creating a Good Data Management Plan

GloriaKopp - July 20, 2017

Part of business planning is arranging how data will be used in the development of a project. This is why…