Cookies help us display personalized product recommendations and ensure you have great shopping experience.

By using this site, you agree to the Privacy Policy and Terms of Use.
Accept
SmartData CollectiveSmartData Collective
  • Analytics
    AnalyticsShow More
    data analytics and truck accident claims
    How Data Analytics Reduces Truck Accidents and Speeds Up Claims
    7 Min Read
    predictive analytics for interior designers
    Interior Designers Boost Profits with Predictive Analytics
    8 Min Read
    image fx (67)
    Improving LinkedIn Ad Strategies with Data Analytics
    9 Min Read
    big data and remote work
    Data Helps Speech-Language Pathologists Deliver Better Results
    6 Min Read
    data driven insights
    How Data-Driven Insights Are Addressing Gaps in Patient Communication and Equity
    8 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-25 SmartData Collective. All Rights Reserved.
Reading: How They Fit Together: Bell Curves, Bayesian Inference and Black Swans
Share
Notification
Font ResizerAa
SmartData CollectiveSmartData Collective
Font ResizerAa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Data Management > Risk Management > How They Fit Together: Bell Curves, Bayesian Inference and Black Swans
CommentaryExclusiveRisk ManagementStatistics

How They Fit Together: Bell Curves, Bayesian Inference and Black Swans

paulbarsch
paulbarsch
5 Min Read
SHARE

Probability is defined as the possibility, chance or odds of likelihood that a certain event or occurrence will take place now or in the future.  In a world where business managers like to “know the odds”, how does probabilistic thinking (Frequentism and Bayesian) mesh with extreme events (i.e. Black Swans) that just cannot be predicted?

Probability is defined as the possibility, chance or odds of likelihood that a certain event or occurrence will take place now or in the future.  In a world where business managers like to “know the odds”, how does probabilistic thinking (Frequentism and Bayesian) mesh with extreme events (i.e. Black Swans) that just cannot be predicted?

Statisticians lament how few business managers think probabilistically. In a world awash with data, statisticians claim there are few reasons to not have a decent amount of objective data for decision making. However, there are some events for which there are no data (they haven’t occurred yet), and there are other events that could happen outside the scope of what we think is possible.

More Read

My ‘Data’ Is Hot!
How Digital Experience Monitoring and Customer-Centric IT Makes Everyone Happy
Python for Business: Optimize Pre-Processing Data for Decision-Making
Here’s Why a Bootcamp Won’t Make You a Data Scientist
The Role of Cookies in Analytics and Privacy Issues they Raise

The best quote to sum up this framework for decision making comes from the former US Defense secretary Donald Rumsfeld in February 2002:

“There are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns – there are things we do not know we don’t know.”

Breaking this statement down, it appears Mr. Rumsfeld is speaking about Frequentism, subjective probability (Bayes) and those rare but extreme events coined by Nassim Taleb as “Black Swans”.

Author Sharon Bertsch McGrayne elucidates the first two types of probabilistic reasoning in her book “The Theory That Would Not Die”.  Frequentism (conventional statistics), she says, relies on measuring the relative frequency of an event that can be repeated time and again under the same conditions. This is the world of p-values, bell curves, coin flips, casinos and actuaries where data driven decision making is objective based on sampling or computations of large data sets.

The greater part of McGrayne’s tome concentrates on defining Bayesian Inference, or subjective probability also known as a “measure of belief”. Bayes, she says, allows making of predications with no prior information at all (no frequency of events).With Bayes, one makes an educated guess, and then keeps refining that guess based on new information, thus updating and revising the probabilities, thus getting “closer to certitude.”

Getting back to Rumsfeld’s quote, Rumsfeld seems to be saying we can guess the probability of the “known knowns” because they’ve happened before and we have frequent data to support objective reasoning. These “known knowns” are Nassim Taleb’s White Swans. There are also “known unknowns” or things that have never happened before, but have entered our imaginations as possible events (Taleb’s Grey Swans). We still need probability to discern “the odds” of that event (e.g. dirty nuclear bomb in Los Angeles), so Bayes is helpful because we can infer subjective probabilities or “the possible value of unknowns” from similar situations tangential to our own predicament.  

Lastly, there are “unknown unknowns”, or things we haven’t even dreamed about (Taleb’s Black Swan).  Dr. Nassim Nicholas Taleb labels this “the fourth quadrant” where probability theory has no answers.  What’s an illustration of an “unknown unknown”? Dr. Taleb gives us an example of the invention of the wheel, because no one had even though or dreamed of a wheel until it was actually invented. The “unknown unknown” is unpredictable, because—like the wheel—had it been conceived by someone, it would have been already invented.

Rumsfeld’s quote gives business managers a framework for thinking probabilistically. There are “known knowns” for which Frequentism works best, “unknown knowns” for which Bayesian Inference is the best fit, and there is a realm of “unknown unknowns” where statistics falls short, where there can be no predictions. This area outside the boundary of statistics is the most dangerous area, says Dr. Taleb, because extreme events in this sector usually carry large impacts.

This column has been an attempt to provide a decision making framework for how Frequentism, Bayes and Black Swans fit together—by using Donald Rumsfeld’s quote. 

What say you, can you improve upon this framework?

TAGGED:Bayes Theorembell curveblack swanTaleb
Share This Article
Facebook Pinterest LinkedIn
Share

Follow us on Facebook

Latest News

data analytics and truck accident claims
How Data Analytics Reduces Truck Accidents and Speeds Up Claims
Analytics Big Data Exclusive
predictive analytics for interior designers
Interior Designers Boost Profits with Predictive Analytics
Analytics Exclusive Predictive Analytics
big data and cybercrime
Stopping Lateral Movement in a Data-Heavy, Edge-First World
Big Data Exclusive
AI and data mining
What the Rise of AI Web Scrapers Means for Data Teams
Artificial Intelligence Big Data Exclusive

Stay Connected

1.2kFollowersLike
33.7kFollowersFollow
222FollowersPin

You Might also Like

Fight Back Against Black Swan Fatigue

5 Min Read

Is the Mother of All Black Swans Coming?

4 Min Read

Of Baby Black Swans and the Race to Zero

5 Min Read

The Human Factor Continually Confounds Probability Models

3 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

data-driven web design
5 Great Tips for Using Data Analytics for Website UX
Big Data
giveaway chatbots
How To Get An Award Winning Giveaway Bot
Big Data Chatbots Exclusive

Quick Link

  • About
  • Contact
  • Privacy
Follow US
© 2008-25 SmartData Collective. All Rights Reserved.
Go to mobile version
Welcome Back!

Sign in to your account

Username or Email Address
Password

Lost your password?