Why I’m Teaching Twitch to Predict the Future

If you devote 20 solid hours to predictions over the next 6 months; I’m 70% confident you’ll evaluate yourself as being 10% better at predicting the future when you review your predictions.

Think about what 10% more foresight is worth to you, wherever you need it most. My confidence comes from

If you devote 20 solid hours to predictions over the next 6 months; I’m 70% confident you’ll evaluate yourself as being 10% better at predicting the future when you review your predictions.

Think about what 10% more foresight is worth to you, wherever you need it most. My confidence comes from

1. Tetlock demonstrating a 10% improvement in forecasters that spent an hour reading these instructions in a controlled experiment.[1]

2. 20 hours of effort is sufficient to get past the frustration of gross incompetence, see meaningful improvements through focused practice, and evaluate if continued investment is worthwhile.

3. My experience building a culture of predictions at Twitch this year. Everyone who has put 5 or more hours in perceives 5–10% improvement when they review their predictions.

Predicting the future is not a theoretical superpower. It is a skill you are already using to make decisions, and like any other skill, it can be rapidly improved with deliberate practice. Predictions provide the ideal feedback loop to learn from reality and the most concise language to communicate expectations. The improved communication is probably even more valuable than the increased foresight for very large organizations.

Much of this evidence only became widely available last year with the release of Superforecasting. You should start making predictions and bring the practice to your organization. Perhaps you’ll join the ranks of the best forecasters, who regularly beat the wisdom of the crowds by 40%. Welcome to the Foresight Revolution.

Predictions are a Probability a Goal will be Observed under a Time Constraint

To build a prediction you start with something everyone is familiar with, a goal.

I hope if you devote 20 solid hours to predictions you’ll perceive yourself as being 10% better at predicting the future

It’s much more clear how you should treat my expectations (earlier prediction) than my hopes (this goal). The goal leaves it unclear if this is a moonshot or basically a sure thing. And without a time constraint, there is no guarantee you’ll be able to observe the goal above.

I expect I can help you significantly in months not minutes or decades. A quantitative statement of confidence is absolutely necessary. Qualitative statements of confidence will constantly be misinterpreted and expose you to hindsight bias.

A Serious Possibility Doesn’t Mean Anything, Embrace Numbers

A classic objection is that numerical probabilities imply too much rigor for estimates made without the benefit of substantial evidence and time. More than a dozen Intelligence experts collaborated on a report with the following key judgment.

An attack on Yugoslavia in 1951 should be considered a serious possibility [2]

What odds do you think the author had in mind? Take a moment. Sherman Kent, the father of intelligence analysis, asked around. Have something? Ok, you can move on.

The lead on the report believed the odds were 65/35 in favor of the attack. Interpretations of this report went all the way from 20/80 to 80/20. The report utterly failed to communicate its primary conclusion, the risk of an attack. For people like the report’s authors, a numerical probability felt too formal and scientific, so they avoided precision. This sort of mistake kept Sherman from bringing more foresight to the CIA.

Successful organizations will understand that actually communicating is worth getting over the hump. They will bump projects because of clear expectations- not hopes, politics, or blind confidence.

Numerical predictions are concise, precise, and explicitly model risk. Predictions are the right way to communicate your motivation and expectations in an uncertain world.

Predictions are the Engine of Science

Most contemporary scientists agree that Science boils down to this [3]

1. Make predictions from assumptions

2. Compare observations with predictions, update assumptions

3. Repeat

The most compact explanation of what I’m trying to do is bring the workhorse of Science to the riskiest and most critical decisions in an organization. Substitute assumptions for “supporting evidence” above and you are ready to leave the world of theory and tackle arbitrarily messy problems. An organization that communicates using predictions harnesses the core engine of science to its advantage. The opportunity for organizations to make better use of evidence is huge.

Worried your org is too data driven, or not data driven enough? Predictions summarize everyone’s data and intuition quantitatively. Predictions will help you appropriately balance your treatment of intuition and data as evidence. A track record of predictions can clearly demonstrate an individual’s intuition as strong quantitative evidence.

In a World

Join me in imagining a world. It’s day 1 for a new member of your organization. They start reading the predictions by your leaders and their teams for the past 2 years. They compare the observed outcomes to those predictions and see foresight. The kind of trust it’s taken years to build in the past is formed.

Intuition and evidence are woven together into predictions to set priorities. Expectations are pushed up from the teams working on projects instead of being pushed down from the executives. Reflection is easier (less hindsight bias). Leaders are more honest (reduced incentive to be overconfident). This is the world of Flash Forecasting.

I believe there is an 80% chance Twitch is this world in 2 years. In this world, the entire organization has leveled up because of the intrinsic motivation to forecast (improved foresight, communication, and reflection). Twitch won’t be alone. Pinterest, GiveWell, Open Phil, and CFAR are all exploring forecasting.

Ok, you are convinced. You are ready to build predictions into your company’s culture, but you don’t know the next step to take. I’ve developed a cheap and straightforward way to bring the benefits of forecasting to your org. I lay out step by step instructions in my post, Flash Forecasting. You’ll probably still have questions. I love all things forecasting, so I’m also down to help you personally. Let me know in the comments!

References

1. Superforecasting:The Art and Science of Predicting by Phillip Tetlock

2. Words of Estimative Probability by Sherman Kent

3. Our Mathematical Universe by Max Tegmark