Tetlock challenges the tendency to attribute broad economic or organizational outcomes to the leader in charge. People want a captain to hold accountable, but the number of levers any individual leader controls may be quite limited.

Leadership attribution often contains superstitious reasoning
Presidents have limited control over many economic outcomes
People are deeply inclined to hold someone accountable
Outcomes occurring on a leader's watch are not necessarily caused by that leader

“I think there's a great deal of superstitious reasoning about the impact that that that leaders have on organizational performance and also the impact that…”
Philip Tetlock · 06:11

“The number of leaders that presidents have to influence economic outcomes is quite limited um, but there is this deep intuition we have that leaders…”
Philip Tetlock · 06:21

#leadership#presidents#attribution#economics

Myth Buster12:33

Credentials and Wealth Do Not Prove Forecasting Skill

Audiences commonly use titles, prizes, wealth, and media access as proxies for expertise. Tetlock says the relationship between these status cues and actual predictive accuracy is often unknown, so credibility should come from tracked performance instead.

Audiences want answers to consequential economic and political questions
Experts arrive with impressive but indirect status signals
The connection between achievement and forecasting accuracy may be unknown
Recognizing reliance on proxy cues improves long-term judgment

“But the relationship between having done X, Y or Z and accuracy is unknown.”
Philip Tetlock · 13:42

“And the more honest we are about our ignorance, the more honest we are about when we're using proxy cues for judging how credible source…”
Philip Tetlock · 13:49

#expertise#credentials#accuracy#status

Myth Buster1:10:15

Not Every After-the-Fact Explanation Is Hindsight Theater

Tetlock distinguishes casual narratives from disciplined postmortems. Investigations in aviation can produce real safety gains because they are constrained by scientific knowledge, physical evidence, and formal procedures, unlike improvised market stories that merely explain an outcome after it occurs.

Prediction and retrospective causal analysis are different tasks
Aircraft investigations use evidence and established procedures
Disciplined postmortems can change rules and designs
Unconstrained narratives risk capitalizing on chance
The key question is whether a domain supports genuine learning

“There are different types of postmortems, and some of them are constrained by well defined bodies of scientific knowledge and investigative procedure that reduced the…”
Philip Tetlock · 1:11:14

“So the big question for us is when is learning possible? When can we learn to do certain categories of things better? And when are…”
Philip Tetlock · 1:12:21

#postmortems#hindsight#aviation#learning

Hot Take· 1

Hot Take20:12

The Worst Forecasters Can Win the Most Attention

Tetlock's research found that hedgehog-style forecasters produced worse overall records but attracted greater media prominence. Extreme predictions create memorable occasional successes and compelling sound bites, while more accurate probabilistic forecasters often sound tentative and dull.

Extreme forecasters may accumulate occasional home runs by chance
Media platforms reward bold and simple narratives
Moderate forecasts make fewer spectacular claims
Entertainment value and predictive value should be evaluated separately

“We also found that the hedgehogs tended to be more prominent, They're more attractive to the media.”
Philip Tetlock · 20:20

“you get on somewhat ironic situation that the worst forecasters have the greatest media prominence.”
Philip Tetlock · 20:52

#media#incentives#sensationalism#experts

Explainer· 5

Explainer04:38

Why We Demand Forecasts Even When Good Forecasts Are Scarce

Tetlock argues that people crave structure and guidance because accepting radical uncertainty feels nihilistic and psychologically dissonant. This creates demand for pundits even when their demonstrated forecasting ability is weak.

People resist the idea that major policy choices may have unpredictable consequences
Predictive guidance supplies psychological structure
Demand for certainty can persist despite a limited supply of accurate forecasts
The appetite resembles historical reliance on psychics and other sources of guidance

“People don't want to live that They want structure.”
Philip Tetlock · 05:14

“There's a demand for it even when the supply of good forecasting is extremely limited.”
Philip Tetlock · 05:29

#punditry#uncertainty#psychology#forecasting

Explainer15:21

How Pundits Stay Credible After Being Wrong

A pundit must sound surprising and informed while preserving an escape route if the forecast fails. Vague phrases such as “distinct possibility” solve that career problem because they can be reinterpreted as either a high or low probability after the outcome.

Media pundits are rewarded for dramatic and novel claims
Long-term credibility requires protection from failed predictions
Elastic probability words provide post-hoc escape routes
Self-serving interpretation can prevent pundits from recognizing their own errors

“I need to make it sound as though I know what I'm talking about.”
Philip Tetlock · 15:25

“Now distinct possibility is one of those wonderful phrases, because if it happens, I can say, hey, I told you there was a distinct possibility.…”
Philip Tetlock · 16:18

#punditry#accountability#language#bias

Explainer43:44

Why Tail Risks Go From Ignored to Overrated

Tetlock describes a recurring imbalance in how people process catastrophic possibilities. They often neglect a tail risk before it becomes salient, then overweight it once a vivid scenario or recent crisis brings it to mind.

Unnoticed tail risks tend to be underrated
Vividly presented tail risks tend to be overrated
Recent crises encourage people to predict repetitions of the last disaster
Effective risk judgment requires balancing neglect against salience

“most of the time people underrate them, but as soon as the scenario has called to their attention, they overrated.”
Philip Tetlock · 43:47

“These folks are still fighting the previous financial crisis and the diplomats try to avoid the last war.”
Barry Ritholtz · 44:43

#tail-risk#recency#risk#behavioral-finance

Explainer58:29

Forecasting Tournaments Bring Moneyball to Intelligence

Tetlock describes intelligence forecasting tournaments as a way to quantify judgments that were previously expressed through vague language. The aim is not to eliminate qualitative experts but to combine and evaluate their inputs using scoring, crowdsourcing, training, and algorithms.

Intelligence estimates can be expressed as probability ranges
Crowd averages often outperform most contributing individuals
Qualitative subject-matter knowledge remains an important input
Quantitative tools can combine human judgments more effectively
Systematic talent spotting depends on tracking scores

“I think that's a great way to describe it. Just money ballfing the intelligence community.”
Philip Tetlock · 1:00:49

“We're never going to do away with people who have deep qualitative insights into the subject matter. There are crucial source of inputs.”
Philip Tetlock · 1:01:07

#intelligence#moneyball#crowdsourcing#analytics

Explainer1:16:34

The Next Frontier Is Human-Machine Forecasting

Tetlock identifies hybrid forecasting as a major research frontier. Competitions can compare humans, statistical models, and combinations of the two to discover which arrangement works best under different conditions.

Machines and humans have different forecasting strengths
The best arrangement may depend on the question and environment
Competitions can test model-only, human-only, and hybrid approaches
Generating better policy questions is as important as forecasting answers
High-quality explanations can also be evaluated and crowdsourced

“The other is another competition which is called h FC, the Hybrid Forecasting Competition, which will be UM, humans and machines and human machine combinations,…”
Philip Tetlock · 1:16:34

“the one truth is we really don't know the answers to these questions right now, and we're hoping to learn more.”
Philip Tetlock · 1:17:08

#ai#human-machine#hybrid-forecasting#research

Story· 1

Story1:08:09

The Soviet Debate Exposed Perfect Hindsight and Zero Foresight

Tetlock recounts serving on a committee where liberals and conservatives held strong but opposing views about the Soviet Union. Neither side anticipated Gorbachev's transformative role, yet both could readily explain the outcome afterward, motivating Tetlock to seek a disciplined way to keep score.

Both ideological camps held confident views about the Soviet future
Neither camp successfully anticipated Gorbachev's transformation
Both sides generated persuasive explanations after the event
The mismatch between prediction and hindsight motivated early forecasting research

“Both sides could readily explain after the fact what happened. UM. So it was this mismatch between virtually zero predictab ability and virtually perfect expost…”
Philip Tetlock · 1:08:45

“It was a way to try, what can we do to to keep score and and if we do keep score, can we identify um…”
Philip Tetlock · 1:09:11

#soviet-union#hindsight#ideology#origin-story

Takeaway· 2

Takeaway27:46

Superforecaster Teams Beat Markets and Classified Analysts

The Good Judgment Project's strongest teams performed better than ordinary teams, prediction markets, and intelligence analysts with access to classified information. Tetlock emphasizes that this result surprised researchers who expected independent judgments to outperform collaboration.

Ordinary forecasting teams improved on the broader crowd
Prediction markets beat ordinary teams
Teams of top forecasters surpassed prediction markets
The strongest teams also outperformed analysts with classified information
The intelligence community deserves credit for funding a potentially embarrassing test

“the teams have super forecasters truly astonished us because the statisticians were telling us the right thing to do here was to have each individual…”
Philip Tetlock · 27:46

“They weren't just outperforming prediction markets in the public sphere, they are also outperforming intelligence analysts who were working, you know, behind a veil of…”
Philip Tetlock · 29:24

#superforecasters#teams#prediction-markets#intelligence

Takeaway47:31

Forecasting Skill Fades Rapidly Beyond About One Year

Tetlock is optimistic about improving forecasts over shorter horizons but increasingly pessimistic at three, five, or ten years. In complex geopolitical and economic systems, accumulating contingencies progressively shuffle away useful foresight, although some unusually constrained questions remain predictable for longer.

Selection, training, and teaming can improve forecasts up to roughly one year
Longer horizons accumulate more unpredictable contingencies
Predictability varies by question category
A compelling distant-future story should not be mistaken for a likely trajectory

“I'm much more optimistic that we can improve forecasting using the right selection, training, teaming tools in shorter time periods up to about a year.”
Philip Tetlock · 47:36

“I become progressively more pessimistic when you go out to the longer reaches of three, five, ten years that were included in Expert Political Judgment,…”
Philip Tetlock · 47:49

#time-horizon#complexity#long-term#prediction

Frameworks in this episode

Anonymous Iterative Team Forecasting

Calibrated Probabilistic Forecasting

Error Balancing

Fermi Decomposition

Fox-Style Multi-Lens Forecasting

Insights & moments

⚡Myth Buster· 3

Presidents and CEOs Control Less Than We Imagine