Error Balancing
Update enough to learn from news without chasing noise or becoming rigid
- Difficulty
- Moderate
- Time to result
- ~weeks to results
- Steps
- 5
- Confidence
- 97%
Error balancing treats judgment as navigation between two opposing failure modes. Excessive volatility means reacting to every new development as though it transforms the underlying situation; excessive rigidity means protecting an established view even when diagnostic evidence accumulates against it. The method begins by separating newsworthiness from informational value, then asks how much the evidence should rationally alter the current probability or decision. The update is recorded so later outcomes can reveal whether the response was too large, too small, or appropriately proportional. Tetlock compares this skill to riding a bicycle because verbal understanding is insufficient: balance emerges through repeated forecasts, feedback, and correction. The objective is not to avoid all errors but to reduce the combined cost of errors on both sides.
Origin
Extracted from Masters in Business, where Philip Tetlock connected superforecasting practice with balancing excessive volatility against excessive rigidity.
Core principles
- 01Every adjustment policy creates opposing error risks
- 02New information varies in diagnostic value
- 03Frequent movement can be as harmful as refusal to change
- 04Good updating resembles balance learned through practice
- 05Feedback is necessary to calibrate responsiveness
How to run it
- 1
Anchor the current view
Record the existing probability, position, or decision and the evidence supporting it. This creates a baseline against which any update can be measured.
Pro tip State in advance what evidence would materially alter the view.
Watch out An undocumented prior makes it easy to exaggerate how responsive or consistent you were.
- 2
Assess diagnostic value
Ask whether the new information changes the causal outlook, base rate, or likelihood of the event. Separate emotionally vivid news from evidence that genuinely discriminates among outcomes.
Pro tip Compare the news with similar signals from past cases.
Watch out Recency and salience can make weak evidence feel decisive.
- 3
Test both error directions
Ask what would happen if you overreacted and what would happen if you ignored the evidence. Consider the probability and cost of each mistake.
Pro tip Name the two errors explicitly before choosing an adjustment.
Watch out Teams often scrutinize only the error that fits their current narrative.
- 4
Make a proportional update
Change the estimate or action only as much as the evidence warrants. Preserve the prior view when the signal is mostly noise, but move decisively when evidence is strong.
Pro tip Use probability increments to make the magnitude of the update visible.
Watch out Splitting the difference mechanically is not the same as balancing evidence.
- 5
Learn from resolution
After the outcome, review whether updates were systematically too large, too small, or too late. Adjust the updating policy and practice again.
Pro tip Look for recurring patterns across many decisions rather than one memorable case.
Watch out Outcome knowledge can distort recollection, so rely on the contemporaneous record.
In the wild
An investor sees a dramatic economic release but checks whether it changes long-term earnings, liquidity, or recession probabilities. Instead of immediately rotating the entire portfolio or ignoring the release, the investor makes a modest probability update and records the rationale.
→ The portfolio responds to meaningful information without being churned by every headline.
A forecaster receives evidence that negotiations have stalled. The evidence raises conflict risk, but previous stalls often resolved, so the forecast moves from 25 to 34 percent rather than jumping to certainty or remaining unchanged.
→ The estimate incorporates the news while preserving relevant historical context.
Common mistakes
Jumping at every headline
Excessive volatility exaggerates the diagnostic value of small or noisy developments.
Defending the original view
Excessive rigidity turns consistency into a reason to ignore accumulating contrary evidence.
Learning only in theory
Error balancing requires repeated decisions and feedback; understanding the two errors conceptually is not sufficient.
Is it for you?
Best for
It is best for investors, forecasters, and leaders who repeatedly revise beliefs or positions as new information arrives.
Not ideal for
It is not ideal when evidence demands an immediate binary safety response or when no feedback is available to calibrate updating.
From the transcript
“One of them is the error of excess of volatility, of um jumping every time there's a little bit of news and exaggerating its diagnostic…”
“And the other big mistake you can make is excessive rigidity and being so committed to a particular preconception about where the future is going…”
“But the only way it's really going to sink into people's heads if they is if they go to forecasting tournaments and they actually practice…”
From the episode
Barry Ritholtz's Masters in Business: Philip Tetlock Interview
Philip Tetlock