The Resulting Audit
Judge decisions by process quality, not by one visible outcome
- Difficulty
- Moderate
- Time to result
- ~weeks to results
- Steps
- 5
- Confidence
- 99%
The Resulting Audit separates decision quality from outcome quality. Before judging a choice, reconstruct what information was available, what alternatives existed, and what probabilities could reasonably have been assigned at the time. Then identify which parts of the result came from controllable execution and which came from luck. A counterfactual test—asking how the decision would be judged if the outcome had reversed—reveals whether hindsight is pulling the evaluation around. The aim is not to ignore outcomes, but to treat each one as noisy evidence rather than a verdict. Repeated reviews across a larger sample reveal whether the underlying process is consistently sound and where it can be improved.
Origin
Annie Duke explains the poker term “resulting” through Pete Carroll’s intercepted Super Bowl pass, showing how observers confused a disastrous result with proof of a disastrous decision.
Core principles
- 01A good decision can produce a bad outcome
- 02A bad decision can produce a good outcome
- 03Single outcomes contain too much noise to prove decision quality
- 04Decision quality becomes clearer across repeated observations
- 05Counterfactual outcomes expose outcome-driven judgments
How to run it
- 1
Freeze the original context
Document the information, constraints, goals, and alternatives that existed before the outcome became known.
Pro tip Maintain a decision journal so hindsight cannot rewrite what you originally knew.
Watch out Do not add facts that became available only after the decision.
- 2
Map possible outcomes
List the plausible futures the decision could have produced and estimate their probabilities.
Pro tip Use ranges when precise probabilities would create false confidence.
Watch out Do not treat the realized outcome as if it were the only possible one.
- 3
Separate process from luck
Identify which elements were controlled by the decision-maker and which arose from randomness or hidden information.
Pro tip Ask what could have been changed before the outcome occurred.
Watch out Do not label every loss unlucky or every win skillful.
- 4
Run the counterfactual test
Imagine the same decision producing a different plausible outcome, then see whether your judgment changes.
Pro tip Use the best and worst plausible outcomes to test both directions of bias.
Watch out A dramatic outcome can exert more emotional influence than its probability warrants.
- 5
Grade and aggregate the process
Score the reasoning independently of the result and compare it with similar decisions over time.
Pro tip Look for calibration and consistency across a meaningful sample.
Watch out One observation rarely distinguishes skill from luck reliably.
In the wild
Pete Carroll called a pass from the one-yard line with seconds remaining. The pass was intercepted, leading commentators to condemn the call, even though an incompletion would have stopped the clock and preserved two running attempts while interception risk was historically tiny.
→ The audit reframes the play as a probabilistic decision with a terrible realized outcome, not self-evident proof of terrible reasoning.
An investor buys a poorly researched stock and earns a large return after an unforeseeable takeover. Instead of treating the gain as validation, the investor reconstructs the original thesis, identifies missing risk analysis, and grades the process separately.
→ The investor avoids reinforcing a reckless process merely because it happened to win.
Common mistakes
Equating losses with mistakes
A negative outcome can follow a high-quality decision because the world contains hidden information and luck.
Using luck as a blanket defense
Calling every loss unlucky prevents examination of process errors that could improve future performance.
Reviewing only memorable outcomes
Extreme wins and losses are emotionally salient but may be less informative than patterns across ordinary decisions.
Is it for you?
Best for
It is best for investors, executives, competitors, and others making repeated decisions under uncertainty.
Not ideal for
It is less useful where actions and outcomes are almost perfectly linked and immediately observable.
From the transcript
“The quality of the outcome should have nothing to do with whether the decision quality was good.”
“We flipped the coin one time.”
“resulting is taking the quality of the result and deciding that that tells you what the quality of the case.”
From the episode
Annie Duke Discusses Decision-Making in Poker and Life
Annie Duke