Process-Outcome Separation
Judge decision quality separately from whether the bet happened to win
- Difficulty
- Moderate
- Time to result
- ~weeks to results
- Steps
- 5
- Confidence
- 91%
Process-Outcome Separation evaluates a decision using two distinct questions: whether the reasoning was sound at the time and whether the eventual result was favorable. Before acting, decision-makers record available evidence, assumptions, possible scenarios, exposure limits, and hedges. Afterward, they reconstruct the decision without allowing hindsight to rewrite what was knowable. This protects teams from rewarding reckless bets that happened to win or punishing sound probabilistic decisions that lost. The framework does not excuse incomplete analysis: the Jane Street election trade captured an informational advantage but failed to consider how US markets might respond after a Trump victory. A useful postmortem therefore preserves what worked, identifies missing second-order thinking, and converts the lesson into a stronger decision rule for the next comparable situation.
Origin
Extracted from Masters in Business through Michael Lewis's account of Jane Street's 2016 election trade, which moved from a $300 million gain to a $300 million loss despite successfully obtaining election data before the market.
Core principles
- 01A good outcome does not prove the process was sound
- 02A bad outcome does not prove the decision was irrational
- 03Evaluate what was knowable when the decision was made
- 04Include second-order scenarios before committing
- 05Use postmortems to improve future decisions rather than assign blame
How to run it
- 1
Freeze the pre-decision record
Document the information, probabilities, assumptions, and constraints visible before the outcome. This becomes the baseline for a fair evaluation.
Pro tip Timestamp the record so later knowledge cannot leak into it.
Watch out A post-hoc reconstruction will usually exaggerate how obvious the result was.
- 2
Map the causal scenarios
Describe what must happen for the decision to work and how other actors or markets could respond. Include at least one scenario beyond the immediate expected reaction.
Pro tip Ask what happens after the event everyone is currently trading.
Watch out An informational edge is not enough when the causal interpretation is wrong.
- 3
Bound and hedge the exposure
Set maximum loss, exit conditions, and offsetting positions before acting. Match position size to uncertainty rather than conviction alone.
Pro tip Hedge components that could react differently to the same event.
Watch out Do not let an early gain substitute for risk control.
- 4
Score process and result separately
After the event, rate the analytical process independently from the financial or operational outcome. Identify which parts were skill, luck, or unmodeled exposure.
Pro tip Use separate columns for process quality and outcome quality.
Watch out Do not call every loss a bad decision or every profit a good one.
- 5
Convert errors into rules
Preserve the valid edge while correcting missing assumptions, hedges, or scenarios. Apply the revised rule to future decisions with similar causal structure.
Pro tip Write the new rule in a form that can be checked before the next trade.
Watch out A postmortem that produces no behavioral change is only storytelling.
In the wild
Jane Street built a system that received decisive state election data before television networks and used it to short markets as Donald Trump's victory became likely. The position showed a roughly $300 million gain overnight, validating the information advantage. US markets then reversed as investors anticipated tax cuts and stimulus, turning the gain into a roughly $300 million loss while international markets stayed down.
→ The episode showed that the data process worked but the trade lacked adequate second-order analysis and market-specific hedging.
Common mistakes
Outcome bias
Judging only by profit or loss confuses luck with analytical quality and teaches the wrong lesson.
Stopping at the first reaction
Predicting the immediate market response without modeling what follows can reverse an otherwise valid edge.
Using postmortems to find a culprit
Scapegoating discourages transparent analysis and obscures failures in the broader decision system.
Is it for you?
Best for
It is best for investors, executives, operators, and teams making repeated decisions under uncertainty.
Not ideal for
It is not ideal when misconduct, ignored controls, or unauthorized risk makes the process clearly unacceptable regardless of outcome.
From the transcript
“This is a very smart place because the process was right, the outcome was a problem.”
“We succeeded. We got the information before anybody else. It wasn't in the market. Nobody else had it.”
“They no stop and say, all right, so what happens if he's elected?”
From the episode
Michael Lewis on Cryptocurrency and the FTX Empire