Random-Sale Counterfactual Audit
Benchmark each sale against a random alternative from the same portfolio
- Difficulty
- Moderate
- Time to result
- ~months to results
- Steps
- 6
- Confidence
- 97%
The audit evaluates selling independently from buying. At every discretionary sale, it records the manager's eligible holdings and creates a simple counterfactual: what would have happened if a random different holding had been sold instead? Actual and counterfactual positions are then tracked over the same horizon, controlling for the opportunity set the manager truly faced. Repeated underperformance against this deliberately easy benchmark signals that selling discretion is destroying value. Interviews and portfolio data can then diagnose mechanisms such as selling whatever is salient, unloading recent purchases before their alpha matures, or treating sales merely as a way to fund new ideas. The output is both a performance score for selling and evidence for redesigning exit criteria, attention allocation, and holding-period rules.
Origin
Alex Imas and collaborators developed the comparison while studying professional fund managers. They found that managers' sales performed worse than randomly selected alternative sales from their own portfolios.
Core principles
- 01Selling is an investment decision
- 02A valid benchmark must reflect the manager's actual opportunity set
- 03Random alternatives expose whether discretion adds value
- 04Salience and attachment can distort exit choices
- 05Buying skill does not guarantee selling skill
How to run it
- 1
Define eligible sales
Identify discretionary sales and exclude transactions compelled by mandates, redemptions, liquidity needs, or other constraints that eliminate choice.
Pro tip Record the stated reason for each sale at decision time.
Watch out Mixing forced and discretionary sales can misdiagnose skill.
- 2
Snapshot the opportunity set
Capture every holding the manager could reasonably have sold at the same moment.
Pro tip Apply the same liquidity, mandate, tax, and concentration constraints to all alternatives.
Watch out A counterfactual is unfair if it includes assets the manager could not sell.
- 3
Generate random alternatives
Randomly sample one or more eligible holdings as substitute sales, using the same transaction size where possible.
Pro tip Use many random draws to estimate a distribution rather than relying on one lucky comparison.
Watch out Do not handpick alternatives after observing their performance.
- 4
Track equal-horizon performance
Compare the subsequent return of the actual sale with the returns of the counterfactual sales over identical horizons.
Pro tip Test horizons that match the strategy's expected alpha decay.
Watch out Unequal horizons can turn timing differences into false evidence of skill.
- 5
Diagnose the pattern
Examine whether poor sales cluster around recent purchases, salient positions, extreme gains or losses, or rushed funding decisions.
Pro tip Combine transaction data with contemporaneous decision notes and interviews.
Watch out Aggregate underperformance alone does not reveal the behavioral cause.
- 6
Redesign and retest
Introduce explicit sale criteria, minimum review effort, or holding-period checks and then repeat the counterfactual analysis.
Pro tip Require comparable analytical rigor for buys and sells.
Watch out A rigid holding rule should not override genuinely material new information.
In the wild
Researchers compared each manager's actual sale with the outcome from throwing a figurative dart at another eligible holding in the same portfolio. The managers' chosen sales subsequently performed worse than the random alternatives.
→ The analysis isolated a selling blind spot despite demonstrated skill in buying securities.
The managers often sold recently purchased positions even though their buying skill generated value that decayed over roughly nine months. They were selling around six months, before fully harvesting that alpha.
→ Extending the review horizon could preserve more of the value created by their purchases.
Common mistakes
Treating sales as administrative
Selling changes portfolio exposure and opportunity cost, so it deserves the same rigor as buying.
Using an unrealistic benchmark
A broad market index does not isolate whether the manager chose the right holding to sell from the available portfolio.
Selling what attracts attention
Salience, attachment, and the excitement of a new idea can substitute for analysis of expected future returns.
Is it for you?
Best for
It is best for portfolio managers and other decision-makers who repeatedly choose which existing asset, project, or product to exit.
Not ideal for
It is not ideal when sales are mechanically forced, holdings are incomparable, or constraints make most alternatives ineligible.
From the transcript
“I'm just going to throw a dart in their portfolio and sell that instead. And they did worse than random.”
“selling is not really that important. It's not really an investment decision.”
“they could maybe double their alpha if they're selling decisions or as good as they're buying decisions.”
From the episode
Special Edition: Nobel Prize Winner Richard Thaler Live from the Economic Club of New York