Strategy-Relative Trader Due Diligence
Test a track record against its market regime, design choices, and human behavior.
- Difficulty
- Expert
- Time to result
- ~weeks to results
- Steps
- 6
- Confidence
- 96%
Strategy-Relative Trader Due Diligence rejects a headline track record as sufficient evidence of skill. For quantitative managers, the allocator prioritizes real-time results, separates performance by market regime, and interrogates choices about instruments, frequency, volatility, drawdowns, and excluded data. Exceptional results during a rare trend or dislocation may reveal regime exposure rather than repeatable edge, while selective market exclusions may reveal overoptimization. For discretionary traders, the same statistical analysis is supplemented with emotional and psychological assessment. The allocator studies the trader’s worst episode, checks whether rules were violated, and evaluates how the trader recovered. The output is not a single performance rank but a view of how the strategy is built, when it works, how it fails, and whether the operator remains disciplined under stress.
Origin
Extracted from Masters in Business as Peter Borish explains how he evaluates quantitative and discretionary traders before allocating capital.
Core principles
- 01Distrust simulated perfection.
- 02Evaluate performance within its market regime.
- 03Investigate exclusions and design choices for overfitting.
- 04Examine volatility, frequency, and drawdowns together.
- 05Assess emotional recovery for discretionary traders.
How to run it
- 1
Validate the evidence
Separate simulated, backtested, and real-time returns. Verify that the record covers actual decisions made with capital at risk.
Pro tip Give substantially more weight to live performance than to optimized simulations.
Watch out A flawless backtest may primarily demonstrate the ability to predict yesterday.
- 2
Segment by regime
Break results into trending, range-bound, volatile, calm, crisis, and ordinary periods. Determine which environments produced most of the profit.
Pro tip Compare the strategy’s best year with the frequency of similar historical regimes.
Watch out Do not annualize the edge from an event that occurs once every decade or more.
- 3
Interrogate design choices
Ask why each market, parameter, and exclusion was chosen. Test whether removing inconvenient instruments or periods artificially improved results.
Pro tip Request results with excluded markets restored where possible.
Watch out A claim that an instrument is simply unsuitable may conceal overoptimization.
- 4
Map the risk profile
Review trading frequency, volatility, leverage, concentration, and drawdown behavior together. Identify the conditions under which losses accelerate.
Pro tip Study both average behavior and extreme observations.
Watch out A smooth return series may hide stale pricing, asymmetric exposure, or rare catastrophic risk.
- 5
Assess human resilience
For discretionary traders, investigate the worst loss or near-death experience, whether rules were broken, and what changed afterward. Evaluate accountability and emotional recovery.
Pro tip Ask the trader to reconstruct the episode decision by decision.
Watch out A trader who externalizes every failure is unlikely to correct the underlying behavior.
- 6
Make a conditional allocation
Define the regime assumptions, risk limits, review triggers, and capital size under which the strategy is acceptable. Monitor whether live behavior remains consistent with the diligence case.
Pro tip Tie increases in capital to verified live behavior rather than promises.
Watch out Past performance alone cannot establish future suitability.
In the wild
A CTA presents its best performance since a previous period of major dislocation. The allocator observes that sustained directional trends created an unusually favorable regime, then examines which markets drove returns, why others were excluded, and how the model performed during ordinary years.
→ The allocator avoids treating a rare regime as the strategy’s normal expected return.
A discretionary trader has strong long-term returns but recently suffered a major loss. The allocator verifies that the trader stayed within limits, asks how the loss developed, and looks for evidence that position sizing and decision processes changed afterward.
→ The allocation reflects both statistical performance and the trader’s demonstrated capacity to recover responsibly.
Common mistakes
Buying the best recent record
The allocator mistakes performance in a favorable regime for evidence that the same return profile will persist.
Accepting unexplained exclusions
Markets or periods removed from a model are accepted without testing whether they were selected to improve the backtest.
Ignoring operator psychology
A discretionary trader is evaluated like a static model even though emotional responses can materially change live execution.
Is it for you?
Best for
It is best for hedge funds, family offices, and allocators evaluating quantitative or discretionary traders.
Not ideal for
It is not ideal for making allocation decisions when reliable real-time records, position data, and risk documentation are unavailable.
From the transcript
“But everybody looks at past performance. So on the quant side, you have to look at real time data and you also have to segment…”
“So you really have to start digging deep, asking very very tough questions.”
“every successful trader has a near death experience. I just hope they have it before we've allocated to them.”
From the episode
Peter Borish on Lessons From the 1987 Market Crash
Peter Borish