Data-to-Model Research Loop
Turn observed data structures into testable financial models
- Difficulty
- Advanced
- Time to result
- ~months to results
- Steps
- 5
- Confidence
- 94%
The Data-to-Model Research Loop begins with close observation rather than a preferred theory. Researchers inspect data, search for stable structures, and decide which features appear important enough to explain. A promising observation is then translated into a model that can be tested, challenged, and refined. In finance, the loop must extend beyond predictive accuracy to portfolio robustness, risk, execution, and trading costs. The mechanism is deliberately empirical: observations generate hypotheses, hypotheses generate models, and model performance determines what survives. This approach is particularly useful in complex systems where interactions can produce unexpected aggregate behavior and where elegant axioms may conceal assumptions contradicted by the evidence.
Origin
Bouchaud describes transferring the empirical habits of statistical physics into CFM's quantitative financial research. Physicists examine complex data, recognize a structure, and build a model capable of explaining or exploiting it.
Core principles
- 01Start with observations rather than axioms
- 02Separate meaningful structures from noise
- 03Convert promising patterns into explicit models
- 04Test whether models remain useful under real constraints
- 05Let empirical evidence outrank theoretical preference
How to run it
- 1
Gather observations
Assemble data that directly represents the system and decision horizon under investigation. Confirm its coverage, granularity, and limitations before searching for patterns.
Pro tip Use multiple instruments, periods, or environments where possible.
Watch out A large dataset is not necessarily a representative dataset.
- 2
Search for structure
Inspect the data for recurring relationships, asymmetries, trends, interactions, or regime-dependent behavior. Distinguish persistent structure from visually compelling noise.
Pro tip Compare the pattern across independent slices of the data.
Watch out Do not begin by forcing the observations to fit a favored theory.
- 3
Select material features
Identify the smallest set of variables that plausibly drives or represents the observed structure. Discard features that add complexity without stable explanatory value.
Pro tip Prefer mechanisms that can be stated plainly.
Watch out Feature abundance can disguise weak causal or statistical foundations.
- 4
Build a testable model
Translate the proposed mechanism into explicit predictions or portfolio rules. Define in advance what evidence would weaken the model.
Pro tip Keep an interpretable baseline against which sophisticated models can be compared.
Watch out A model that cannot fail a test cannot earn confidence.
- 5
Validate practical robustness
Test the model outside its discovery sample and evaluate its sensitivity to assumptions. Include portfolio construction, risk, execution, and costs before deployment.
Pro tip Stress the model under periods unlike the training sample.
Watch out Predictive accuracy alone does not make a tradable model.
In the wild
A research team observes that price moves across many futures markets sometimes persist. It measures the effect across instruments and decades, formulates a simple long-short trend rule, and then tests whether the result survives realistic costs, volatility controls, and changing correlations.
→ The team obtains a transparent strategy whose empirical structure can be compared with more complex alternatives.
Common mistakes
Starting with the desired answer
Choosing a theory first can make researchers reinterpret every observation as support for it. Begin with the data and specify evidence that could invalidate the model.
Mistaking complexity for structure
A complicated model may fit more historical variation without capturing anything stable. Require improvements to persist outside the discovery sample.
Ignoring implementation
A statistical relationship can disappear after portfolio constraints, execution, and costs are introduced. Test the full investment process rather than an isolated signal.
Is it for you?
Best for
It is best for quantitative researchers and data-rich teams investigating complex, poorly understood systems.
Not ideal for
It is not ideal when data is sparse, unreliable, or incapable of representing the mechanism being studied.
From the transcript
“You can identify what's important and what's not.”
“Experiments is about everything else.”
“We're really there to, you know, make models that work, build portfolios that are robust, model risk, model execution, control costs.”
From the episode
The Intersection of Science and Finance with CFM's Jean-Philippe Bouchaud