Assumption-and-Condition Model Audit
Define a model's operating boundaries before trusting its output
- Difficulty
- Moderate
- Time to result
- ~weeks to results
- Steps
- 5
- Confidence
- 96%
The Assumption-and-Condition Model Audit turns hidden model risk into an explicit operating boundary. The model builder records the assumptions and conditions required for the model to remain appropriate, then identifies observable circumstances in which those assumptions fail. Users compare the current environment with that boundary before relying on the output. Derman illustrates the mechanism with an option model for Apple that can reasonably ignore stable interest rates, but should not be transferred unchanged to an emerging market where rates can move sharply. The audit does not promise that a model is true. It makes the model's limited domain visible so that a change in rates, volatility, market behavior, or another structural input triggers human review rather than blind continued use.
Origin
Derman says Goldman's firm-wide risk group recommended requiring model writers to specify their assumptions and conditions so users could identify when those conditions no longer held.
Core principles
- 01Every model depends on assumptions about how the world behaves.
- 02A useful model has an explicit domain of validity.
- 03Changed conditions can invalidate an otherwise sound model.
- 04Mathematics implements an underlying idea rather than replacing it.
How to run it
- 1
State the underlying idea
Describe the economic or financial idea the mathematics implements. Separate the mechanism from the equations used to express it.
Pro tip Use plain language that a model user can challenge.
Watch out Do not treat the mathematics as the idea itself.
- 2
List the assumptions
Record every material assumption about rates, volatility, liquidity, behavior, and market structure that the model relies on.
Pro tip Prioritize assumptions that can move quickly during stress.
Watch out An unstated assumption cannot trigger a timely review.
- 3
Define valid conditions
Translate each assumption into the conditions under which the model remains appropriate. Make the boundary observable where possible.
Pro tip Describe both normal conditions and plausible edge conditions.
Watch out Do not assume a model built for one market transfers unchanged to another.
- 4
Specify invalidation signals
Name the changes that would show an assumption no longer holds and therefore require intervention.
Pro tip Tie each signal to a specific assumption rather than a generic loss threshold.
Watch out A model can fail because the world changed even when its implementation is correct.
- 5
Recheck before use
Compare current conditions with the documented boundary before relying on the output. Replace, revise, or suspend the model when a boundary is crossed.
Pro tip Make the boundary review part of model approval and monitoring.
Watch out Do not continue mechanically simply because the model worked in the previous regime.
In the wild
Derman contrasts pricing options on Apple, where a model may reasonably treat interest rates as stable, with using that approach in an emerging-market country where rates can move sharply. The changed rate environment violates an assumption that was harmless in the original setting.
→ The boundary check shows that the original option model is not the right model for the new market.
A derivatives desk documents that its pricing model assumes volatility remains within its recent range. A sustained volatility jump crosses the stated boundary, so the desk pauses automated reliance and reviews the structural assumptions before resuming use.
→ The team intervenes because a documented condition changed, not merely because a recent trade lost money.
Common mistakes
Leaving assumptions implicit
Users cannot recognize a boundary crossing when the model builder never stated the boundary.
Confusing mathematics with mechanism
A precise equation does not rescue an economic or financial idea that no longer describes the world.
Transferring models across regimes
Conditions that are safely ignored in one market may dominate outcomes in another.
Is it for you?
Best for
It is best for teams building or approving financial models exposed to changing rates, volatility, or market structure.
Not ideal for
It is not ideal for decisions that have no explicit model, assumptions, or observable operating conditions.
From the transcript
“every time somebody write a model, they'd be forced to specify all the assumptions and conditions they were they were making so that they could…”
“you ought to be able to make some sort of judgment as to whether the world is still behaving according to the assumptions that you…”
“most models get their inspiration out of some economic or or or financial idea first, and the mathematics is just the implementation.”
From the episode
Interview With Emanuel Derman: Masters in Business (Audio)
Emanuel Derman