Dynamic Adaptive Questioning
Adapt each question to reach an accurate preference estimate faster
- Difficulty
- Moderate
- Time to result
- ~weeks to results
- Steps
- 5
- Confidence
- 97%
Dynamic adaptive questioning replaces a fixed questionnaire with a sequence that changes in response to each answer. Begin with a broad probe, such as whether a proposed portfolio is too risky. Use the response to update the plausible range of the person's risk tolerance, then present a more informative alternative closer to the estimated boundary. Repeat until the estimate is sufficiently precise. The mechanism resembles chemical titration: each adjustment reveals how far the current mixture is from the target. The method can reduce irrelevant questions and improve personalization, but its practical value depends on choosing the next question quickly enough for a natural real-time interaction. A stopping rule prevents marginal information gains from turning into unnecessary respondent burden.
Origin
Colin Camerer described a patented application developed with Caltech computer scientist Andreas Krause and Daniel Golovin. Extracted from Masters in Business.
Core principles
- 01Treat every answer as information for the next question
- 02Customize the sequence rather than administering a fixed survey
- 03Move progressively toward the respondent's decision boundary
- 04Balance information gain against computational and time constraints
How to run it
- 1
Define the target
Specify the latent preference or parameter to estimate, along with the level of precision required.
Pro tip Choose a target that can be expressed as an ordered range or decision boundary.
Watch out A vague target produces adaptive questions that are personalized but not informative.
- 2
Ask a broad probe
Present an initial option that divides the plausible preference range and is easy for the respondent to evaluate.
Pro tip Start near the middle of the expected range.
- 3
Update the estimate
Use the response to eliminate inconsistent possibilities and revise the current estimate.
Pro tip Track uncertainty as well as the most likely value.
Watch out Do not treat one answer as perfectly reliable.
- 4
Choose the next question
Generate the question expected to reduce uncertainty most while respecting computational and interaction constraints.
Pro tip Prefer questions near the respondent's apparent indifference point.
Watch out A theoretically optimal question is useless if it cannot be generated in real time.
- 5
Stop at sufficient precision
End the sequence once the remaining uncertainty is small enough for the intended decision.
Pro tip Set the stopping threshold before collecting responses.
Watch out Continuing too long can create fatigue and lower answer quality.
In the wild
An adviser shows a client a moderate-risk portfolio and asks whether it is too risky or not risky enough. After the client requests more risk, the system presents a somewhat riskier portfolio rather than continuing through unrelated fixed questions. Each answer narrows the client's risk-tolerance range until a suitable allocation is identified.
→ The adviser reaches a personalized risk estimate with fewer irrelevant questions.
Common mistakes
Following a fixed script
Asking every respondent the same remaining questions discards the information gained from earlier answers.
Ignoring response uncertainty
Noisy answers can send the sequence in the wrong direction if each response is treated as definitive.
Optimizing without latency limits
A question-selection algorithm that takes too long breaks the interaction even if it is statistically efficient.
Is it for you?
Best for
Researchers, advisers, and product teams that need efficient individualized preference estimates.
Not ideal for
Situations where questions cannot be changed dynamically or respondents must receive identical instruments.
From the transcript
“that's called active learning.”
“You know, you kind of change the mixture of the chemicals.”
“can you do it in real time”
From the episode
Advancing Behavioral Economics with Colin Camerer
Colin Camerer