Question-Data-Inference Problem-Solving Loop
Frame the question, gather evidence, and turn data into action
- Difficulty
- Moderate
- Time to result
- ~weeks to results
- Steps
- 6
- Confidence
- 96%
This loop treats problem-solving as a sequence rather than a hunt for an immediately correct answer. First, define and pose the question so that the analysis targets the real decision. Next, gather data capable of distinguishing among plausible answers. Then use appropriate tools and models to organize that evidence and draw inferences. The process finishes by deciding how to act, while recognizing that the model and available data remain incomplete. Its value comes from preserving alignment between the original problem, the evidence collected, and the eventual action. It also makes disagreements easier to diagnose: participants can determine whether they differ about the question, the data, the model, the interpretation, or the decision.
Origin
Extracted from Masters in Business during Gerard O'Reilly's comparison of scientific, engineering, and financial problem-solving.
Core principles
- 01A well-posed question determines the quality of the answer
- 02Relevant data should address the question rather than merely be available
- 03Models organize evidence but do not reproduce reality
- 04Interpretation must end in a decision or action
- 05Conclusions should remain open to revision
How to run it
- 1
Frame the decision
State the practical decision or outcome that the analysis must support. Separate it from interesting but nonessential questions.
Pro tip Write the decision in one sentence before beginning the research.
Watch out A vague objective encourages analysis that is impressive but unusable.
- 2
Pose the right question
Translate the decision into a precise question that evidence can address. Check whether answering it would genuinely change the intended action.
Pro tip Ask what observation would distinguish one possible answer from another.
Watch out Do not assume that the first formulation of the problem is the correct one.
- 3
Gather relevant data
Collect evidence directly related to the question, including data that could contradict the preferred answer. Assess its provenance and limitations.
Pro tip Define the necessary data before inspecting the available data.
Watch out Convenient data can become a substitute for relevant data.
- 4
Organize the evidence
Choose tools and models that expose relationships in the data. Use more than one representation when the decision is consequential.
Pro tip Prefer the simplest model that captures the mechanism needed for the decision.
Watch out Do not mistake a model's precision for certainty about reality.
- 5
Draw actionable inferences
Explain what the evidence implies and what remains uncertain. Connect each conclusion explicitly to the decision it supports.
Pro tip Record both the inference and the conditions under which it would fail.
Watch out Correlation or historical fit alone may not justify a forward-looking action.
- 6
Act and update
Make the best decision supported by the current evidence, then monitor new information. Revise the model or action when conditions change.
Pro tip Set a review trigger before implementing the decision.
Watch out Waiting for complete certainty can become another form of decision failure.
In the wild
An investment committee begins with the question of which risks investors should be paid to bear, rather than asking which fund recently performed best. It gathers long-horizon return and valuation data, compares several explanatory models, identifies robust expected-return drivers, and chooses a diversified implementation with explicit review conditions.
→ The committee selects a strategy connected to evidence and client goals rather than recent performance.
A software team reframes “Why are users leaving?” as “Which first-week behaviors distinguish retained accounts from cancelled accounts?” It combines usage, support, and billing data, tests several explanations, and changes onboarding around the strongest actionable finding.
→ The team moves from broad speculation to a measurable intervention.
Common mistakes
Solving the first question asked
The initial question may reflect a symptom or an untested assumption rather than the underlying decision. Refine it before collecting evidence.
Starting with available data
Beginning with whatever data is easiest to obtain encourages questions tailored to the dataset rather than the real problem.
Treating the model as reality
A model is an incomplete organizing device. Build decisions that remain acceptable when its assumptions are imperfect.
Is it for you?
Best for
It is best for researchers, investors, engineers, and leaders making consequential decisions with incomplete evidence.
Not ideal for
It is not ideal for trivial choices where the cost of formal analysis exceeds the value of the decision.
From the transcript
“how do I number one pose the question correctly? How do I ask the right question? Because that's as important as trying to solve the…”
“Then how do I gather data to help me address and find the answer to this question?”
“What are the tools and the models that I can use such that I'm going to be able to organize these data in such a…”
From the episode
Gerard O’Reilly on Academic Research and Stocks