Comparable-Case Forecasting
Project possible futures from historically similar cases
- Difficulty
- Advanced
- Time to result
- ~months to results
- Steps
- 5
- Confidence
- 96%
Comparable-Case Forecasting starts with a current subject, identifies the past cases that most closely resemble it, and examines what happened next across that reference class. Silver describes PECOTA using a player's age, history, and attributes to find similar players, then turning their subsequent careers into many possible paths rather than one certain outcome. The mechanism is a base-rate forecast: relevant similarities define the comparison set, the observed outcomes form a distribution, and that distribution becomes the starting probability range for the present case. The forecaster then adjusts cautiously for meaningful differences. This works especially well when the underlying activity is stable and the data is rich, consistent, and measurable. It becomes less dependable when the environment shifts or the historical measurements are weak.
Origin
Nate Silver explains how PECOTA used baseball's rich history and comparable players to project a range of future career paths.
Core principles
- 01Use history to constrain expectations
- 02Compare cases on relevant attributes rather than one headline trait
- 03Represent the future as multiple plausible paths
- 04Treat projections as probabilities rather than certainties
How to run it
- 1
Define the forecast target
Specify the subject, time horizon, and outcome you are trying to estimate. Avoid changing the target after seeing the data.
Pro tip Use an outcome that past records measured consistently.
Watch out A vague target invites selective interpretation.
- 2
Choose comparable attributes
Select characteristics that plausibly relate to the outcome, such as age, prior performance, and relevant operating conditions.
Pro tip Prefer attributes available for both the current case and the historical cases.
Watch out Superficial resemblance can create a misleading comparison set.
- 3
Assemble the reference class
Find a broad set of past cases with similar attributes and histories. Preserve variation rather than choosing only memorable examples.
Pro tip Include failed and ordinary cases, not just successes.
Watch out Cherry-picking analogues destroys the base rate.
- 4
Map subsequent paths
Record what happened next for the comparable cases and form a distribution of outcomes. Keep distinct paths visible instead of collapsing them immediately into one number.
Pro tip Show downside, central, and upside paths.
Watch out An average alone can hide consequential tail risks.
- 5
Adjust for real differences
Compare the present case with the reference class and adjust only for differences supported by evidence. Report the final result as a probability range.
Pro tip Document every adjustment so it can be challenged later.
Watch out Too many bespoke adjustments turn a base-rate forecast back into a hunch.
In the wild
Silver describes taking a current player such as Curtis Granderson at age thirty-three, finding historical players with similar attributes and histories at the same age, and using their later careers to construct many possible career paths.
→ The forecast expresses a range of plausible futures grounded in baseball history rather than one deterministic prediction.
A product team could compare a new cohort with past cohorts that had similar acquisition channels, onboarding completion, and first-week usage, then inspect the later retention paths of those cohorts.
→ The team gets an evidence-based retention range and a visible downside case instead of relying on the latest cohort's early excitement.
Common mistakes
Choosing only famous analogues
Memorable successes and failures are not a representative reference class. Include the ordinary cases that establish the true base rate.
Ignoring environmental change
Historical resemblance does not guarantee predictive value when rules, measurements, or incentives have changed.
Is it for you?
Best for
It is best for recurring situations with reliable historical data and enough comparable cases.
Not ideal for
It is not ideal for unprecedented situations or domains whose conditions and measurements change rapidly.
From the transcript
“the idea of Pakoda was to use baseball is very rich history to project the future”
“go back in history and see who were the guys who were like Curtis Granderson at age thirty three and similar attributes, similar history, similar…”
“here are a hundred different career paths”
From the episode
Statistics With Nate Silver: Masters in Business (Audio)
Nate Silver