Methodology-First Data Reading
Understand how a statistic is built before using it to explain the economy
- Difficulty
- Moderate
- Time to result
- ~weeks to results
- Steps
- 6
- Confidence
- 96%
McBride treats methodology as part of the evidence rather than background documentation. The analyst first learns what an agency counts, excludes, samples, revises, and cancels, then compares the resulting series with other available measures. This matters because two sources can describe similar activity while recording it differently: public home builders report net sales, while the Census Bureau follows a house and handles later cancellations through revisions. The method then uses graphs and multi-month context to distinguish a meaningful trend from ordinary variability or a temporary outlier. Transparent agencies make this work possible because their procedures and changes are public and staff can answer questions. Finally, the analyst names known gaps, such as inconsistent household-formation surveys or omitted high-rise condominiums, instead of presenting the statistic as complete. The output is a qualified interpretation tied to the measurement process.
Origin
Extracted from Masters in Business
Core principles
- 01Measurement must precede explanation and policy evaluation.
- 02A statistic is inseparable from what its producer counts and excludes.
- 03Revisions and cancellation rules can change the meaning of a reported number.
- 04Graphs reveal trends, variability, and outliers that isolated observations obscure.
- 05Transparent methods make disagreement testable.
How to run it
- 1
Define the question
State the phenomenon and decision the data must inform. This determines whether the available series actually measures the thing under discussion.
Pro tip Separate activity, prices, inventory, and economic contribution instead of treating housing as one number.
Watch out A precise statistic can still answer the wrong question.
- 2
Read the construction rules
Document the source, sample, inclusions, exclusions, and timing rules. Note whether the measure comes from an official agency, an industry body, or a private provider.
Pro tip Use the producer's methodology page and ask the agency when an important rule remains unclear.
Watch out A familiar label does not guarantee that two series count the same event.
- 3
Trace revisions and cancellations
Find out how later information changes earlier observations. Determine whether the headline number is gross, net, preliminary, or revised.
Pro tip Follow an outlier through later releases before treating it as a durable turn.
Watch out A first release may move sharply in the opposite direction after revision.
- 4
Triangulate the result
Compare the series with independent public, private, or related measures. Investigate divergence rather than selecting whichever number supports the preferred story.
Pro tip Use related measures that differ in collection method to reduce shared blind spots.
Watch out Two correlated series are not independent confirmation if they use the same underlying sample.
- 5
Graph the context
Plot enough history to show trend, seasonality, and normal variability. Use averages or a multi-month view when the monthly series is noisy.
Pro tip Mark revisions or methodology changes on the chart when they affect comparability.
Watch out A single observation can look decisive when detached from its historical range.
- 6
Report the boundary
Explain what the data supports and what remains uncertain or unmeasured. Keep the conclusion within the limits of the collection system.
Pro tip Name exclusions that are becoming economically important, such as omitted housing types.
Watch out Do not turn a measurement gap into evidence that the missing activity is zero.
In the wild
Public home builders reported net sales, while the Census Bureau tracked a house and later removed a canceled contract through its revision process. During the crisis, understanding this difference helped analysts anticipate how reported new-home sales would catch up with builder results.
→ Methodology explained an apparent disagreement between two housing-sales measures.
An unusually strong new-home-sales month is plotted against the series' normal range and reviewed with a three-month average. The following weak month and subsequent revisions show that the spike was variability rather than a decisive change in trend.
→ The analyst reports improvement without overstating one volatile observation.
Common mistakes
Reading only the headline
The headline omits the inclusion, exclusion, cancellation, and revision rules that determine what the figure means.
Treating an outlier as a trend
A volatile monthly series can reverse or be revised, so one observation needs historical and multi-month context.
Hiding known measurement gaps
Inconsistent surveys and excluded housing types limit the confidence of any conclusion based on them.
Is it for you?
Best for
It is best for journalists, investors, researchers, and policymakers interpreting official or private economic data.
Not ideal for
It cannot rescue a conclusion when the underlying phenomenon is not measured reliably at all.
From the transcript
“you can go to their website, you can understand exactly what they do, what they don't do”
“one of the reasons I do graphs anyway. It is because it kind of can show you where you're at.”
From the episode
Interview With Bill McBride: Masters in Business (Audio)