Source Provenance and Reliability Audit
Trace information to its incentives, evidence, and historical accuracy
- Difficulty
- Easy
- Time to result
- ~weeks to results
- Steps
- 5
- Confidence
- 96%
The audit treats every news claim as the output of a chain rather than as a self-validating fact. Start by locating the original data, study, filing, or statement behind the article. Then inspect who produced, funded, or commissioned it and what incentives, ideology, and history may shape its presentation. Compare the published interpretation with the raw material and seek independent corroboration. Finally, update a running judgment of the source as its claims are confirmed, refuted, or debunked over time. The mechanism converts vague impressions of credibility into evidence-based trust: provenance reveals incentives, primary material tests interpretation, and a historical record shows whether the source deserves future attention.
Origin
Extracted from Masters in Business, where Michael Hiltzik explained how he evaluates financial reporting, commissioned research, and information sources.
Core principles
- 01Treat reported data as a second-order account
- 02Identify who funded or commissioned the evidence
- 03Judge sources by incentives and historical behavior
- 04Prefer raw material over summaries when practical
- 05Build trust from repeated accuracy, not reputation alone
How to run it
- 1
Locate the Original Source
Trace the claim back from the article or post to the underlying report, dataset, filing, interview, or press release.
Pro tip Search for the report title or quoted statistic rather than relying on the publisher's link.
Watch out Do not assume a news article is the primary source merely because it quotes numbers.
- 2
Map the Incentives
Identify the producer, funder, commissioner, and distributor of the information. Note what each party gains if readers accept the claim.
Pro tip Ask directly whether an apparently independent study was commissioned by an interested organization.
Watch out A credible institutional name does not eliminate conflicts of interest.
- 3
Inspect the Raw Material
Compare the headline and reported interpretation with the original methodology, definitions, sample, and results.
Pro tip Focus first on whether the source material supports the article's central claim.
Watch out Do not reject evidence solely because its sponsor has an agenda; test the evidence itself.
- 4
Check Independent Context
Compare the claim with other reliable evidence, established facts, and relevant real-world experience.
Pro tip Look for sources with different incentives that reach the same conclusion.
Watch out Personal experience can flag a discrepancy, but it cannot replace representative data.
- 5
Update the Track Record
Observe whether the claim is later confirmed, corrected, refuted, or debunked and use that result to recalibrate trust in the source.
Pro tip Maintain a short list of consistently reliable sources and recurring offenders.
Watch out Do not grant permanent trust after one accurate prediction or permanent distrust after one honest error.
In the wild
An investor sees a trade association claim that demand for its members' products will double. She finds the underlying study, discovers that the association commissioned it, checks its growth assumptions against government data, and compares the forecast with an independently funded analysis.
→ She recognizes that the headline rests on unusually optimistic assumptions and avoids treating it as an objective forecast.
Instead of immediately buying a cold-call recommendation, an investor records the pitch, watches its subsequent performance, and repeats the process with later recommendations before assigning the broker any credibility.
→ The investor builds an evidence-based record of the broker's reliability without risking capital on an untested source.
Common mistakes
Stopping at the Publisher
A reputable outlet may still be repeating a press release or a reporter's rushed interpretation. Trace the claim to its original evidence.
Ignoring Who Paid
An apparently independent report may have been commissioned by an interested party whose incentives shape its questions or presentation.
Trusting One Successful Call
One accurate claim is insufficient evidence of reliability. Evaluate performance across time and multiple claims.
Is it for you?
Best for
It is best for investors, researchers, and decision-makers evaluating consequential claims from news, studies, trade groups, or advisers.
Not ideal for
It is unnecessary for low-stakes information that cannot materially affect a decision.
From the transcript
“I think you always have to assess the source.”
“if they can go to the raw material and make a judgment for themselves, intelligent investors should be able to do that.”
“some of these sources will compile a record of inaccuracy or dishonesty, and some will compile the record of reliability.”
From the episode
At The Money: Filtering Noisy News