Blind Evaluation Misvaluation Test
Use objective evidence first, then diagnose why the market gets value wrong
- Difficulty
- Moderate
- Time to result
- ~weeks to results
- Steps
- 6
- Confidence
- 97%
The Blind Evaluation Misvaluation Test separates two questions that markets often blur: what produces value, and why is that value priced incorrectly? First define measurable performance and evaluate candidates without appearance, pedigree, charisma, or other irrelevant cues. Only after producing an evidence-based ranking should evaluators restore the contextual information. Strong performers who look wrong to conventional gatekeepers become candidates for misvaluation. The mismatch is diagnostic: it suggests that the broader market is discounting a superficial characteristic rather than a true performance constraint. Decision-makers can then acquire or support that overlooked candidate while the bias persists. The method requires careful metric validation because a narrow statistic can omit essential qualities. Blindness improves judgment only when the selected evidence genuinely predicts the desired outcome.
Origin
Michael Lewis described on Masters in Business how the Oakland A’s used performance statistics and became especially interested when promising players looked unlike conventional professional athletes, revealing why other teams undervalued them.
Core principles
- 01Evaluate performance before seeing status or appearance cues
- 02Separate evidence of value from explanations of market price
- 03Look for strong results paired with traits the market dislikes
- 04Use visible mismatch to diagnose the source of misvaluation
- 05Verify that the objective measure predicts the outcome you need
How to run it
- 1
Define actual value
Specify the outcome the decision must produce and distinguish it from traditional proxies for quality. Select measures with a defensible relationship to that outcome.
Pro tip Use historical results to test whether the measure predicts success.
Watch out A precise metric can still measure the wrong thing.
- 2
Remove irrelevant cues
Hide names, photographs, pedigree, branding, or other information not required for the first evaluation. Present every candidate in a standardized format.
Pro tip Audit the format itself for clues that indirectly reveal identity or status.
Watch out Do not conceal information that is legitimately required for safety or role performance.
- 3
Rank on evidence
Score candidates using the predefined measures before discussing narrative impressions. Record the ranking so it cannot be quietly revised later.
Pro tip Use multiple independent scorers when interpretation remains necessary.
Watch out Changing criteria after seeing favored candidates reintroduces bias.
- 4
Reveal market context
Restore qualitative and market information after the blind ranking. Identify high performers receiving unexpectedly weak prices, ratings, or opportunities.
Pro tip Focus on large, repeated mismatches rather than marginal differences.
Watch out A low price may reflect a genuine constraint absent from the initial data.
- 5
Diagnose the discount
Investigate which visible trait causes others to reject the candidate and whether it affects actual performance. Treat the mismatch as a hypothesis, not proof.
Pro tip Compare similar cases sharing the disliked trait.
Watch out Do not romanticize every unconventional candidate as hidden talent.
- 6
Act and measure
Select the undervalued candidate when the evidence survives contextual review, then track real outcomes. Update both metrics and discount hypotheses from the results.
Pro tip Preserve a decision log so later success or failure can improve the model.
Watch out Once the strategy becomes widely known, the market discount may disappear.
In the wild
The A’s evaluated players through performance statistics and noticed that the strongest bargains often looked wrong to traditional scouts. Their appearance helped explain the gap between demonstrated contribution and market price.
→ Oakland assembled competitive teams despite a much smaller payroll.
An investor screens small companies using cash generation, retention, and balance-sheet resilience without initially viewing brand prestige or executive media profiles. After ranking them, the investor finds a strong business discounted because its industry and presentation are unfashionable, then validates that no operational risk explains the gap.
→ The process surfaces a potentially undervalued company while preserving a contextual risk check.
Common mistakes
Using convenient metrics
Blind evaluation fails when the chosen statistics are easy to collect but poorly connected to the desired result.
Ignoring legitimate context
Context should be delayed, not discarded. Safety, ethics, compatibility, and hidden constraints still require review before acting.
Assuming every mismatch is opportunity
An unconventional appearance can explain misvaluation, but it can also coincide with real weaknesses. Diagnose the discount before selecting the candidate.
Is it for you?
Best for
Decision-makers selecting talent, investments, suppliers, or opportunities in markets shaped by subjective impressions.
Not ideal for
Situations where the supposedly hidden qualitative factors are genuinely necessary for safety, ethics, or successful performance.
From the transcript
“we are evaluating them blind.”
“when we find the player whose statistics are are promising but they look wrong, we know why the market's misvaluing them.”
“If those people can be misvalued because of the misvalue because of the way they look, who can't be everybody can be.”
From the episode
Bonus: An Evening with Michael Lewis, From "Liar’s Poker" to Today