Innovation Claim Audit
Test technology narratives against mechanisms, incentives, and existing rules
- Difficulty
- Moderate
- Time to result
- ~weeks to results
- Steps
- 7
- Confidence
- 96%
The Innovation Claim Audit starts by translating promotional language into a precise claim about what a technology does and for whom. It then inspects the underlying mechanism rather than accepting labels such as AI, blockchain, disruption, or democratization as evidence. The auditor compares the product with existing tools and historical analogues, asks whether current law already governs the underlying activity, and identifies whether legal exemptions rather than technical superiority create the commercial advantage. Independent domain experts test physical and operational feasibility, while an incentive map reveals who benefits from the narrative through funding, lobbying, subsidies, or rapid exits. The resulting decision separates demonstrated utility from constructed novelty and specifies which claims require further proof.
Origin
Allen's work on financial crises produced skepticism toward innovation rhetoric, which she later applied to blockchain, fintech lending, crypto, AI, and venture-backed technology. Extracted from Masters in Business.
Core principles
- 01Novel branding does not prove functional novelty
- 02Technical capability and marketing claims must be separated
- 03Existing rules may address old behavior in new packaging
- 04Commercial incentives shape innovation narratives
- 05Domain expertise is essential when testing feasibility
- 06Optimism should be balanced by informed realism
How to run it
- 1
Translate the pitch
Rewrite the promotional story as a testable statement describing the user, problem, mechanism, and measurable improvement. Remove prestige labels and vague promises.
Pro tip Replace words such as revolutionary or intelligent with the specific operation the system performs.
Watch out Do not let a compelling mission substitute for a falsifiable product claim.
- 2
Inspect the mechanism
Determine what the technology actually does, what inputs it requires, and how it produces its output. Compare those capabilities with the outcomes claimed in the pitch.
Pro tip Ask a technical expert who has no financial interest in the product to explain the mechanism plainly.
Watch out A product can perform a real technical operation without delivering the broader benefit being marketed.
- 3
Test functional novelty
Compare the offering with existing products and historical analogues. Identify whether it solves a new problem, solves an old problem better, or merely repackages an old activity.
Pro tip Compare outcomes and risks, not interfaces or terminology.
Watch out A digital wrapper does not necessarily make the underlying financial or commercial activity new.
- 4
Audit the legal advantage
List the rules governing comparable incumbent products and identify which ones the new entrant avoids. Determine whether that difference is justified by reduced risk or is simply regulatory arbitrage.
Pro tip Ask whether the business model would survive if functionally equivalent rules applied.
Watch out A cost advantage created by avoiding protections is not evidence of technical superiority.
- 5
Add domain reality
Consult practitioners who understand the underlying field and ask what physical, professional, or operational constraints the pitch overlooks. Test whether the desired result is technically possible.
Pro tip Give special weight to experts responsible for real-world failures and safety outcomes.
Watch out Do not dismiss domain objections merely because they conflict with an optimistic technology narrative.
- 6
Map narrative incentives
Identify investors, promoters, lobbyists, subsidies, and exit timelines behind the product. Assess how each party benefits if buyers or policymakers accept the innovation story.
Pro tip Separate evidence created by independent users from evidence distributed by interested promoters.
Watch out Consensus among financially aligned parties is not independent validation.
- 7
Reach an evidence-based verdict
Classify each major claim as demonstrated, plausible but unproven, contradicted, or dependent on favorable legal treatment. Approve, reject, or constrain adoption accordingly.
Pro tip Specify the evidence that could change the verdict later.
Watch out Avoid both automatic enthusiasm and automatic rejection; evaluate the actual mechanism and evidence.
In the wild
A law firm claims an LLM will replace junior research work. The audit finds that the model generates fluent text but cannot independently guarantee citation accuracy. Experienced lawyers must review every output, while junior staff using it from the start may not develop the expertise needed to detect errors. The firm limits the tool to drafting and retrieval tasks with source verification.
→ The firm gains limited efficiency without treating probabilistic text generation as autonomous legal judgment.
A lender argues that AI underwriting makes it fundamentally different from payday lending. The audit compares fees on an annualized basis, borrower outcomes, and applicable consumer protections. It finds familiar short-term credit economics despite a new screening interface and recommends equivalent disclosure and lending safeguards.
→ The product is regulated according to its economic function rather than its technology branding.
Common mistakes
Treating branding as mechanism
Terms such as AI or blockchain describe broad technology categories and do not establish how a product creates value.
Assuming new means unregulated
The underlying behavior may closely resemble an activity that existing law already addresses.
Ignoring interested storytellers
Promoters, investors, and lobbyists may amplify claims because favorable narratives attract capital and legal exemptions.
Is it for you?
Best for
It is best for investors, policymakers, buyers, and leaders assessing heavily marketed technology products.
Not ideal for
It is not designed to forecast speculative returns when reliable evidence about the product or mechanism is unavailable.
From the transcript
“the value add comes from crypto has never been blockchain technology as a technology. It's been whipping up stories about that technology that have justified…”
“this construction of novelty is something that is done intentionally as a narrative.”
“the more you know about these technologies, the less impressive they seem”
From the episode
How AI Could Freeze Progress with Hilary Allen
Hilary Allen