Two-Dimensional ETF Performance Scorecard
Judge index funds by market quality and delivery of the promised index outcome
- Difficulty
- Moderate
- Time to result
- ~ongoing to results
- Steps
- 5
- Confidence
- 98%
The scorecard evaluates an ETF on two connected dimensions. First, market quality measures how effectively investors can trade the vehicle, including secondary-market volume, liquidity, premiums and discounts, transparency, and resilience under stress. Second, index-outcome delivery measures whether the portfolio supplies the exposure investors selected, despite indexes containing securities that may be costly or impractical to replicate perfectly. Managers therefore optimize holdings and operations to reduce friction while continuously monitoring the ETF's exchange behavior. A fund succeeds only when both dimensions hold: accurate exposure without dependable tradability is insufficient, while excellent liquidity cannot compensate for failure to deliver the stated index objective. The framework turns the vague idea of passive performance into a practical operating scorecard.
Origin
Extracted from Masters in Business during Samara Cohen's explanation of her responsibilities as BlackRock's chief investment officer for ETFs and index investments.
Core principles
- 01Index investing still requires active operational management
- 02Benchmark-relative return alone cannot measure ETF performance
- 03Investors should receive the exposure described by the index objective
- 04Liquidity, transparency, and premium-discount behavior are investment outcomes
- 05Performance must be monitored in both normal and stressed markets
How to run it
- 1
Specify the promised exposure
Write down the index objective and the outcome investors reasonably expect from owning the product. Identify any replication constraints created by the number, liquidity, or accessibility of index constituents.
Pro tip Express the objective in observable terms rather than relying only on the index name.
Watch out An ambiguous objective makes subsequent performance judgments unreliable.
- 2
Measure market quality
Track secondary-market volume, spreads, liquidity, premium-discount behavior, and transparency. Review these measures across ordinary and stressed trading conditions.
Pro tip Include stress-period observations because normal-market averages can hide structural weaknesses.
Watch out Do not mistake high assets under management for consistently strong trading quality.
- 3
Assess index delivery
Measure how closely the portfolio produces the intended index outcome after replication constraints and operating friction. Examine tracking behavior through rebalances, corporate actions, and market disruptions.
Pro tip Separate unavoidable index implementation costs from correctable operating problems.
Watch out Perfect security-by-security replication may be impractical for large or illiquid indexes.
- 4
Diagnose weak dimensions
Investigate any deterioration with portfolio teams, product engineers, market makers, and other ecosystem partners. Determine whether the cause lies in holdings, operating structure, hedging difficulty, or exchange conditions.
Pro tip Use several metrics together rather than treating one premium or spread observation as decisive.
Watch out Optimizing one dimension can unintentionally damage the other.
- 5
Improve and monitor continuously
Change portfolio processes or product structure where appropriate, then verify that both index delivery and market quality improve. Continue monitoring throughout the product's lifetime.
Pro tip Retain before-and-after measurements for every structural change.
Watch out ETF performance management does not end when the fund launches.
In the wild
A product committee reviews a bond ETF whose underlying securities trade infrequently. It compares the ETF's exchange volume, price formation, spreads, and premium-discount behavior with activity in the underlying bonds, then checks whether the fund continues to deliver its intended credit and duration exposure. The review recognizes that visible ETF prices may reflect current market information faster than stale bond quotes.
→ The committee distinguishes genuine product failure from the ETF's role in revealing prices during an illiquid market.
An ETF begins trading wider than comparable products. Managers confirm that index tracking remains sound but learn from market makers that one structural feature makes hedging difficult. Product engineers modify how the market interacts with the fund and monitor subsequent spreads, liquidity, and tracking behavior.
→ The product preserves its index exposure while improving tradability for investors.
Common mistakes
Measuring tracking alone
A fund can follow its index while still delivering poor liquidity, transparency, or premium-discount behavior to investors.
Treating index management as passive
Replication, rebalances, corporate actions, market quality, and operational friction require continuous human and technological oversight.
Ignoring stressed markets
Normal conditions do not reveal whether the ETF structure remains useful when liquidity and price formation deteriorate.
Is it for you?
Best for
It is best for ETF managers, product committees, institutional allocators, and analysts evaluating index products.
Not ideal for
It is not ideal for evaluating unconstrained active funds whose objective is persistent benchmark outperformance.
From the transcript
“And in the case of E t F S and index, it's two things. It's first what we call market quality.”
“And the other part is delivering on those index outcomes”
“you need to optimize to deliver that index outcome with as little friction as possible.”
From the episode
Samara Cohen on Managing ETFs (Podcast)