Larry Swedroe: 'The Pool of Victims' Is Expanding

Episode Summary

Executive Summary: Larry Swedroe argues that disciplined, evidence-based investing should focus on persistent, pervasive, robust factors while avoiding overconfidence, data mining, and performance chasing. He explains why popularity can erode premiums, defends diversification across factors and asset classes, critiques risky credit and speculative trading, and cautions retirees to prioritize spending safety over higher withdrawal rates.

Main Topics: How to evaluate academic investing research: Swedroe explains his framework for separating useful findings from noise, emphasizing five criteria—premium, persistence, pervasiveness, robustness, intuition, and transaction-cost survivability—plus outside expert review to reduce confirmation bias. Factor investing and what qualifies as a real premium: He discusses momentum, profitability/quality, value, and size, arguing that factors must work across regimes and measures, and that using multiple metrics is better than overfitting to one historical winner. When popular strategies decay: Swedroe revisits beliefs he changed on commodities and low beta/quality, arguing that popularity and large inflows can compress or eliminate expected premiums even when a strategy once worked. Risk, return, and investor behavior: He challenges the idea that more risk always means more reward, highlighting behavioral biases, lottery-stock demand, and limits to arbitrage as reasons mispricings persist. Portfolio construction and fixed income: He recommends a barbell approach: take risk in equities or equity-like alternatives, keep the defensive sleeve very safe, and avoid high-yield bonds as a substitute for true ballast. Retirement withdrawals and spending discipline: Swedroe defends a conservative 3% withdrawal baseline and says retirees needing more should adjust lifestyle, work longer, or downsize rather than chase higher-return risk. ESG and speculative trading opportunities: He says ESG screening may eventually lower expected returns, though flows could produce temporary outperformance; he also thinks retail speculation creates opportunities because retail investors often buy what institutions sell.

Key Arguments: Swedroe uses a strict evidence screen for factors: a genuine premium must be persistent, global, robust across definitions, intuitively explainable, and able to survive trading costs. He believes factor premia can decay due to popularity, regime shifts, or cash flows, even when the underlying logic remains sound. Commodities became too popular and moved into sustained contango, destroying the expected return case that once justified holding them. Low beta and quality may have worked historically, but he views them as more vulnerable to crowding and growth-regime headwinds than core factors like value and momentum. Investor behavior, not pure risk, helps explain why lottery-like stocks and speculative trades persist despite poor long-run outcomes. Diversification should be across distinct sources of risk, not just within a market-cap-weighted portfolio that is effectively dominated by market beta. High-yield bonds are not a true safe asset; they behave like equity risk in crises and offer little credit premium after fees. A 3% withdrawal rate is prudent because sequence risk and longevity risk make higher withdrawals dangerous for many retirees. ESG tilts may reduce expected returns in equilibrium because screening changes demand and raises prices without changing earnings. Retail speculation on platforms like Robinhood can enlarge the pool of mispriced securities, but only for investors with genuine edge and discipline.

Data Points: Assets under management: $50 billion - Buckingham Wealth Partners size, cited while describing access to academic research and experts. Active managers beating benchmark 20+ years ago: ~20% pre-tax / ~10% after-tax - Swedroe contrasts historical active-management success rates with today’s lower odds. Active managers beating benchmark today: ~2% - He says current studies show the proportion of successful active managers has collapsed. Commodities collateral yield at the time: 2% to 3% - Historical TIPS yields when he expected a commodities allocation to modestly outperform T-bills. Target excess return from commodities expected initially: 1%+ over T-bills - He expected a small premium plus diversification/inflation hedging before financialization changed returns. Typical commodity allocation recommendation: 3% to 5% - He notes his prior advice on adding commodities to portfolios. Low beta performance regime dependence: Works in value regimes; negative in growth regimes - His explanation for why low beta did not make the cut as a core factor. Momentum formation/holding periods: 3, 6, 9, 12 months - He says momentum remains robust across these lookback periods. Small/value underperformance window: Since October 2016 - He argues most of the nearly two-decade underperformance is concentrated in recent years. Value drawdowns versus growth: More than 30% historically; current drawdown about 50%+ - He compares the current slump to prior historical episodes. Value stock profitability gap historically: ROAs 4%-5% lower; ROEs 11%-12% lower - Historical quality/profitability difference between value and growth stocks. SP 500 underperformance vs T-bills: Three periods of at least 13 years - He cites 1929-43, 1966-82, and 2000-12 as examples that long slumps are normal. Japanese stocks negative return span: 30 years - Used to show that very long stretches of poor returns can happen. Long-term Treasury outperformance vs large-cap growth: 40 years (1969-2008) - Illustrates that even defensive assets or categories can outperform unexpectedly for decades. Portfolio risk share in 60/40: About 85% market beta risk - He argues the risk is more concentrated than many investors assume. Retail investor annual lottery spending: About $400 per year - He references lottery behavior as analogous to speculative stock-picking. Alternative lending expected return: 6% to 7% - Example of an illiquidity premium asset he views as potentially superior to high-yield bonds. Alternative lending volatility: About 5% - Compared with roughly 20% for stocks. Stock market volatility: About 20% - Used to compare against alternative lending's lower volatility. COVID drawdown example for alternative lending: Down 3% to 4% - He says the strategy held up much better than equities during the pandemic crash. Equity drawdown in COVID example: Down 34% - Benchmark comparison for the same period. Recommended retirement withdrawal rate: 3% - His baseline for a conservative retirement plan. Illustrative higher withdrawal scenario: 4% with ~80% odds - He uses this to show that better odds can still be unacceptable given tail risk. State insurance-guarded annuity example: 5 years at 3% - An example of a safer cash-like fixed-income alternative he purchased.

Pivotal Quotes: "popularity killed that golden goose" — Larry Swedroe: His explanation for why commodities stopped working as an attractive diversifying investment. "you want to hyper-diversify, and that means there'll always be some part of your portfolio that's doing poorly" — Larry Swedroe: His core philosophy on diversification and accepting short-term underperformance. "I don't care what you think the odds are. You have to consider the risk of being wrong" — Larry Swedroe: His argument for conservative retirement withdrawal rates and tail-risk awareness.

Implications: Listeners should expect long stretches of underperformance in even sound strategies and build portfolios around evidence, diversification, and patience. For firms and investors, crowding, flows, and behavior can erase premiums, making discipline and plain-vanilla risk control more valuable than chasing yield or recent winners.