The Bet with Buffett (Capital Allocators, EP.05)

Episode Summary

Executive Summary: Ted Seides and Patrick O'Shaughnessy dissect Seides’ famous charity bet with Warren Buffett: whether five fund-of-funds could beat the S&P 500. They conclude the headline—"hedge funds lost because fees are high"—is too simplistic. The real story involves valuation, market exposure, investor behavior, benchmarks, and the differences between paper returns and what investors actually earn.

Main Topics: Origins and structure of the Buffett bet (Priority: 5/5): Seides explains how the wager emerged from Buffett’s public comments, how letters negotiated the terms, why collateral and legal structure mattered, and why the bet ended up as a long-term charitable contract rather than a simple handshake. Why the S&P 500 was the real opponent (Priority: 5/5): The conversation emphasizes that Seides was largely betting against an expensive U.S. market at the start of the bet, not simply betting on hedge funds. He argues the S&P’s valuation made it vulnerable over a long horizon. Hedge funds versus the S&P: different exposures (Priority: 5/5): They unpack why equity long/short fund-of-funds and the S&P 500 are not directly comparable: hedge funds had lower net market exposure, more diversification, and different sector/geographic tilts, while the S&P was a concentrated U.S. large-cap equity bet. Fees, replication, and the limits of simple narratives (Priority: 4/5): Buffett’s fee critique is acknowledged as valid, but Seides argues fees alone do not explain the outcome. The discussion explores how factor replication and cheap products from firms like AQR have changed the market, while some hedge fund edge remains in niche or flexible strategies. Benchmarking, risk, and what counts as success (Priority: 4/5): The hosts debate whether total return, Sharpe ratio, or custom risk-adjusted benchmarks would be more appropriate. They conclude that benchmark choice heavily shapes conclusions and that alternative strategies should be evaluated relative to their intended role. Investor behavior and real-world returns (Priority: 5/5): A major theme is the gap between time-weighted performance and dollar-weighted investor experience. Seides notes that volatility and drawdowns often cause people to sell at the wrong time, meaning the average investor likely earned much less than the headline S&P return. Permanent capital and long-term ownership (Priority: 4/5): The episode closes with a broader reflection on permanent capital vehicles like Berkshire, Constellation Software, Fairfax, and family-owned business investing. Seides highlights the strategic advantages of patient capital and the opportunity set in permanent equity investing.

Key Arguments: The bet was not only about hedge fund fees; it was also a view that the S&P 500 was starting from a very high valuation and was therefore likely to underperform over the next decade. Equity long/short hedge funds are not directly comparable to the S&P 500 because they typically have lower market beta and broader geographic/size exposure. The market’s long-run valuation matters "eventually," even though timing is highly uncertain and outcomes can vary widely over shorter periods. Low-cost investing is appropriate for most people, but that does not mean the S&P 500 is always the right passive choice; global diversification may be more sensible. Many hedge fund strategies have been partially replicated with cheaper public-market products, reducing the case for paying high fees unless the manager has a real edge. Hedge funds still matter in certain niches and in less trafficked markets, but the bar is higher because competition, crowding, and lower rates have made the industry harder. The right benchmark depends on the strategy’s purpose; a simple S&P comparison is informative but imperfect, especially for absolute-return or hedged strategies. The real-world investor experience can differ dramatically from index returns because volatility induces bad timing decisions and behavior gaps. Permanent capital creates advantages for both allocators and operators: it can improve sourcing, patience, and strategic decision-making. In Seides’ view, the best investing edge is not necessarily size but allocator sophistication, resources, and the ability to access top talent and stick with it.

Data Points: Length of bet: About 9.5 years - The wager began on January 1, 2008 and was discussed in the context of nearly a decade later. Initial collateral: $1 million - The parties pre-funded the charity wager with a bond that would accrete to $1 million over 10 years. Initial collateral value: About $640,000 - Seides said the zero-coupon bond started around this level before compounding. Midway collateral value: About $950,000 - After four or five years, the bond had grown close to the target amount. S&P 500 decline in 2008: Down 37% - The market fell sharply during the crisis year after the bet started. Hedge fund drawdown in 2008: About 24% - Seides referenced hedge funds falling less than the S&P during the crisis, though still badly. S&P decline from bet start to Feb. 2009: Down 50% - Seides said that from the start of the bet through the trough, the S&P was cut in half. Shiller P/E level at the time of discussion: Around 30 - He noted that the market was at roughly 30 on the Shiller P/E, only the third time ever at that level. Average Shiller P/E historically: About 16–17 - Used to argue that the market was expensive relative to long-run norms. Correlation of Shiller P/E and future 10-year real returns: Around 0.7–0.8 - Used to support the argument that valuation is predictive over long horizons. Typical hedge fund market exposure: About 50% net exposure - Seides described equity long/short hedge funds as having roughly half the market exposure of the S&P. Short interest then vs. now: 2–3% historically vs. 15–20% today - Illustrated how crowding has made shorting harder over time. Active-to-passive flows: Majority into the S&P 500 - He argued that persistent inflows into passive indexing have supported large-cap U.S. equities. Berkshire underperformance period: First prolonged stretch in decades - Seides noted Berkshire lagged the S&P during this same era, underscoring how unusual the cycle was. Collateral compound growth: About 300% - He said the charity collateral invested in Berkshire compounded far better than the original bet basket.

Pivotal Quotes: "The reason really was, what bet was I making? ... I think the right analogy is to ask, which is the better sports team, the Chicago Bulls or the Chicago Bears?" — Ted Seides: Explaining that the wager compared different exposures, not apples-to-apples performance. "Price matters eventually." — Ted Seides: A core valuation principle used to justify why the S&P looked vulnerable at the bet’s start. "If you don't have an allocator's edge, and you don't think you should have it, you probably shouldn't play the game." — Ted Seides: On when alternatives or hedge funds make sense for institutions or individuals.

Implications: The episode argues investors should look beyond headlines: compare like with like, respect valuation and behavior, and avoid paying for complexity without an edge. It also reinforces the growing importance of permanent capital and selective, not blanket, use of alternatives.