Ben Cohen on the Science of Winning Streaks

Episode Summary

Executive Summary: Ben Cohen discusses The Hot Hand, arguing that streaks in basketball and beyond are real, but often misunderstood because randomness and sample bias distort intuition. The conversation explores the history of hot-hand research, its reversal, and its implications for sports analytics, player valuation, random systems, and creative or professional productivity.

Main Topics: The hot hand debate: Cohen explains the classic dispute over whether success breeds more success, using basketball as the main case study and framing hot hand as a broader human-behavior question. The Gilovich-Tversky paper and its backlash: The 1985 Stanford paper argued people wrongly perceive patterns in randomness; Cohen says it became conventional wisdom despite early resistance and partly because the underlying psychological insight remained valid. Josh Miller and Adam Sanjurjo’s statistical correction: Their work showed the original hot-hand analysis had a subtle sampling bias, meaning people were misreading the probability of success after success. Sports analytics and roster construction: The discussion connects hot-hand thinking to NBA strategy, plus-minus metrics, three-point optimization, and the rise of role players such as Shane Battier and Duncan Robinson. Hot streaks in creativity and careers: Cohen broadens the concept beyond basketball to show that writers, directors, scientists, and other creators often produce their most memorable work in clusters. Randomness, product design, and user psychology: Examples like Spotify shuffle and NBA Jam show that systems often need to be made less random to feel more natural to humans. Writing, reporting, and research process: Cohen reflects on balancing daily journalism with a book project, the importance of sources and acknowledgments, and how analytical thinking improves sports storytelling.

Key Arguments: The hot hand is not just a basketball myth; it describes a real tendency for success to cluster, though not in the exaggerated way people imagine. The original Gilovich-Tversky paper was still important because it correctly identified that humans over-detect patterns in randomness. Miller and Sanjurjo’s correction changed the interpretation of the data: what looked like evidence against the hot hand was partly a statistical artifact. In modern basketball, analytics have rewarded players and teams that identify efficiencies, especially three-point shooting and undervalued role players. Systems designed for humans often need to be less random than true randomness to feel right, as seen in music shuffling and game design. Creative careers often contain hot streaks; memorable work is frequently concentrated in periods when talent, timing, and opportunity align. Valuation in sports is often distorted by outdated metrics, and advanced stats like plus-minus better capture a player’s real impact.

Data Points: Book length: 280 pages - Barry Ritholtz describes Cohen’s book as a compact but comprehensive treatment of the topic. Original hot-hand paper year: 1985 - Cohen references the Gilovich, Vallone, and Tversky paper as the foundational work in the debate. Time to write the book: 3 years - Cohen says the idea expanded after his reporting and became a long-form book project. Hot-hand bias effect: about 12–13 percentage points - Cohen says the corrected interpretation changes the perceived size of the effect substantially. Steph Curry comparison: 12 points ≈ difference between Steph Curry and a league-average shooter - Used to illustrate how large a 12-point swing in shooting efficiency can be in NBA terms. Steph Curry game vs. Knicks: 54 points, 11/13 from three, 48 minutes - The example is used as a signature hot streak and a turning point in Curry’s career. Warriors shot share from three: about 35% of shots - Cohen notes the NBA’s increasing reliance on three-point attempts. Three-point share a decade earlier: about 22% of shots - Used to show the rise in three-point volume over time. Potential team upper bound for threes: 65% of team shots - An analytics source at Sloan suggests the theoretical ceiling for three-point shot frequency. NBA Jam revenue: $1 billion in quarters in the first year - Cohen cites the arcade game’s success as evidence of how compelling the hot-hand fantasy is. Daryl Morey tweet fallout estimate: about $400 million - Adam Silver’s estimate of the financial impact of the Hong Kong tweet controversy on NBA-China relations. Steph Curry plus-minus impact: +4,000 points over five years in his minutes - Cohen cites this as evidence of Curry’s extraordinary on-court value. Cohen’s Wall Street Journal tenure: since 2009 - He explains he stayed after a sports page was launched at the Journal. Book research volume: ~500-page binders of scholarly papers - Cohen says he assembled and reviewed extensive literature to prepare the manuscript. Podcast archive mention: 300+ conversations - Ritholtz references the depth of prior episodes, including many on behavioral finance and decision-making.

Pivotal Quotes: "What I like to think of it as when success leads to more success." — Ben Cohen: Cohen’s simplest definition of the hot hand. "The real one is the one that looks fake." — Ben Cohen: Explaining why true random sequences often appear unnatural to humans. "The movie I want to make is The Princess Bride." — Ben Cohen: Cohen uses Rob Reiner’s career as an example of a hot streak enabling unlikely projects.

Implications: Listeners should rethink how they judge streaks, randomness, and value creation. The discussion suggests analytics can reveal hidden advantages in sports, business, and creativity—and that intuition alone often misreads when performance is truly “hot.”