Joshua Miller Discusses the Hot Hand Phenomenon (Podcast)

Episode Summary

Executive Summary: The episode is a deep dive into Joshua Miller and Adam Sanjurjo’s challenge to the classic “hot hand” critique. They argue the original analysis mis-specified the benchmark: in fixed samples, even fair coins and consistent shooters should appear worse after streaks because of selection effects. Reanalyzing the data reveals evidence of genuine streakiness and broader lessons about probabilistic inference, bias, and reproducibility.

Main Topics: The hot hand debate revisited: Barry Ritholtz interviews Joshua Miller about the famed 1985 Gilovich-Tversky paper and why its conclusion—that the hot hand is a myth—has now been challenged. Miller explains that the central issue is not whether streaks exist in real time, but how they should be measured in retrospective data. The statistical error in the original benchmark: The key methodological claim is that the original paper compared shooting percentages after made vs. missed streaks without accounting for selection effects in finite samples. Miller argues that even a perfectly random or constant shooter should look worse after streaks, so equal percentages are actually evidence of hot-hand behavior. Coin flips, finite-sample bias, and the Monty Hall analogy: Miller uses coin flips to show the counterintuitive result that after analyzing a finite sequence, the next flip following prior heads can be more likely to be tails in the observed subset. He links this to the Monty Hall problem and the principle of restricted choice to make the intuition accessible. Modern shot-tracking and live-game complexity: The discussion addresses how modern basketball data now includes shot difficulty, defender proximity, and optical tracking. These tools make it possible to separate true streak effects from defensive adjustments, which were difficult to control for in 1985. Behavioral economics, decision-making, and institutional data: Miller situates his work within behavioral economics and judgment/decision research, emphasizing that information presentation, attention, and institutional context affect choices in finance, sports, and policy. He also notes that modern research increasingly uses richer data than simple lab choices. Reproducibility, pre-registration, and simulation: The conversation ends on broader research-method lessons: researchers should pre-register hypotheses, publish negative results, and use fake-data simulation to test whether statistical methods produce misleading conclusions under different assumptions.

Key Arguments: The original hot-hand paper did the arithmetic correctly but used the wrong benchmark for finite samples, so its null result was misleading. If you analyze streaks in a finite dataset, a fair process can look anti-hot because prior outcomes are being conditioned on and effectively removed from the remaining pool. The fact that players do not shoot worse after streaks, once the selection bias is accounted for, is evidence of a genuine hot hand. Modern data on shot difficulty and defenders allows better separation of player skill from defensive reaction than was possible in the 1980s. Coin-flip simulations show that retrospective analysis can generate surprising asymmetries even when the underlying process is perfectly random. The hot hand can arise from multiple mechanisms—confidence, reduced overthinking, defensive pressure, and team effects—so the term should not be treated as one single causal explanation. The same kind of selection bias may apply to clustering and spatial problems, not just time series or sports. Good statistical practice requires simulation, pre-registration, and attention to what happened to the data you did not publish.

Data Points: Hot-hand effect size: About 10 percentage points - Miller says the corrected analysis implies players may be roughly 10 points better after made streaks than after missed streaks, a large NBA-level difference. Craig Hodges streak: 19 makes in a row - Used as an example of a clearly hot shooter in three-point contest data. Craig Hodges shooting volume: Over 500 shots - Miller says Hodges had over 500 shots in their three-point contest data, giving enough volume to detect streak effects. Original paper year: 1985 - The Gilovich-Tversky paper that argued the hot hand was a myth. Miller’s discovery date: February 2015 - He says he and Adam Sanjurjo found the key result in February 2015. First paper online: June 2015 - Their initial paper was posted online in June 2015. Journal publication: Econometrica (November issue, online first) - Miller notes the paper was formally published in a top economics journal. Contest shot sample size: Roughly 100 shots for many players - He notes that many three-point contest players have only around 100 shots, limiting statistical power. Chance benchmark in coin flip: 50-50 - Used throughout to explain why retrospective conditioning changes observed frequencies without changing underlying randomness. Observed retrospective bias: Around 40% instead of 50% - Miller explains that selecting flips conditionally on prior heads can make the next observed outcome look like 40% rather than 50%. Prediction advantage in betting data: About 7 percentage points - He says reanalysis of betting/prediction data showed subjects were about 7 points better when predicting a make than a miss. Academic timeline: 2009 move to Italy; later to Spain - He went to Bocconi in Milan after the 2008–09 academic job crisis, then later joined the University of Alicante.

Pivotal Quotes: "I have been in a thousand arguments, won them all, but convince no one." — Amos Tversky (as quoted by Miller): Referenced to illustrate the stubborn resistance researchers faced when presenting the original hot-hand skepticism to practitioners. "You have to go out and simulate or go sit down and really calculate." — Joshua Miller: Miller explains that understanding whether a statistical result is meaningful requires generating fake data or alternative worlds, not just looking at one empirical dataset. "It's not simply that we found that the original analysis was invalid... you find evidence everywhere." — Joshua Miller: He summarizes the broader implication that the corrected method replicates across multiple datasets, not just the original example.

Implications: The episode suggests that widely accepted statistical “myths” can persist because of flawed benchmarks, not flawed intuition. For sports analytics, finance, and research broadly, it underscores the need for finite-sample corrections, simulation, pre-registration, and careful interpretation of streaks and clusters.