Our Noisy Minds

Episode Summary

Executive Summary: The episode explores Daniel Kahneman’s concept of “noise”: unwanted variability in human judgment that causes inconsistent, costly decisions across insurance, medicine, asylum, and criminal justice. It contrasts noise with bias, explains why noise is harder to detect and fix, and argues for audits, averaging, decision hygiene, and sometimes algorithms to reduce randomness and improve fairness.

Main Topics: Defining noise in human judgment: Kahneman explains noise as unwanted variability in judgments or decisions when different people—or the same person at different times—reach different conclusions from the same information. Noise in business and insurance: An insurance company example shows underwriters giving wildly different quotes for similar clients, revealing large hidden costs and customer unfairness. Noise in medicine and diagnosis: Doctors, radiologists, and fingerprint examiners can disagree on the same evidence, causing misdiagnosis, inconsistent treatment, and harmful variability in care. Noise in criminal justice and sentencing: Sentencing disparities can be extreme even for similar offenses; noise audits exposed large variation, prompting reforms that reduced variability but triggered resistance from judges. Noise versus bias: The episode distinguishes statistical noise from intentional bias, arguing that noise is harder to tell stories about and therefore often ignored in public debate. Tools for reducing noise: Suggested solutions include noise audits, averaging independent judgments, decision hygiene, and in some domains using algorithms as a more consistent decision aid or final arbiter. Human resistance to mechanization: Judges, doctors, and others often dislike rules and algorithms because they reduce discretion, even when the evidence shows those tools improve consistency and accuracy.

Key Arguments: Wherever there is judgment, there is noise; human decisions are often more variable than people realize. Noise is costly because even small deviations in repeated decisions compound into major financial, medical, and legal harms. Noise differs from bias: bias has an identifiable cause or story, while noise is random variability visible mainly in aggregates. Noise audits can reveal hidden inconsistency by giving professionals identical cases and comparing their responses. Averaging independent judgments reliably reduces noise, though it does not necessarily reduce bias. Algorithms can outperform humans partly because they are not noisy and produce the same output on the same input. Reducing noise often also reduces bias because it constrains arbitrary discretion and opportunities for discrimination. People resist anti-noise reforms because they prefer discretion, feel their judgments are already correct, and dislike being “robotized.” Decision hygiene offers a general, prevention-oriented approach to improving judgment, analogous to public health measures like handwashing. When organizations have repeated judgments by interchangeable decision-makers, they should measure noise and work with decision-makers to improve consistency rather than impose rigid rules from above.

Data Points: Time-counting experiment: 5.43 seconds, 5.2 seconds, 5.59 seconds - Shankar Vedantam’s opening illustration of small judgment errors while counting five seconds without looking at the phone. Expected variability in insurance underwriting: 10% - Executives’ estimate of differences between underwriters before measurement. Measured variability in insurance underwriting: 55% - Actual variability found in one insurance company’s underwriting judgments. Asylum grant rate by judge: 88% vs 5% - Miami asylum-court study showing one judge granting asylum to most applicants and another to very few. Judges in a sentencing noise audit: 208 federal judges - Judge Marvin Frankel–inspired study of sentencing variability. Cases in the sentencing audit: 16 cases - Federal judges evaluated the same set of cases to assess noise. Average sentence in the audit: 7 years - Mean sentence assigned across the evaluated cases. Probable difference between two judgments: Over 3 years - Estimated spread between two judges’ sentences for the same case. Cost of medical mistakes: More than $17 billion per year - Estimate cited for the economic burden of medical errors in the U.S. Patients experiencing hospital mistakes: 1 in 3 - Statistic cited in the discussion of medical errors during hospital stays. Crowd estimate of ox weight: Within 2 pounds of correct - Francis Galton’s 1907 county-fair averaging example. Actual weight of prize ox: 1,198 pounds - Correct weight used in the Galton example. Noise reduction with 4 independent judges: One-half - Kahneman’s explanation of how averaging independent judgments reduces noise. Noise reduction with 100 independent judges: 90% - Further reduction from averaging many independent judgments. Bail algorithm effect: 42% reduction in jail population without increasing crime risk - Study described using an algorithm to inform bail decisions. Algorithm-human performance gap: 25% to 90% better - Berkeley Dietvorst excerpt describing how algorithms often outperform humans.

Pivotal Quotes: "Wherever there is judgment there is noise, and there is more of it than you think." — Daniel Kahneman: Core definition of noise given early in the interview. "The mind is hungry for causes. And that leads us very naturally to think in terms of biases." — Daniel Kahneman: Explaining why people prefer bias narratives over statistical variability. "If it takes a mechanistic science to produce justice, then I think we should seriously consider some mechanistic science." — Daniel Kahneman: His response to judicial resistance against sentencing guidelines and reduced discretion.

Implications: Noise audits, averaging, and decision hygiene can make institutions fairer and more accurate. Listeners should expect variability in human judgment, question anecdotes, and support systems that measure and reduce inconsistency—even when they feel less intuitive than pure discretion.