Episode Summary
Executive Summary: The episode centers on Matt Kaplan’s book about scientists ridiculed or punished for being right, using history and COVID as case studies to argue that science is highly effective but structurally clunky. Kaplan and Shermer discuss replication, fraud, grant incentives, peer review, and how science should better tolerate uncertainty, replication, and failure rather than suppress unconventional ideas.
Main Topics: Science’s self-correction and its failures (Priority: 5/5): The conversation frames science as a powerful but imperfect system that often eventually corrects itself, though it can take decades or longer when consensus hardens and dissent is punished. COVID-19 as a test case for modern science (Priority: 5/5): They review what science got right during COVID—rapid global data sharing and vaccine development—and what it got wrong, including institutional overconfidence, suppression of dissent, and delayed correction. Historical examples of ridicule and exile (Priority: 5/5): Past cases such as Galileo, Semmelweis, Lister, Wegener, and Pasteur illustrate how social pressure, prestige, and career incentives can block correct ideas and reward bad behavior. Replication crisis and publication incentives (Priority: 4/5): The discussion emphasizes that journals favor novelty over replications, which undermines reliability; the guests argue for explicit funding and rewards for replication studies. Fraud, p-hacking, and academic pressure (Priority: 4/5): They distinguish outright fraud from p-hacking and selective reporting, arguing that intense competition for grants and prestige makes unethical behavior more likely. Reforming funding and peer review (Priority: 4/5): Kaplan proposes reforms such as better-funded peer review, replication support, and a ‘golden ticket’ for committees to back bold ideas that conventional review would otherwise reject. Open-minded skepticism and the ‘shelf’ model (Priority: 3/5): Rather than dismissing uncertain or unconventional ideas outright, they advocate shelving them at appropriate confidence levels and revisiting them as evidence accumulates.
Key Arguments: Science works best when it allows disagreement, replication, and civil criticism instead of mob-like attacks on dissenters. COVID showed both science’s power—especially rapid vaccine development—and its weaknesses, including institutional rigidity and delayed correction. The mRNA vaccine breakthrough depended heavily on Kati Karikó’s marginalized work and her collaboration with Drew Weissman, showing the importance of protecting unconventional research. Replication is essential to avoid building a house of cards; journals and funding systems should incentivize it directly. Outright fraud, p-hacking, and selective reporting are serious because they waste money, distort follow-up research, and can endanger people. Peer review and grant committees are biased toward safe, conventional projects; mechanisms are needed to reward bold but plausible ideas. Historical cases show that correct theories can be rejected for decades due to status, pride, and professional self-interest. Unproven ideas should not be treated as true or false in a binary way; they can be provisionally shelved and reassessed over time.
Data Points: COVID vaccine timeline: under a year - Kaplan notes that developing a COVID vaccine in under a year was unprecedented. Replication of old vaccine timelines: measles, mumps, rubella, whooping cough all took years - He contrasts COVID vaccine speed with historical vaccine development timelines. Year of paleontology conference conflict: 2014 - Kaplan recalls witnessing an aggressive attack on a young researcher at a 2014 conference. Time for Lister’s exile: 20 years - Shermer and Kaplan discuss Joseph Lister being forced into exile after advocating antiseptic ideas. Maternal mortality from puerperal fever: roughly 1 in 10 women - Oliver Wendell Holmes’s work on puerperal fever is described as identifying a disease that killed about 10% of affected women. Retracted papers in medical research: 18,000 in 2023 - Kaplan cites a striking number of retractions as evidence of serious integrity problems. Howard Hughes-style grant duration: 7–8 years - These grants are praised for allowing long-term, high-risk research. Grant funding examples: $500,000 - Used as an example of how a bad or massaged result can waste substantial public research funds. Private prize amount: $100,000 to $250,000 - Referenced in the discussion of the Kremer prize for human-powered flight across the English Channel. Current rhinoceros population: 43 - Kaplan mentions only 43 of a particular rhino species remain. Fruit species tied to rhino ecology: 46 or 47 species - The rhino is described as helping germinate dozens of fruit species in its ecosystem.
Pivotal Quotes: "We had a vaccine in record time, under a year, and that had never been accomplished before." — Matt Kaplan: Discussing what science got right during COVID. "We need to be having the conversations rather than destroying and exiling people." — Matt Kaplan: Arguing for civil scientific debate instead of attacking dissenters. "If you can say, look, this is an idea. This is an idea that might have merit. We don't have the mechanisms yet... it should be part of the literature." — Matt Kaplan: Explaining his ‘shelving’ approach to uncertain scientific claims.
Implications: Listeners are urged to support replication, reward bold research, and resist treating science as infallible or binary. Better incentives and more humility could speed genuine breakthroughs while reducing waste and harm.