Episode Summary
Executive Summary: The episode critiques psychology’s credibility crisis and explores a major fraud scandal in business-school psychology. Derek Thompson interviews social psychologist Adam Mastriani about replication failure, weak practical payoff, and the field’s overconfidence, then Atlantic reporter Dan Engber about Francesca Gino, a “many co-authors” audit, and suspicious datasets that suggest deeper structural problems in academic incentives and fraud detection.
Main Topics: Psychology’s replication crisis (Priority: 5/5): The conversation opens with the idea that many famous findings in social psychology—such as power poses and ego depletion—failed to replicate, undermining confidence in the field’s core claims. What psychology has genuinely contributed (Priority: 4/5): Mastriani argues psychology’s best achievement is overturning folk psychology and exposing how intuition about behavior, bias, and situational influence often misleads us. Three critiques of social psychology (Priority: 5/5): The field is criticized for failing to replicate, for producing findings that don’t matter much even when true, and for turning out technologies and interventions that often underperform or remain unpredictable. Business-school psychology fraud scandal (Priority: 5/5): Engber explains the Francesca Gino case, where alleged fraud in multiple papers triggered a broader audit of business-school psychology and raised questions about the prevalence of misconduct. The Many Co-Authors Project and forensic auditing (Priority: 4/5): Researchers launched a large-scale audit of Gino-linked papers to assess data integrity, which exposed suspicious patterns in additional studies and demonstrated how one scandal can reveal a wider pattern. Incentives, p-hacking, and the culture of publication (Priority: 5/5): Both guests discuss how questionable research practices, prestige incentives, and competitive pressure can encourage exaggeration or misconduct while making honest correction costly. What progress in psychology should mean (Priority: 4/5): The episode ends by asking whether psychology should aim for hard-science style falsification, better interventions, or simply a more honest and exploratory discipline oriented around better questions.
Key Arguments: Psychology has done real work by overturning folk beliefs about the mind, but many celebrated findings rest on weak foundations. Replication failure is important, but the deeper problem is that many findings were not practically meaningful even before they were shown to be unreliable. The field often asks the wrong first question: not whether a result is true, but whether it would matter if it were true. Psychology’s best-known conceptual buckets—cognitive bias and situational influence—are useful, but some iconic demonstrations are too fragile to support the weight placed on them. Big Five personality testing is not dramatically better than folk or non-scientific frameworks at predicting certain real-world outcomes. Behavioral nudges can work in some cases, but large-scale trials often show mixed or weak effects, and experts struggle to predict which interventions succeed. Fraud and questionable research practices are made more likely by a publication system that rewards flashy, positive results and punishes uncertainty. The Gino scandal suggests that misconduct may be more widespread than people assume, but the field lacks a true random audit of fraud prevalence. Open-science reforms and data provenance tracking could improve trust, but they impose real burdens, especially on junior and less-resourced researchers. Psychology may be in an early developmental stage, not a failed one; the danger is premature certainty rather than the existence of doubt.
Data Points: Average lying rate: Once every 10 minutes - Used in the promo for Brian Phillips’ new podcast Truthless to frame deception as common human behavior. Replication survey size: 100 published psychological experiments - Brian Nosek’s 2015 effort to have hundreds of psychologists re-run experiments as part of the replication crisis. Time frame of replication crisis: About 9–10 years - The episode notes psychology has lived with the replication crisis for nearly a decade. Core cognitive-bias history: About 50 years - Mastriani dates the modern cognitive-bias paradigm back to Kahneman and Tversky in the 1970s. Big Five factors: 5 - The personality framework includes openness, conscientiousness, extroversion, agreeableness, and neuroticism/emotional stability. Many Co-Authors Project scope: 134 papers - The project reviewed Francesca Gino-linked publications after fraud allegations emerged. Initial Gino fraud allegations: 4 papers - Data Colada and Harvard’s internal investigation focused first on four papers tied to Gino. Authored papers attributed to Gino: About 138 - Engber says Gino co-authored roughly 138 papers with prominent academics and younger scholars. Ritual effect paper sample: Study 1B heart-rate data - A suspicious dataset in a paper about rituals lowering anxiety showed a strangely ordered run of heart-rate values. Nudge mega-study interventions: 53 interventions - A gym-attendance mega-study tested many nudges simultaneously to see which worked. Nudge mega-study result: About half failed - Roughly half the 53 interventions produced no detectable effect. Prediction accuracy: Non-experts outperformed experts in at least one mega-study - In a vaccine-uptake prediction task, experts were less accurate than non-experts at anticipating which interventions would work. Television watching: 2.7 hours per day - Mastriani uses this to pose a basic research question about why people enjoy watching ordinary activities on TV.
Pivotal Quotes: "We’re doing all this research in psychology, but are we learning anything? We run these studies and publish these papers and then what? The stack of papers just gets taller?" — Adam Mastriani: His critique of psychology’s productivity and whether the field is accumulating real knowledge. "Would it matter if it were true?" — Adam Mastriani: He argues the field asks about truth before asking whether a result has any practical significance. "I’m sorry, I would have just back up for one half step just to say that some of those big topics you talked about, paradox of choice, for example, and procrastination and creativity, those are not specifically represented in the Medico Authors Project." — Dan Engber: Engber clarifies that the Many Co-Authors Project is about Gino-linked papers, not every famous business-school psychology topic.
Implications: The episode suggests psychology needs deeper reform: stronger transparency, better incentives, and more humility about what its findings can support. For listeners, it’s a warning to treat flashy behavioral claims cautiously and a reminder that scientific progress often begins with embarrassment.