The Past and Future of AI (with Dwarkesh Patel)

Episode Summary

Executive Summary: Russ Roberts and Dwarakish Patel discuss The Scaling Era and the recent history of AI, emphasizing that progress has come less from singular breakthroughs than from scaling compute, data, and training. They explore transformers, inference scaling, AI labor markets, fully automated firms, the limits of current models, and the social/emotional disruption AI may bring even if it boosts productivity dramatically.

Main Topics: AI progress as a story of scaling (Priority: 5/5): Patel argues the last six years of AI are best understood through exponential growth in compute and data, not just algorithmic genius. The transformer and next-token prediction became transformative because massive scale made their strengths visible. Transformers, pretraining, and inference scaling (Priority: 5/5): Roberts asks for a plain-English explanation of transformers and how recent models like o1/o3 and DeepSeek shifted focus from larger pretraining runs to reasoning-time compute, training on verifiable tasks, and inference scaling. Model differences, user experience, and practical utility (Priority: 3/5): The conversation compares Claude, ChatGPT, Gemini, and others in everyday use—brainstorming, translation, research, travel planning—and asks whether labs know why models differ and whether those differences matter to normal users. Automation limits: agency, computer use, and hallucinations (Priority: 5/5): A central concern is why models can solve hard math and coding problems yet still struggle with basic embodied or workflow tasks like booking flights, using computers reliably, or operating with common sense over long sequences. Fully automated firms, hive minds, and economic transformation (Priority: 5/5): Patel’s essay on fully automated companies leads to discussion of digital workers that can be copied, merged, distilled, and scaled. He argues the big change may be collective AI coordination rather than a single superintelligent agent. Human flourishing, sadness, and cultural change (Priority: 4/5): Roberts expresses that AI may make the world materially richer while making many ordinary human joys less available or less meaningful for future generations. Patel responds that people adapt to major transitions and that new forms of flourishing may emerge. Interview craft, curiosity, and discovering overlooked experts (Priority: 3/5): The two podcasters compare preparation styles, the art of eliciting candor, and the value of finding highly knowledgeable but less famous guests such as Sarah Payne. They stress rapport, challenge, and asking the real crux questions.

Key Arguments: AI progress from 2019 to 2025 is driven primarily by scaling compute and data, with architecture changes like transformers becoming valuable because scale made them workable. The biggest recent shift is from pure pretraining to inference scaling/reasoning models that spend more compute at answer time and are trained on verifiable tasks like coding and math. Current models still lack reliable agency and common sense: they can ace frontier benchmarks but struggle to consistently use computers, complete real-world workflows, or act as dependable assistants. The most important future development may be a world of many digital workers/hive minds, not one omniscient AI; digital beings can be copied, merged, and deployed at scale across the economy. Even if AI delivers huge productivity gains, it may also reshape daily life, institutions, and sources of meaning in ways that reduce some traditional human pleasures while creating new ones. The book’s interview-based format is meant to reveal that AI knowledge comes from many perspectives across the stack—hardware, software, economics, history, and philosophy—rather than one definitive theory. Podcasting and research both benefit from curiosity-driven synthesis and strong interview rapport; the best conversations come when interviewer and guest feel like peers debating real cruxes rather than performing for an audience.

Data Points: Book coverage window: 2019 to November 2024 - The book’s knowledge cutoff is explicitly stated as November 2024, while the conversation is on March 25, 2025. Compute growth per generation: ~100x - Patel says moving from GPT-2 to GPT-3 or GPT-3 to GPT-4 generally required about 100 times more compute. Human brain power use: ~20 watts - Used as a benchmark for the physical efficiency of human intelligence versus AI hardware. H100 power use: on the order of 1,000 watts - Patel contrasts GPU power draw with the brain to discuss efficiency gaps. Current AI investment: hundreds of billions of dollars - Patel describes AI as moving from an academic hobby a decade ago to massive industrial-scale investment. AI interview count by Patel: about 20 - Patel says he has interviewed roughly 20 people for the book/project. Podcast interview count by Roberts: about 15 - Roberts mentions he has done around 15 AI-related interviews. Podcast interview count by Patel: close to 100 - Patel says he has done nearly a hundred interviews as a podcaster. Herculaneum scrolls era: 79 AD - Patel cites the Vesuvius eruption and burned scrolls as an example of AI-enabled historical reconstruction. AI hard-task example: 5,000 pages of thinking - Patel refers to a reasoning benchmark where more inference-time thinking kept improving performance.

Pivotal Quotes: "the backdrop is just these big picture trends, these trends most importantly in the buildup of compute, in the buildup of data" — Dwarakish Patel: Patel explains why the last six years of AI should be seen as a scaling story rather than a story of isolated algorithmic breakthroughs. "we still don't know. It's almost entirely just a contingent empirical fact" — Dario Amodei, quoted by Dwarakish Patel: Patel recalls Anthropic’s CEO on why scaling works and why AI appears to gain intelligence from more compute. "Everyone is sleeping on the collective advantages AIs will have, which has nothing to do with raw IQ, but rather with the fact that they're digital." — Dwarakish Patel: Roberts reads from Patel’s essay on fully automated firms, prompting a broader discussion of AI’s structural advantages over humans.

Implications: AI may soon change not just productivity but institutions, work, and culture. Users should expect better tools, but also a world in which some human experiences become rarer while new forms of intelligence, coordination, and flourishing emerge.