Moonshots with Peter Diamandis
Moonshots with Peter Diamandis

Should we slow down AI progress? | MOONSHOTS #288

The mates sit down with Emad Mostaque to discuss mounting warnings from AI labs, a researcher’s claim that we’re “gambling with our lives,” calls to slow AI development, and the accelerating race toward superintelligence. Get access to metatrends 10+ years before anyone else - https://qr.diamandis.c

Topics Discussed

Episode Summary

Executive Summary: The episode argues that AI has entered a new phase where data quality, compute efficiency, and test-time scaling are rapidly turning “hard” scientific problems into solvable engineering tasks. The hosts debate the credibility and politics of AI doom warnings, highlight breakthroughs in math, biology, and world models, and emphasize that enterprise data, compute access, and regulatory strategy are becoming decisive competitive moats.

Main Topics: AI safety, doom, and internal lab dissent (Priority: 5/5): A long debate centers on resignations, public warnings, and whether AI labs are acting responsibly. Some hosts view the rage-quits and risk claims as virtue signaling or political theater, while others argue the labs genuinely fear catastrophic outcomes and that policy will soon react strongly. Data as the real AI moat (Priority: 5/5): The group argues that better training data, data pipelines, and proprietary data matter more than architecture advances. They frame data assets as highly valuable, time-sensitive, and potentially worth more than the core company itself. Math and science being 'cooked' by AI (Priority: 5/5): OpenAI’s reported Navier-Stokes progress and the broader idea of AI bulk-solving classic challenges are used to argue that mathematics and science are entering an era where compute-rich agent swarms can outperform human researchers on many verifiable tasks. Compute, GPUs, and memory as the new scarce infrastructure (Priority: 4/5): The panel discusses GPU rental economics, HBM bottlenecks, and data-center planning. They argue chips are now revenue-producing assets and that scarcity in memory and compute will shape investment, industrial policy, and enterprise strategy. Longevity and genomics breakthroughs (Priority: 5/5): AI-designed drugs and DeepMind’s genome-wide variant prediction are presented as examples of biology becoming programmable. The hosts frame these as evidence that longevity escape velocity and genotype-to-phenotype mapping are already emerging. World models, video, and robotics (Priority: 4/5): ByteDance’s push into spatial/world modeling and the convergence of video generation with robotics are presented as signs that visual intelligence will become the next major frontier, especially for embodied AI and consumer/media products. Economic disruption and redistribution (Priority: 5/5): Anthropic’s economic impact report is used to forecast massive GDP growth, labor displacement, and social instability. The panel argues governments will need new redistribution mechanisms, UBI-like schemes, and updated macroeconomic models.

Key Arguments: Better data improves AI more than better architecture because curated datasets are proprietary, hard to copy, and often the true moat. AI labs are solving milestone scientific problems so quickly that many grand challenges are shifting from human ingenuity to machine compute. The bottleneck is moving from intelligence to physical deployment: chips, memory, energy, data centers, and real-world manufacturing. Safety warnings from insiders may be sincere, but they are also deeply political, strategic, and sometimes self-serving. Enterprise proprietary data has a short shelf life as a moat, but it can still be monetized if organizations act quickly. AI-driven biology is moving from prediction to design, making drug discovery, aging research, and genomics far more programmable. Economic models based on GDP may fail to capture post-singularity value creation, so new measures of wealth, labor displacement, and distribution are needed. AI regulation and alignment debates are likely to intensify rapidly because the public, policymakers, and labs are all reacting to visible capability jumps.

Data Points: Data efficiency gain: 12x - Better training data versus compute efficiency improvement in the Dwarkesh Patel/Jerry Hahn analysis Architecture/training recipe gain: 3.7x - Compare against data improvement in the same six-year AI progress study AI warning tweet views: 138.3 million - Jacob Coxon resignation tweet reportedly reached this many views P-Doom estimate: greater than 10% within a decade - Anthropic alignment lead Evan Hubinger’s public statement Navier-Stokes solve cost: millions of dollars - OpenAI’s reported multi-agent result cost level O3 ARC AGI scoring cost: about $500,000 - OpenAI O3 cost to reach 87.5% on ARC AGI 1, cited by Noam Brown Astra score cost: $20 - Noam Brown’s comparison showing cost collapse for a higher benchmark score Cost decline: 25,000x - Derived from $500,000 to $20 over roughly two years H100 GPU rental price: $3.28/hour - Orn H-100 Price Index; rental price rose 22% in a month HBM memory trend: 5x - HBM memory described as having increased about fivefold in value Anthropic quarterly revenue run rate: $6.5 billion - NPR deep dive on Anthropic’s growth AI coding market share: 42% - Anthropic’s estimated share of the AI coding market Cloud deal: $35 billion - Reported infrastructure deal associated with Anthropic Launch year: 2021 - Anthropic’s founding date, used to compare its growth to Google Google revenue benchmark: 8 years - Time Google took to reach comparable revenue scale GDP growth scenario: 15% per year - Anthropic’s extreme macro scenario for AI-driven growth Labor share decline: 60% to 45% - Anthropic scenario for income distribution shift Cognitive worker unemployment: nearly 1 in 5 - Projected under Anthropic’s extreme scenario AI-designed drug stage: Phase 3 - In Silico’s rentoceratib advanced to late-stage trials Biological age reversal: 3 to 6 years - Proteomic clocks shifted younger in phase 2A analysis Peak clock effect timing: week 4 - The strongest biological age-reversal signal appeared four weeks after treatment Genome variants analyzed: ~9 billion - DeepMind Alpha Genome Atlas predicts effects of every possible single-letter mutation Genomic size: 3.2 billion letters per haploid genome - Used to explain the variant-count calculation Data-center build composition: 40% HBM memory - Claim that 40% of current U.S. capex buildout in AI is tied to HBM Compute collapse estimate: 4x decrease in requirement - If HBM bottleneck is removed, overall requirement falls sharply Startup/enterprise engineering velocity: 5x - Blitzy marketing claim about autonomous software development Fountain Life cancer detection rate: 3.3% - Members believed healthy but were found to have cancers on screening Real GDP Now tracker: 4.7% annualized - Atlanta Fed Q3 GDP estimate cited in discussion Second-quarter GDP: 1.5% - Used as comparison to the stronger Q3 estimate

Pivotal Quotes: "The quiet work of extracting, filtering, and curating what the models read has driven 3x more of the gains." — Peter/Dwarkesh summary as discussed by panel: Discussion of why data quality matters more than architecture breakthroughs "We really do earnestly believe AI could kill all humans. I personally think it is greater than 10% within the next decade." — Evan Hubinger: Alignment lead at Anthropic responding to Jacob Coxon’s resignation "The natural question is: what becomes the limiting factor? Increasingly, the bottleneck becomes." — Peter Diamandis: Framing the transition from model intelligence to physical-world constraints

Implications: AI is moving from speculative capability to measurable economic, scientific, and biological impact. Expect pressure on regulation, a premium on proprietary data/compute, rapid shifts in labor markets, and new winners in genomics, robotics, and infrastructure.

🔓 Sign Up for Unlimited Episode Search

About Moonshots with Peter Diamandis

View all episodes from Moonshots with Peter Diamandis