Agency over AI? Allan Dafoe on Technological Determinism & DeepMind's Safety Plans, from 80000 Hours

Episode Summary

Executive Summary: The episode centers on Alan Dafoe’s framework for understanding AI development through technological determinism, military-economic competition, cooperative AI, and frontier safety. Dafoe argues that while technology doesn’t force choices on its own, competition often does, making the path to AGI shaped by strategic pressures. He also explains Google DeepMind’s safety evals, governance approach, and why cooperation capabilities may matter as much as alignment.

Main Topics: Technological determinism and macro-history (Priority: 5/5): Dafoe argues that technology is not autonomous in a literal sense, but macro-scale patterns in history and technological development often emerge from competition, constraints, and path dependence rather than individual choice alone. Military-economic competition as the force behind adoption (Priority: 5/5): The core synthesis of his academic work is that technologies spread because groups that adopt useful technologies gain advantages, forcing others to follow or lose out. This applies especially in war, geopolitics, and large-scale industrial competition. Cooperative AI as a parallel to alignment (Priority: 5/5): He argues that good outcomes may require not just aligned AI systems, but AI systems that can bargain, coordinate, and cooperate with humans and each other—especially in a world of multiple powerful actors. Frontier safety frameworks and dangerous capability evals (Priority: 5/5): Dafoe describes Google DeepMind’s work on evaluating frontier models for persuasion, cyber, self-proliferation, and self-reasoning, and how those evals inform staged deployment and safety commitments. Structural risks and governance (Priority: 4/5): He distinguishes structural risks from direct misuse or accidents, emphasizing that some AI harms emerge from broader social and geopolitical systems and may require government or international governance rather than company-level controls. AGI as a high-dimensional space (Priority: 4/5): The discussion stresses that AGI is not a single point but a broad space of capabilities; the trajectory toward AGI matters, and different mixes of capability, cooperation, and safety could produce very different outcomes. Positive applications of AI (Priority: 3/5): Despite the focus on risk, the conversation ends with optimistic use cases such as self-driving cars, medical assistance, education, sustainability, and scientific discovery.

Key Arguments: History often looks constructivist at the micro level, but at macro scale it is shaped by competition, resource constraints, and selection pressures. 'Technology doesn’t force us' is incomplete; military-economic competition can force adoption because groups that fail to adapt lose out to those that do. The Meiji Restoration illustrates how a society can resist technology for a time, but external pressure eventually makes modernization unavoidable. Differential technological development is highly important but hard to execute well because it requires predicting both future pathways and their consequences. Safety and alignment are already partially incentivized by the market, so the key question is what work would not otherwise be done in time. Cooperative AI matters because aligned agents can still produce bad outcomes if they operate in conflict with one another or in unstable bargaining dynamics. AI cooperation may be easier than human cooperation in some ways, but harder in others because AIs can be opaque, alien, backdoored, or used in adversarial settings. Dangerous capability evals are only as good as capability elicitation, which means they must be staged, iterated, and supplemented by real-world observation. Structural risks arise when social or geopolitical arrangements make harmful uses of AI more likely; these cannot be fully addressed by model evals alone. AGI is useful as a term because it points to a broad space of systems better than humans at most tasks, which is likely to have major economic and political effects.

Data Points: DeepMind safety team pillars: 3 - Frontier safety, frontier governance, and frontier planning Frontier safety score for persuasion: 3/5 - Google DeepMind’s eval of Gemini 1.0 on persuasion/deception-related tasks Frontier safety score for cybersecurity: 2/5 - Gemini 1.0’s score on cyber capability evals Frontier safety score for self-proliferation: 2/5 - Gemini 1.0’s ability to help start new model instances in the wild Frontier safety score for self-reasoning: 1-2/5 - Gemini 1.0 showed limited situational awareness in self-reasoning evals Algorithmic efficiency trend: ~10x cost reduction every 2 years - Dafoe cites EPOC AI estimates that algorithmic efficiency improves about three times per year Foundational model cost trend: 100M -> 10M -> 1M - Illustrative example of how training costs could fall over successive two-year periods if trends continue Training cutoff awareness eval: 1 search query - A self-reasoning test where the model must choose which of two events to search for based on its training cutoff Meiji Restoration timeline: ~15 years - Period of rapid transformation after Commodore Perry’s arrival in 1853 Tokugawa regime duration: ~200-250 years - Japan’s extended period of relative isolation and firearms suppression Waymo safety report: 2x fewer police-report crashes; 6x fewer injury crashes - Used as an example of AI delivering concrete benefits in transportation Google DeepMind eval domains: 4 main domains - Persuasion/deception, cyber capabilities, self-proliferation, self-reasoning

Pivotal Quotes: "Technology doesn't force us to do anything. It merely opens the door. It makes possible new ways of living, new forms of life." — Alan Dafoe: Opening framing of the technological determinism discussion "Technology doesn't force us. It merely opens the door. And it's military-economic competition that forces us through." — Alan Dafoe: Core synthesis of his academic argument "One of the main challenges to alignment... is this notion of collusion amongst our AI systems, so that we can't rely on any of these institutions of AI to make us safe." — Rob Wiblin: Discussion of why cooperative AI could matter for AGI safety

Implications: AI governance should focus not only on model alignment, but also on cooperation, deployment structure, and structural risks. Frontier labs, governments, and civil society will need better evals, forecasting, and staged release if they want AGI to be safe and beneficial.