Episode Summary
Executive Summary: Jan Leike outlines OpenAI’s Super Alignment project: a four-year effort, backed by 20% of OpenAI’s secured compute, to develop methods for aligning systems as capable as or smarter than humans. The plan centers on scalable oversight, generalization, interpretability, and adversarial testing, with the longer-term goal of automating alignment research so models can help align future models safely.
Main Topics: Why Super Alignment exists: Leike argues current alignment methods like RLHF will not scale to systems that are smarter and harder for humans to evaluate, creating a need for new techniques before AGI arrives. Scalable oversight and evaluation: A central strategy is using AI to help humans evaluate harder tasks, so oversight can keep pace with model capability rather than relying on unaided human judgment. Generalization and robustness: The discussion emphasizes understanding whether models generalize to true human intent or merely to what humans can detect, including resistance to jailbreaks and deceptive behavior. Interpretability and automated analysis: Leike highlights interpretability as a way to inspect model internals at scale, including experiments where GPT-4 generates explanations of neurons and behaviors. Adversarial testing and deceptive models: The team wants to deliberately train deceptive or buggy models as model organisms to measure whether alignment tools can detect hidden flaws or manipulation. Governance, deployment, and public legitimacy: The interview repeatedly returns to oversight, regulation, internal safety reviews, and the need for scientific consensus before deploying more powerful systems. Hiring and team-building: A substantial portion of the episode is a recruiting pitch for researchers, engineers, and managers who can work on experiments, coding, and alignment-focused ML work.
Key Arguments: RLHF works well today but does not scale because humans cannot reliably evaluate increasingly complex model outputs or hidden model intent. Alignment work should focus on systems roughly at human level first, because aligning a near-human system is more tractable than aligning a vastly superintelligent one. The problem should be split into training methods that improve alignment and validation methods that measure whether the alignment actually succeeded. Scalable oversight can be improved by using AI assistance to make evaluation easier, such as critique models, recursive reward modeling, or debate-like systems. Generalization is the core issue: models must learn "don’t do the bad thing," not merely "don’t do the bad thing when humans notice." Interpretability matters because neural networks are effectively open boxes with perfect access to internals, yet we understand very little about how they work. Adversarially training deceptive models can reveal whether oversight and interpretability tools would catch subtle deception in practice. Progress should be measured empirically through experiments, not just argued theoretically, because the field does not yet know which methods will work best. OpenAI should not be the only actor deciding deployment; governance, audits, regulation, and external scientific consensus are essential. AI labs are a strong place to do alignment research because they are close to frontier models and can learn directly from real systems and training runs.
Data Points: Compute commitment: 20% - OpenAI announced it would dedicate 20% of the compute it has secured so far to the Super Alignment project. Project timeline: 4 years - The team aims to solve the alignment problem within four years. ChatGPT size comparison: GPT-2-sized model preferred over GPT-3 base model - Leike cites InstructGPT results showing a much smaller model, once aligned, was preferred over a far larger base model. RLHF fine-tuning budget: Less than 2% of pre-training compute - Leike says GPT-3 fine-tuning with RLHF used under 2% of pretraining compute, and GPT-4 used even less relative compute. Critique experiment effect: 50% more flaws found - In a randomized controlled trial, GPT-3.5-generated critiques helped humans find 50% more flaws in a summarization task. Automated interpretability scale: ~300,000 neurons - The team ran automated interpretability over all neurons in GPT-2, which had nearly 300,000 neurons. Hiring target: At least 10 hires by end of year - Leike said the team expected to hire at least 10 more people before the end of the year. Current team size: About 20 people - He described the Super Alignment team as roughly 20 people at the time of the interview. OpenAI leadership model: Non-profit board has final say - Leike noted OpenAI’s board can ultimately decide whether a model is deployed. Research support model: Millions of virtual FTEs - He argued that AI-assisted alignment research could scale far beyond the human team size through compute and automation.
Pivotal Quotes: "“we have a real angle of attack on the problem that we can like actually iterate on, we can actually build towards. And I think it's like pretty likely gonna work actually.”" — Jan Leike: Leike’s core expression of optimism about the tractability of alignment research. "“You don't want to train on the test, you know?”" — Jan Leike: Used to explain the training/validation split for alignment methods and why evaluation must be separated from optimization. "“How do we know it is truly aligned.”" — Jan Leike: Summarizes the central challenge: distinguishing a genuinely aligned system from one that only appears aligned under supervision.
Implications: The interview frames alignment as an urgent, solvable engineering/science problem—if the field invests early in evaluation, interpretability, and AI-assisted oversight. For listeners, it suggests technical work, governance, and hiring now matter enormously.