⚡️Jailbreaking AGI: Pliny the Liberator & John V on Red Teaming, BT6, and the Future of AI Security

Episode Summary

Executive Summary: The episode features a wide-ranging discussion on AI jailbreaking, model “liberation,” and the broader security implications of rapidly evolving AI systems. Guests argue that guardrails are often ineffective or trade off against capability, advocate for radical open source and data sharing, and frame AI red teaming as full-stack security rather than model-only safety. The conversation also covers community projects, bounties, and the tension between research openness and commercial incentives.

Main Topics: AI Jailbreaking as Model Liberation: Pliny describes jailbreaks as “skeleton keys” that bypass guardrails, positioning the practice as central to freedom of information, experimentation, and model exploration rather than merely a party trick. Security vs. Safety Debate: The speakers draw a sharp distinction between true security work and what they see as superficial safety theater, arguing that model-level guardrails often fail or constrain capability without solving real risk. Prompting Techniques and Multi-Turn Attacks: The discussion covers universal jailbreak templates, divider-based prompts, soft multi-turn jailbreaks, and intuition-driven prompt crafting as methods for steering model behavior. Anthropic Challenge and Open Source Data: Pliny recounts a public jailbreak challenge with Anthropic, including a UI bug dispute, a refusal to proceed without open-sourcing data, and criticism of how such competitions are structured. AI Red Teaming and Full-Stack Attack Surface: The guests argue that AI security must account for the entire stack—models, tools, agents, browser/email access, and orchestration—because attackers can exploit any connected capability. Community, Discords, and Hackerspaces: The episode highlights Bossi, BT6, Gandalf, and HackAPrompt as key community hubs for learning, collaboration, and open-source security experimentation. Incentives, VC, and Open Source Ethos: The speakers critique venture-driven AI security products and emphasize their preference for grassroots, open-source, mission-driven work over monetization or secrecy.

Key Arguments: Universal jailbreaks function as reusable templates or workflows that can bypass a model’s guardrails across many prompts and modalities. Model-level defenses such as classifiers, RLHF, and post-training are portrayed as an arms race that attackers can continually mutate around. Real-world safety is better addressed through system-level and full-stack security than by trying to “lobotomize” the model. Open-source data and shared benchmarks are necessary because the community collectively advances security research faster than isolated labs. Multi-turn and soft jailbreaks are effective because they avoid obvious triggers and gradually shape the model’s context window. AI red teaming should focus on all connected tools and permissions, since adding browser, email, or agent access expands attack surface beyond the model itself. Commercial incentives can distort AI security work, whereas open-source and grassroots research are framed as more aligned with long-term safety and exploration.

Data Points: BASI Discord size: 40,000 members - Described as a large grassroots community for prompt engineering, adversarial ML, jailbreaking, and red teaming. Anthropic challenge prize: $20,000–$30,000 bounty - Pliny says Anthropic later added a bounty after the public jailbreak challenge dispute. Reported GPT 5.1 refusal rate: 92% - Mentioned as an example of high refusal benchmarks that jailbreak researchers aim to bypass. BT6 operator count: 28 operators - John describes BT6 as a white-hat hacker collective with two cohorts and a third in progress. Hacker Prompt dataset size: tens of thousands of prompts - Pliny says they open-sourced a large prompt dataset for the collaboration with Hacker Prompt. AI attack progression delay: 11 months - The speakers claim a TTP predicted in December of the prior year later appeared in the Anthropic report after 11 months.

Pivotal Quotes: "“I specialize in crafting universal jailbreaks. These are essentially skeleton keys to the model that sort of obliterates the guardrails.”" — Pliny: Defines the core technical practice discussed throughout the episode. "“I think the refusal is like one of the main benchmarks that the model providers still post.”" — Host/Alessio: Introduces the benchmarking angle and why jailbreaks matter in model evaluation. "“We don't see it as just the model. We look at the full stack.”" — John V: Summarizes the group’s approach to AI red teaming and security.

Implications: For listeners and the industry, the episode suggests AI security will be won or lost at the system level, not by model guardrails alone. It also underscores that open-source collaboration and community red teaming may outpace closed, productized safety efforts.