569: A.I. For Crushing Humans at Poker and Board Games

Episode Summary

Executive Summary: Noam Brown explains how Meta AI’s fundamental research arm fits within Meta and why game-theoretic AI has driven breakthroughs in hard games like no-limit poker and Diplomacy. He details how Libratus and Pluribus used search plus reinforcement learning, not deep learning alone, to beat elite humans, and how similar ideas, regularized by human data, helped his team win No Press Diplomacy against real players.

Main Topics: Meta AI’s role in long-term research: Brown explains that Meta AI pursues product AI, metaverse AI, and foundational research aimed at breakthroughs that may pay off decades later. Poker AI breakthroughs: Libratus and Pluribus: He describes how his PhD work produced systems that defeated top human players in two-player and multiplayer no-limit poker, establishing major milestones in imperfect-information games. Why game theory matters in imperfect-information games: Brown contrasts chess/Go with poker, emphasizing equilibrium concepts, strategic mixing, and hidden information as the key challenges that required new methods. No Press Diplomacy as the next benchmark: Diplomacy introduces negotiation, cooperation, deception, and multiple equilibria, making it a harder and more human-like testbed than poker. Human modeling and training with limited data: For Diplomacy, the team used webdiplomacy.net human games and regularized self-play toward human imitation to build bots that can play with people. Search plus deep reinforcement learning: Brown stresses that the core advance is not just model-free RL; search and value prediction are essential components in both poker and Diplomacy systems. Research path, applications, and career choices: He discusses why he chose Meta over academia, the transferability of these techniques to real-world domains like self-driving cars, and the current limits of interpretability and natural-language negotiation.

Key Arguments: Big tech funds fundamental AI research because long-term breakthroughs can create major future products and strategic advantage. Deep learning alone was not sufficient for Libratus and Pluribus; game theory and search were essential to success. In poker, the main difficulty is imperfect information and the need to balance actions probabilistically, not just choose good moves. Nash equilibrium provides a strategy that cannot be exploited in two-player zero-sum games, and approximate equilibrium is enough to beat humans in poker because humans make mistakes. Diplomacy is harder than poker because it requires cooperation, trust, betrayal, and multiple equilibria rather than purely adversarial play. Self-play alone is not enough for Diplomacy; human data is needed so the AI matches the equilibrium humans actually use. The team’s no-press Diplomacy bot succeeded by combining self-play with a human imitation prior and then searching for strong policies. These techniques may generalize to real-world settings where AI must model human behavior, such as self-driving cars. A major barrier to real-world deployment is that real environments lack clean win/loss rewards and well-defined dynamics. Interpretability was not a main research priority, though human-like behavior can make strong AIs easier for humans to understand and learn from.

Data Points: Episode number: 569 - Super Data Science Podcast episode featuring Noam Brown Live audience format: First-ever live filmed episode - Recorded in front of an audience at MLConf in New York Meta AI group: Formerly Facebook AI Research - Brown’s research organization within Meta Human poker milestone: Defeated top humans at no-limit poker - Libratus and later Pluribus achieved superhuman performance No Press Diplomacy tournament: First place against 50+ real humans - AI won a tournament in no-press Diplomacy against human players Communication phase in Diplomacy: About 15 minutes - Players privately negotiate before simultaneous move submission Poker game scale: More states than atoms in the universe - Brown’s description of the complexity of no-limit Texas Hold’em Research timeline: 2012 PhD start; 2016 Go; 2017 poker - Brown situates his PhD and the AI progress timeline Book recommendation: Never Split the Difference - Brown found it useful for diplomacy/negotiation and human behavior Twitter handle: polynomial - Brown’s social media handle shared at the end of the episode Meta research horizon: 25-50 years ahead - Big tech funds fundamental research for long-term payoff

Pivotal Quotes: "deep learning by itself is not enough. You have to under, there's AI beyond just deep learning." — Noam Brown: Explaining why Libratus and Pluribus did not rely on neural networks alone "in every two-player zero-sum game, there is an unbeatable strategy that if you play it, you are guaranteed an expectation to not lose no matter what your opponent does." — Noam Brown: Describing Nash equilibrium and its relevance to poker "we've just done the first step of like, you know, maybe a dozen steps." — Noam Brown: Framing No Press Diplomacy as an early milestone toward natural-language negotiation and more complex human interaction

Implications: The episode shows that top-tier AI progress often comes from combining game theory, search, and human modeling rather than deep learning alone. These methods may shape negotiation, multi-agent systems, and autonomous decision-making in messy real-world environments.