AI won't plateau — if we give it time to think | Noam Brown

Episode Summary

Executive Summary: Noam Brown argues that AI progress will not plateau because a new scaling dimension exists: system two thinking. Drawing on poker, chess, Go, and OpenAI’s o1, he shows that giving models more time to think can yield gains comparable to massive increases in training scale, making slower, costlier inference worthwhile for high-value tasks.

Main Topics: AI progress has been driven by scaling (Priority: 5/5): Brown explains that frontier AI models have improved largely through bigger datasets, more compute, and longer training on the transformer architecture rather than radical algorithm changes. Poker as the first proof of concept for thinking-time scaling (Priority: 5/5): His PhD research on poker AIs showed that instant decisions were a major limitation; adding deliberate thinking dramatically improved performance against top human players. System one vs. system two thinking in AI (Priority: 5/5): Brown uses Kahneman’s framework to distinguish fast, intuitive inference from slower, deliberative reasoning, arguing that the latter was underutilized in earlier AI systems. Thinking time can rival massive training scale (Priority: 5/5): Experimental results in poker showed that a small amount of deliberation could match performance gains that would otherwise require enormous increases in model size and training time. Broader evidence from chess and Go (Priority: 4/5): Deep Blue and AlphaGo also benefited from longer thinking time, and later research suggested a strong relationship between more thinking and better performance across games. Inference-time scaling as a new AI paradigm (Priority: 5/5): Brown argues that instead of only spending more to train models, we can spend more per query to let models reason longer, opening a new path for performance gains. Real-world value of slower, stronger models (Priority: 4/5): He points to OpenAI’s o1 and high-stakes problems like cancer treatment, solar panels, and math proofs to argue that waiting longer or paying more can be justified when the output is valuable.

Key Arguments: AI will likely not plateau because there is still untapped room to scale system two thinking, not just training compute. In poker, the key weakness of earlier bots was not only learning capacity but the lack of time to deliberate at inference. Twenty seconds of thinking in poker produced performance gains equivalent to scaling model size and training by 100,000x. Human experts outperform instant AI in hard tasks partly because they can spend more time on difficult decisions. The same thinking-time advantage appears in chess and Go, suggesting the effect generalizes beyond poker. Training-scale improvements are becoming extremely expensive, but inference-time thinking remains relatively cheap and flexible. Models like o1 demonstrate that language AI can also benefit from deliberate reasoning before answering. For important real-world problems, higher cost and latency can be acceptable if the value of the answer is high enough.

Data Points: GPT-2 training cost (2019): About $5,000 - Used to illustrate how much cheaper earlier frontier models were compared with today’s systems. Frontier model training cost today: Hundreds of millions of dollars - Shows how scaling training has become dramatically more expensive. Poker AI training duration: Almost a trillion hands over about three months - Brown’s bot trained extensively before the human competition. Bot decision latency: About 10 milliseconds - The poker bot acted instantly during competition, unlike humans who could think. Human experts’ lifetime experience: Maybe ten million hands - Contrasts human experience with the bot’s massive training volume. Thinking-time performance gain: 20 seconds of thinking = 100,000x model/training scale - Central experimental result from Brown’s poker research. Pioneer poker competition prize: $120,000 - 2015 challenge against four of the world’s top poker players. 2017 poker competition prize: $200,000 - Redesigned AI later beat top poker pros by a wide margin. Betting odds at announcement: About 4 to 1 against the AI - Poker community initially doubted Brown’s team would win. Betting odds after three days: About 50-50 - Early results shifted expectations but uncertainty remained. Deep Blue thinking time: A couple of minutes per move - Example showing that successful game AIs already use deliberation. AlphaGo thinking time: A couple of minutes per move - Another case where longer thinking improved performance. 2021 game-research relationship: 10x thinking time ≈ 10x model size/training - Scientific paper cited to quantify the tradeoff between deliberation and scale. Inference cost for frontier models: Fractions of a penny per query - Brown argues there is room to spend more per query for higher-quality answers. Example cost tradeoff: From a penny per query to ten cents per query - Illustrates the practical cost of increasing model thinking time.

Pivotal Quotes: "Spending twenty seconds thinking in a hand of poker got the same boost in performance as scaling up the size of the model and the training by 100,000 X." — Noam Brown: He describes the most striking result from his poker experiments. "We are just at the very beginning of scaling up in this direction." — Noam Brown: He emphasizes that inference-time reasoning is a newly available frontier for AI progress. "wanna bet?" — Noam Brown: His closing challenge to skeptics who believe AI will plateau.

Implications: AI’s next leap may come from longer reasoning, not just bigger training runs. That means slower, pricier models could be worth it for high-value tasks in science, medicine, and research, reshaping how AI is built and monetized.