820: OpenAI's o1 "Strawberry" Models

Episode Summary

Executive Summary: The episode argues that OpenAI’s O1 (“Strawberry”) marks a major leap in public AI capability by using reinforcement learning to “think” before answering, especially on math, coding, and other complex reasoning tasks. The host demonstrates strong performance across benchmarks and live tests, while also noting safety gains, access limits, and that O1 is not always best for everyday chat or creative work.

Main Topics: OpenAI O1 as a new state of the art: The host presents O1 Preview as the best publicly available text model, surpassing GPT-4, Claude 3.5 Sonnet, and Gemini on difficult reasoning tasks. System 2 reasoning and private chain of thought: O1 is described as being trained to think slowly and iteratively before responding, analogous to Kahneman’s system two thinking. Scaling inference-time compute: A major theme is that AI progress can now be scaled by giving models more thinking time at inference, not just by more data, parameters, or training time. Benchmark gains and empirical comparisons: The episode cites large benchmark improvements in coding, olympiad math, and PhD-level science questions, including in-development model results. Live demonstrations on math and programming tasks: The host tests O1 on counting letters, philosophical reasoning, calculus problems, a difficult geometry/optimization question, and interactive HTML generation. Safety, jailbreak resistance, and alignment: OpenAI’s reasoning-based safety training is discussed as a way to improve jailbreak resistance and alignment behavior. Practical use cases and limits: The host explains when to use O1 versus Claude 3.5 Sonnet or Gemini, and emphasizes that O1 is not ideal for all tasks and is not conscious.

Key Arguments: O1 is a breakthrough because it can iteratively reason before answering, unlike prior LLMs that mostly produced immediate responses. Longer inference-time thinking improves performance on hard problems, creating a new scaling dimension for AI. O1 is especially strong on math, coding, scientific reasoning, and complex analytical tasks, but not necessarily on everyday chat or writing. OpenAI’s benchmark results suggest huge jumps over GPT-4 on advanced reasoning categories and competitive programming. The model’s internal step-by-step process can support better safety and jailbreak resistance. The host’s live tests suggest O1 can solve or substantially aid in solving problems that previous models struggled with, including tricky calculus and code generation. Despite the anthropomorphic language, these systems are not conscious and should be understood as powerful computational tools.

Data Points: O1 Preview access limit: 50 messages per week - ChatGPT Plus/Team user interface access mentioned near the end of the episode O1 Mini access limit: 50 messages per day - Expanded access described by the host O1 Preview API rate limit: 100 requests per minute - Available to tier five developers O1 Mini API rate limit: 250 requests per minute - Available to tier five developers O1 model type: Reinforcement learning with private chain-of-thought style reasoning - Core training approach described throughout the episode Competitive programming result (in development O1): 89th percentile - Codeforces performance cited by the host Competitive programming result (O1 Preview): 62nd percentile - Codeforces performance cited by the host Competitive programming result (GPT-4.0): 11th percentile - Comparison benchmark cited by the host IMO qualifying exam result (in development O1): 83% - OpenAI-reported score on mathematics olympiad qualifying exam IMO qualifying exam result (O1 Preview): 62% - OpenAI-reported score on mathematics olympiad qualifying exam IMO qualifying exam result (GPT-4.0): 13% - OpenAI-reported score on mathematics olympiad qualifying exam Jailbreak test score (GPT-4.0): 22/100 - OpenAI safety evaluation cited by the host Jailbreak test score (O1): 84/100 - OpenAI safety evaluation cited by the host Time to answer: strawberry letter count: 6 seconds - O1 correctly counted the three Rs in “strawberry” Time to answer: hot dog sandwich reasoning: 7 seconds - O1 broke down the debate into multiple steps Time to answer: calculus partial derivatives: 12 seconds - O1 solved a second-order partial derivatives problem Time to answer: definite integral: Not specified, but completed successfully - O1 solved a definite integral over the interval 3 to 4 Time to answer: complex sphere radius problem: 95 seconds - Longest example the host tested; model explored multiple approaches Time to answer: HTML neural network visualization: 39 seconds - O1 generated interactive HTML for a dense neural network Time to revise HTML arrows issue: 12 seconds - O1 attempted to fix missing directed arrows in the visualization

Pivotal Quotes: "OpenAI's new O1 series of models, which represents a tremendous leap forward in AI capabilities." — John Crone: Opening the main discussion of the episode "The O1 large language model was trained with reinforcement learning to quote-unquote think before responding." — John Crone: Explaining the model’s core technical innovation "The difference in capabilities is night and day." — John Crone: Describing the gap between O1 and previous text models on complex tasks

Implications: Listeners should expect faster progress in reasoning-heavy AI, especially for math, coding, and science. The industry may shift toward inference-time compute as a new scaling frontier, with major gains possible but higher costs and stronger safety concerns.