AGI is Being Achieved Incrementally (OpenAI DevDay w/ Simon Willison, Alex Volkov, Jim Fan, Raza Habib, Shreya Rajpal, Rahul Ligma, et al)

Episode Summary

Executive Summary: The episode is a rapid-fire, in-person and remote reaction to OpenAI Dev Day, focusing on how GPT-4 Turbo, the Assistants API, GPTs, multimodal inputs/outputs, and pricing changes reshape the AI stack. Speakers praise the breadth and polish of the launch, but repeatedly flag sharp edges: unclear pricing, weak documentation, prompt-injection risk, and product confusion between GPTs, assistants, plugins, and actions. The consensus is that OpenAI has set new defaults while leaving room for startups to specialize.

Main Topics: OpenAI Dev Day as a major platform reset (Priority: 5/5): Participants describe the event as unusually dense, polished, and strategically important, with OpenAI shipping many features at once and signaling a clearer product direction for the next year. GPT-4 Turbo improvements and pricing (Priority: 5/5): The group highlights longer context, faster responses, lower cost, and better function calling as the most immediately valuable model-level changes for developers. Assistants API, threads, retrieval, and code interpreter (Priority: 5/5): A major theme is the new developer API stack: stateful threads, file upload/retrieval, code execution, and tool orchestration. Speakers like the convenience but want more transparency and control. GPTs, plugins, and the new app-store-like ecosystem (Priority: 5/5): The conversation frames GPTs as a replacement for plugins and a new distribution layer for custom behaviors, with revenue share and link-based sharing making them more viable than the old plugin model. Multimodality: vision, speech, and DALL·E integration (Priority: 4/5): Speakers are excited by GPT-4 Vision, text-to-speech, Whisper 3, and the unified multimodal experience, but note that some capabilities are still uneven or not fully exposed in the UI/API. Security, prompt injection, and guardrails (Priority: 4/5): Several guests warn that actions, browsing, and external integrations create serious prompt-injection and safety risks, and that confirmation flows and guardrails are essential. Startup impact and the future of AI products (Priority: 5/5): Founders debate what OpenAI has commoditized versus what remains for startups, concluding that OpenAI is setting defaults while leaving room for specialized workflows, vertical products, and better UX.

Key Arguments: OpenAI under-promised and over-delivered, shipping a broad set of features that will take months for the community to absorb. GPT-4 Turbo’s lower cost and longer context materially improve the economics of building AI products, especially for high-volume use cases. The Assistants API is a major simplification for developers because it bundles state, retrieval, code execution, and tools into one framework. OpenAI’s new defaults are useful, but lack of documentation about retrieval, truncation, and pricing makes them hard to trust for serious engineering. GPTs are a better abstraction than plugins because they are shareable, more constrained, and easier to discover, with revenue sharing making them more attractive to builders. Multimodal input/output is becoming a core interface pattern, not a novelty, and will change how front ends are designed. Prompt injection becomes much more dangerous when models can take real-world actions, so confirmation and user awareness are critical. OpenAI’s product choices suggest it is focusing on synchronous, chat-based and long-tail tasks, leaving async agents and deeper workflow automation to startups. Fine-tuning remains important as a post-prompt optimization step, especially for specialized behavior and reliability. OpenAI’s pricing and scale make self-hosting less obviously economical for many teams, even when open-source alternatives exist.

Data Points: GPT-4 Turbo context window: 128K tokens - Described as a major upgrade for longer-context use cases. Assistant storage limit per assistant: 10 GB - Derived from up to 20 files at 512 MB each. File limit per assistant: 20 files - Used in the Assistants API file upload/retrieval workflow. File size limit: 512 MB per file - Maximum size for each uploaded file in an assistant. Assistant storage pricing: $0.20 per GB per assistant per day - Mentioned as expensive compared with S3. S3 comparison: ~$0.02 per GB per month - Used to contrast OpenAI’s storage pricing. Free access window for assistants: Until November 17 - Assistants were said to be free until that date. API credit given at conference: $500 - Attendees received API credits at the event. OpenAI Dev Day live audience in Twitter Space: 8,000 listeners - Reported for the live coverage of the event. Code Interpreter live audience in prior emergency space: 22,000 listeners - Used as a comparison for interest in prior OpenAI announcements. OpenAI watch party attendance at Newton: ~60 people - In-person watch party size mentioned by Alessio. OpenAI event timing: Finished around 45 minutes past the hour - Alex noted the keynote ended on time despite extra content. GPT-4 Vision cost example: $180 - Estimated cost to ingest all eight Harry Potter movies at one frame per second. GPT-4 text cost example: $15 to read / $45 to write all seven Harry Potter books - Used to illustrate cheaper GPT-4 Turbo economics. Whisper 3: Open-sourced / released - Announced briefly as a new speech-to-text model. Voice model count: 6 public voices + secret pirate voice - Discussed as part of the new text-to-speech API. OpenAI Dev Day venue: Converted multi-story car park - Described as the main conference venue. Julius user count: 100,000 users - Rahul noted Julius crossed this milestone.

Pivotal Quotes: "I feel like it's going to take us as a community several months just to completely absorb all of the stuff that they dropped on us in one giant batch." — Simon Willison: Reaction to the breadth of OpenAI’s announcements "I think GPTs are a much better abstraction on what plugins were supposed to be." — Surya Dantuluri: On the shift from plugins to GPTs and why plugins felt dead on arrival "The future programming is natural language or something like that." — Shreya Rajpal: On GPT creation and the broader no-code/low-code direction

Implications: OpenAI has turned Dev Day into a platform moment: it commoditizes basic AI building blocks while creating a new distribution layer for custom assistants. Builders should expect faster, cheaper multimodal defaults, but also more competition, more safety concerns, and more opportunity in specialized workflows and guardrails.