Inside the Model Factory — Eiso Kant, Poolside AI

Episode Summary

Executive Summary: Debu explains Poolside’s shift toward open weights and open research as both a philosophical and strategic response to a future where only a few labs control intelligence. The conversation centers on building a model factory, using RL and engineering rigor to speed iteration, and making smaller models far more capable through behavior improvements, persistence, and better tooling.

Main Topics: From closed thinking to open weights and open research (Priority: 5/5): Debu traces Poolside’s evolution from a more guarded stance to a commitment to releasing weights and research, arguing that the world should have many foundation model companies rather than a tiny oligopoly. Why Poolside exists: AGI, abundance, and choice (Priority: 5/5): He frames the company’s mission as building toward AGI while resisting a future where a handful of firms control access to intelligence, emphasizing diversity of providers and user choice. The model factory: engineering as the core of model building (Priority: 5/5): Debu argues foundation model development is mostly an industrial engineering problem: data pipelines, distributed training, reproducibility, reliability, and rapid experimental turnaround are the real moat. Post-training, RL, and behavior over raw scale (Priority: 5/5): A major theme is that model quality increasingly comes from post-training behaviors like persistence, verification, and backtracking, not just parameter count or raw pretraining scale. Smaller models, stronger capabilities (Priority: 4/5): He highlights Laguna S as a proof point that an 8B-active sparse model can outperform much larger systems in coding-like tasks, suggesting more juice remains in smaller weight classes. Harnesses, tools, and the future of agents (Priority: 4/5): Debu argues models should use minimal harnesses and increasingly write code directly rather than rely on bloated tool-call systems, with agents already taking over much of internal research and training work. Safety, regulation, and the case for competition (Priority: 4/5): He supports open innovation while acknowledging future constraints may be needed; he warns against premature restrictions that could entrench a small oligopoly and reduce competition.

Key Arguments: Poolside changed course because a world with 100 foundation model companies is preferable to one with five, even if Poolside were one of the five. Foundation model work is primarily an engineering discipline: improving data, improving compute efficiency, and building reliable systems that turn ideas into reproducible experiments. RL and post-training matter enormously because they shape behaviors like persistence, verification, and tool use, which can be more predictive than raw intelligence for many tasks. Small models can achieve surprising capability gains if trained with the right behaviors; Laguna S shows this by outperforming larger models in coding tasks while remaining highly deployable. Open research is more valuable than open weights alone because sharing methodology, lessons, and systems helps others build faster and more safely. Model factories and strong infrastructure enable high experimental throughput, reproducibility, and eventual agentic automation of research and training tasks. The industry should not over-restrict open models too early; governance should focus on real misuse risks and involve democratic oversight rather than unilateral company decisions. Models and hardware are co-designed; future gains may come from better precision, better networking, and smarter hardware allocation, especially for RL bottlenecks.

Data Points: Years spent on first open-source code-model company: About 4-5 years - Debu described working on Sourced from roughly 2015 to the end of 2019. Investor money burned in earlier startup: $12 million - He referred to the earlier company as his biggest career failure at the time. Poolside team size: Less than 70 researchers and about 35 engineers - He described the company as a small team running large numbers of experiments. Monthly experiment volume: 10,000 to 20,000 experiments per month - Used to illustrate the scale of the model factory and infra demands. Laguna Access2 pretraining-to-launch time: 5 weeks - He cited the model factory enabling very short turnaround from pretraining start to launch. Laguna S pretraining-to-launch time: 8 weeks - The newly released model was trained and launched on an accelerated schedule. New medium pretraining run: 39-day pre-training run - He said the next larger Laguna M run had started the day before. Laguna S size: 118B total parameters / 8B active - He emphasized the sparse architecture and deployment-friendly weight class. Earlier XS size: 30B-ish - He referenced the prior XS model as the basis for scaling up. Older medium size: 200B - He noted the previous medium model was slated for deprecation. Model runtime on device: 30-40 tokens/sec on a DGX Spark - He used this to argue the model fits practical local deployment. Internal compute cluster: 10K H200 cluster - He described current infrastructure before scaling further. Global knowledge work share: 25% of global economy / $25T - He used this to frame the market size for model-enabled work. Named modeling cadence: Weeks between models - He contrasted current cadence with the earlier months-or-years pace. Foundation-model company staffing ratio: Less than 150 engineers and researchers total - He used this to emphasize leverage and individual impact.

Pivotal Quotes: "I would rather live in a world that has 100 foundation model companies than a world that has five, even if I was one of the five." — Debu: Explaining why Poolside chose open weights and open research as an ecosystem strategy. "Model building is ultimately 90% engineering." — Debu: Describing the model factory view: data, distributed systems, reproducibility, and reliability are the core work. "I think MCP and tools are stupid." — Debu: Arguing that future agent systems should use minimal harnesses and have models write code directly instead of relying on many tool calls.

Implications: The episode suggests frontier AI is becoming more industrial, more reproducible, and more competitive. Smaller, well-trained models may remain highly valuable, while open research and more entrants could prevent intelligence from consolidating into a few firms.