Gavin Uberti - Real-Time AI & The Future of AI Hardware - [Invest Like the Best, EP.356]

Episode Summary

Executive Summary: Patrick O’Shaughnessy interviews 21-year-old Etched founder Gavin Uberti about why transformers, not generic AI chips, will drive the next AI infrastructure wave. Uberti argues model capability will scale through bigger data centers, specialized ASICs, and lower latency, enabling real-time AI, robotics, and new applications.

Main Topics: Transformers as the core AI breakthrough (Priority: 5/5): Uberti says transformers unlocked modern AI because they scale well and still have room to improve. No fast takeoff, but a spectrum toward superintelligence (Priority: 5/5): He argues AI will become superhuman gradually as compute and infrastructure expand. Inference latency as the next bottleneck (Priority: 5/5): Real-time use cases need much faster first-token and per-token response times. Transformer ASICs and specialized silicon (Priority: 5/5): He believes hard-coded chips for transformers will beat general-purpose GPUs on cost and speed. Massive data centers and power build-out (Priority: 4/5): Future frontier models will require centralized facilities with extreme power, cooling, and bandwidth. Chip design, verification, and faster iteration (Priority: 4/5): He explains the semiconductor pipeline and why AI-era chip development must accelerate. Competitive strategy and the AI stack (Priority: 4/5): He maps winners across model labs, hyperscalers, chip designers, fabs, memory makers, and startups.

Key Arguments: Transformers won because pretraining + RLHF scale better than earlier AI methods. Superintelligence will emerge gradually, not via sudden fast takeoff. GPT-4 is already a better lawyer than him, showing capabilities are advancing now. Real-time AI needs milliseconds latency for speech, robotics, and agentic use cases. Transformer ASICs can pack far more compute and >90% utilization than GPUs. The AI build-out will require 2 GW data centers and new cooling/power infrastructure. Training runs and chip design cycles must shrink from years to months to keep up.

Data Points: GPT-2 parameters: 1.5 billion - Size of the earlier model cited in the scaling progression GPT-3 parameters: 175 billion - Size of the model used to illustrate scaling gains GPT-4 parameters: multi-trillion - Uberti’s description of the latest generation scale GPT-4 usage: 50, 100 times a day - Patrick says he uses GPT-4 constantly GPT-5 build time: two years - Uberti’s estimate for the next generation model cycle GPT-6 build time: three, five years - Uberti’s estimate for building new supporting data centers Data center power: 20 megawatt - Example of a small current facility size Data center power: two gigawatt - Projected scale for frontier AI infrastructure GPU rack bandwidth: double the bandwidth of a literal undersea cable - Used to argue for centralized data centers Batch size: 2,500 - Transformer ASIC inference batching example Compute utilization: more than 90% utilization - Claim about efficiency from specialized chips Training cycle: four or five years - Current conception-to-production chip timeline Hobbyist chip timeline: weeks, months - Old-node chip development using automated tools Target chip iteration time: a couple of months - Uberti’s hoped-for future chip development speed Chip design investment: $100 million - Example cost to justify building a custom ASIC Model training cost: $1 billion - Threshold where ASIC economics start to make sense Model training cost: $10 billion - Scale where custom ASICs become a no-brainer Chip vendor count: two or three firms - Analogy to the concentrated fab market and future AI data centers

Pivotal Quotes: "I think that AI is very unique." — Gavin Uberti: He explains why AI still has relatively low-hanging fruit compared with older sciences "I don't believe in this concept of fast takeoff." — Gavin Uberti: His view of how superintelligence will arrive gradually through scaling "The only techniques that will really advance the field of artificial intelligence are those that are able to leverage the cost of compute getting cheaper." — Gavin Uberti: He summarizes the 'Bitter Lesson' and why scale dominates

Implications: The unresolved race is whether specialized hardware and infrastructure can be built fast enough to support the next AI wave; firms should prepare for consolidation and extreme capital intensity.