E29: The AI Chip Revolution with Andrew Feldman of Cerebras

Episode Summary

Executive Summary: Andrew Feldman explains how Cerebras was built to transform AI compute by designing an enormous, purpose-built chip and full system stack around AI’s communication-heavy, sparse workloads. He contrasts Cerebras’s “one big chip” approach with GPU clusters, arguing it removes distributed-compute bottlenecks, simplifies training, and enables far faster iteration for large models.

Main Topics: Why Cerebras built a giant AI-first chip (Priority: 5/5): Feldman says the company’s goal was not incremental improvement but industry transformation: build a machine optimized specifically for AI, where minimizing off-chip communication and maximizing on-chip locality yields large gains. What a chip is and why specialization matters (Priority: 4/5): The conversation starts with basic chip fundamentals and expands into the diversity of chip types, emphasizing that optimal designs depend on workload-specific trade-offs between compute, memory, and communication. AI workload characteristics vs GPU architecture (Priority: 5/5): Feldman argues AI training/inference are often communication-bound and sparse, while GPUs are general-purpose and therefore waste real estate and introduce contention, sharding, and distributed coordination overhead. Engineering a 2.5 trillion-transistor system (Priority: 5/5): He describes the end-to-end design process: architecture, simulation, wafer manufacturing, packaging, cooling, power delivery, compiler/software, and systems integration, all co-designed to make the giant chip usable. Manufacturing and supply-chain innovation (Priority: 5/5): The team convinced TSMC to fabricate the chip using existing manufacturing steps, then invented techniques to communicate across scribe lines and avoid cutting the wafer into many separate chips. Scaling from chips to supercomputers and open models (Priority: 4/5): Cerebras now clusters its systems into supercomputers, including Andromeda, and has used them to train and release open-source GPT models, aiming to reduce barriers to large-scale training. Broader implications for AI, edge, and society (Priority: 4/5): Feldman discusses where big chips fit in the AI stack, why training remains a data-center problem, the limits of edge deployment, and the opportunities and risks AI brings to society.

Key Arguments: AI workloads benefit from very large chips because moving data on-chip is thousands of times faster and more energy-efficient than moving it off-chip across a board or cluster. GPUs are effective but fundamentally general-purpose; they must spend silicon and software complexity on non-AI features, creating memory-bandwidth and contention bottlenecks for modern models. AI is often sparse linear algebra, so skipping zero-valued work is a major advantage; Cerebras’s architecture is designed to exploit sparsity rather than compute everything densely. A giant chip can hold activations and parameters in place, eliminating most distributed-compute orchestration and making large-model training much simpler than on multi-GPU systems. Cerebras’s strategy was to optimize the whole stack—chip, motherboard, enclosure, cooling, power, compiler, and management software—rather than sell a standalone component. The company’s manufacturing breakthrough came from persuading TSMC to use existing tools and inventing methods to communicate across normally cut scribe lines on a wafer. Large-scale AI is increasingly a data-center phenomenon; edge devices are better suited to smaller classification tasks than to heavy autoregressive inference or training. AI progress and deployment bring both benefits and risks: the technology can improve medicine, productivity, and safety, but also enables bias, surveillance, scams, and oppression.

Data Points: Transistors on Cerebras chip: 2.5 trillion - The world-record chip discussed by Feldman Chip size relative to prior art: 56x larger - Feldman says the chip was designed to be 56 times larger than anything built before Architecture exploration time: about 6 months - Time spent solving major architecture problems early in development Initial development budget for hard problems: $20 million - Feldman says they solved the early big-chip problems in about six months for this amount Initial team size: 20–25 people - Approximate nucleus of engineers as the company began building the system Founding capital process: 8 pitches, 8 term sheets - He says the company was funded quickly after eight pitches Wafer size: 300 square millimeters - He describes the wafer as a 300-square-millimeter circle of silicon in the fab explanation Process node at founding: 16 nanometers - The company initially worked at the 16nm node in 2016 On-chip core count: 850,000 cores - Feldman says the architecture has enough cores for the largest neural network layers to fit without sharding Memory capacity claim: trillions of parameters - He states the machine has enough memory for trillions of parameters Cluster size for Andromeda: 16 machines - The supercomputer announced in November of the prior year Andromeda core count: 13.6 million cores - Compute scale of the Cerebras supercomputer Largest supercomputer comparison: 8 million cores - He compares Andromeda to the then-largest supercomputer on Earth Model release count: 7 GPT models - He says they released seven GPT models into the open-source community Licensing: Apache 2 - The open-source models were released under Apache 2 license Corporate scale of NVIDIA: $750 billion - The host cites NVIDIA’s market capitalization in the introduction GPT-4 training cost estimate: $100 million or more - Referenced as an example of the cost of large-scale model training ChatGPT-like output scale: thousand to ten thousand tokens - Used when contrasting autoregressive inference with classification tasks

Pivotal Quotes: "This wasn't about making money. This wasn't about moving the ball forward a little bit. We wanted to move an industry forward." — Andrew Feldman: He explains the founding motivation behind Cerebras "We wanted to build a chip that was optimized for one thing, and that was for AI work." — Andrew Feldman: He describes the core product strategy and specialization philosophy "We thought how silly is that to break Humpty Dumpty up into little pieces and then pay for expensive switching to try and get it to behave like it was together again." — Andrew Feldman: He explains why they rejected the standard chip-splitting model "We hold the activations on the chip. And we don't move those." — Andrew Feldman: He summarizes how the architecture reduces communication overhead

Implications: The episode frames AI infrastructure as a new compute paradigm: bigger, more specialized chips can simplify training, cut cost, and accelerate iteration. For industry, it suggests a bifurcation between cloud-scale training hardware and lighter edge inference devices, with major stakes for geopolitics, open-source models, and safety.