Episode Summary
Executive Summary: CoreWeave’s co-founder explains how AI is forcing a rebuild of cloud infrastructure: legacy clouds were designed for serializable web workloads, not massive parallel GPU jobs. Demand for H100/A100 compute, networking, power, and data center capacity is outstripping supply, with AI inference driving sustained growth. CoreWeave claims a differentiated software-plus-physical stack, strong NVIDIA dependence, and a multi-year infrastructure bottleneck.
Main Topics: From crypto GPU hosting to AI cloud infrastructure (Priority: 5/5): CoreWeave began by renting GPUs to crypto miners, especially GPU-friendly Ethereum mining, but shifted in 2019 into a cloud built specifically for AI and highly parallel workloads. Why AI workloads break traditional cloud design (Priority: 5/5): The guest argues hyperscale clouds were optimized for websites and data lakes, while AI requires a fundamentally different orchestration, networking, and provisioning model built from scratch for parallel compute. Economics of GPU clusters and supercomputer-scale demand (Priority: 4/5): The conversation covers the unit economics of renting GPUs, why scale changes pricing power, and how large contiguous fabrics of 10,000+ GPUs become scarce and difficult to replicate. Power, data centers, and cooling as the new bottlenecks (Priority: 5/5): The biggest constraints are not just GPUs but power delivery, data center capacity, heat removal, and reliability. The industry is moving toward direct-to-chip liquid cooling and better baseload power access. NVIDIA’s software and hardware moat (Priority: 5/5): NVIDIA’s CUDA software, InfiniBand networking, and training fabric are presented as key reasons customers stay on NVIDIA, especially for large-scale training workloads. Inference growth and AI commercialization (Priority: 4/5): Inference is expected to dominate future GPU demand as user queries scale. Products like copilots, search, and AI-driven ads are highlighted as the first major commercial use cases. Long-term market outlook and capacity buildout (Priority: 5/5): CoreWeave says it is sold out for the year, rapidly expanding staff and data centers, and does not expect infrastructure supply to catch up until late this decade.
Key Arguments: AI software adoption is scaling faster than any prior technology, forcing a rebuild of physical cloud infrastructure at unprecedented speed. Legacy hyperscaler clouds were designed for serializable workloads like websites and storage, not highly parallel AI compute. CoreWeave’s differentiation comes from a proprietary orchestration layer plus physical infrastructure optimized for AI rather than adapted from web hosting. Scale changes economics: a single GPU is commodity-like, but a 10,000-GPU contiguous fabric is rare and engineering-intensive, creating scarcity and pricing power. Power is a major constraint, but data center capacity, network fabric, and cooling are equally critical bottlenecks. GPU-based workloads are more power-efficient than CPUs for the same AI task, even though they consume more power per unit of density. NVIDIA remains the dominant platform because its software stack, driver ecosystem, and InfiniBand networking create high switching costs. Inference, not just training, will drive the next wave of demand because every user query adds GPU load. AI will first scale in products that fit into existing user workflows, especially copilots and integrated assistance. Advertising is likely to become a major AI use case because generative personalization can greatly improve ad effectiveness. The industry may not normalize until near the end of the decade because supply of power, data centers, and specialized infrastructure lags demand. Open-source or alternative accelerators may exist, but large-scale consumers are expected to remain on NVIDIA for performance and compatibility reasons.
Data Points: CoreWeave data centers built this year: 28 across North America - The guest says the company is building 28 data centers this year and still cannot keep up with demand. Revenue growth target: About 10x this year - CoreWeave expects revenue to increase tenfold year over year. Employee count: About 500 today, closer to 800 by year-end - The company is rapidly expanding headcount to support buildout. GPU cluster size: 10,000 to 30,000 GPUs - CoreWeave serves very large contiguous fabrics used for model training and inference. Typical server configuration: 8 GPUs per server - The guest describes the node/server architecture used in its infrastructure. Approximate server cost: Upwards of $250,000 - A server loaded with eight high-end GPUs can cost a quarter million dollars or more. GPU hourly rental price: About $4 per hour per GPU - Used as a rough example of rental economics for H100-class chips. Power cost share: Roughly 10% of cost - The guest says power is around a tenth of delivery cost; infrastructure depreciation is the bigger expense. Infrastructure depreciation life: 6 years - GPU infrastructure is depreciated over a six-year life in the discussion. Cooling and networking overhead: 0.2 to 0.3 units for every 1 unit of energy - Force-air cooling and networking add substantial energy overhead beyond the compute load. Liquid-cooled efficiency ratio: About 1.1 instead of 1.3 - Direct-to-chip cooling is expected to improve overall efficiency versus current forced-air systems. Heat output: Upwards of 100 decibels - The guest notes extreme noise in dense GPU data centers due to heat and airflow. Connection complexity: 48,000 discrete connections in a 16,000-GPU fabric - He cites the cabling and networking complexity of large contiguous GPU clusters. Fiber cabling: 500 miles - A 16,000-GPU fabric can require hundreds of miles of fiber optic cabling. Inference cost: Cents per query - The guest says typical AI query costs are measured in cents, depending on workload. Meta-like scale example: Low millions of GPUs - Used as an illustration of the scale of planned AI infrastructure by major players.
Pivotal Quotes: "It wasn't built for parallelizable workloads. And it's like you're having to rebuild the cloud... at the pace of AI software adoption." — Brannon McBee: Explaining why traditional cloud architecture cannot handle modern AI workloads. "We actually built our cloud from a no-compromises engineering solution for running AI workloads and highly parallelizable workloads." — Brannon McBee: Describing CoreWeave’s core product strategy and differentiation. "We just don't see the path to resolve the amount of infrastructure that needs to be built... within, you know, at minimum, the next few years." — Brannon McBee: On persistent demand and the long timeline for supply to catch up.
Implications: AI’s growth is turning compute, power, cooling, and networking into strategic constraints. Winners will be firms that secure infrastructure early, especially for training and inference at scale, while NVIDIA and AI-native cloud operators remain advantaged.
About This Week in Startups
Jason Calacanis covers startups, tech, markets, media, and all the hottest topics in business and technology. He also interviews the world’s greatest founders, operators, investors, and innovators.