How CoreWeave Sees the Market for Compute Right Now

Episode Summary

Executive Summary: The episode examines the rapid growth of AI inference demand and what it means for CoreWeave, data-center capacity, GPU economics, and the emergence of a possible compute market. Brandon McBee argues demand is still unrelenting, but the real bottlenecks have shifted from chips to powered shells, logistics, and operational execution, while also making the case that NVIDIA remains the dominant choice for both training and inference.

Main Topics: Explosion in AI inference demand (Priority: 5/5): The hosts frame inference as the new center of gravity in AI spending, with companies discovering that token usage can quickly become expensive and CFOs reacting to budget overruns. CoreWeave’s customer diversification (Priority: 5/5): McBee explains that CoreWeave has moved beyond a narrow base of hyperscalers and AI labs into a broader enterprise client mix, including financial-services customers and direct enterprise contracts. Infrastructure longevity and model routing (Priority: 4/5): The conversation highlights the idea that not every workload needs the newest frontier model, implying that model routing could extend the useful life of GPUs and reshape depreciation assumptions. NVIDIA’s continued dominance (Priority: 4/5): Despite speculation about alternative silicon, McBee says CoreWeave still sees NVIDIA as the most efficient, scalable, and reliable option across training and inference, with demand for non-NVIDIA hardware still limited. Bottlenecks shifting from GPUs to power and execution (Priority: 5/5): The supply constraint is increasingly about powered shells, electrical infrastructure, electricians, backup systems, and the ability to actually deliver billable GPU hours, rather than just securing chips. Financing AI infrastructure (Priority: 4/5): CoreWeave describes a maturing financing stack backed by long-duration take-or-pay contracts, SPVs, insurance capital, and improved investor confidence as execution risk has fallen. Prospects for compute markets and commoditization (Priority: 5/5): The hosts and guest debate whether GPU compute can become a tradable commodity; McBee argues fungibility is still lacking because performance varies widely by deployment and operation.

Key Arguments: Inference demand is real and still expanding; there is no sign of a meaningful pullback in CoreWeave’s customer base. Enterprise adoption is broadening AI use beyond labs and hyperscalers, making demand less dependent on a small set of frontier-model users. Model routing should allocate workloads to cheaper, appropriately capable models, which could reduce waste and extend the life of existing GPU fleets. NVIDIA remains the default platform because it is proven, reliable, and supported by a mature ecosystem; alternatives are still mostly experimental. The biggest constraint in AI infrastructure is now powered-shell availability and execution capacity, not simply chip supply. CoreWeave’s ability to monetize infrastructure through long-term contracts and structured financing has lowered its cost of capital. A liquid compute futures market is unlikely in the short term because compute is not yet fungible enough across clouds and deployments.

Data Points: CoreWeave clients: 9 of the top 10 AI labs - McBee says CoreWeave serves nearly all major AI labs. Enterprise client growth: Twice as many logos in Q4 as in any previous quarter - Used to illustrate rapid diversification of CoreWeave’s enterprise customer base. Financial services backlog: Tens of billions of dollars - McBee says financial-services demand is now material and growing. Direct financial services backlog: Approaching $10 billion - Clarification that this refers to direct financial-services customers, not via AI labs. Client count: Over 10 clients above $1 billion - McBee cites the number of very large clients on the platform. Financing raised year to date: Over $21 billion - Shows scale of capital raised to support infrastructure buildout. Active power: Over 1 gigawatt - McBee cites the amount of active power in CoreWeave’s deployments. Contract duration trend: 3 years → 4 years → 5 years - Take-or-pay contract terms have lengthened over time as customers seek longer access. Contribution margin: 25% - McBee describes economics inside the SPV financing stack. DDTL4 financing pricing: SOFR + 225 - Pricing on a first-of-its-kind investment-grade GPU financing deal. GPU access at small scale: On the order of $5 for a hobby project - Joe notes it was relatively easy and inexpensive to get limited GPU capacity for a small model project. Inference share of utilization: Well in excess of 50% - McBee says inference workloads now account for a majority of infrastructure utilization on CoreWeave's platform. Electrician training period: Five-year plus apprenticeship - Used to explain why labor is a structural bottleneck for data-center buildout.

Pivotal Quotes: "we're not seeing any pullback on what they're doing on inference today. If anything, it just remains this unrelenting demand" — Brandon McBee: On whether AI customers are slowing their spending or usage "the bottleneck today is having a powered shell" — Brandon McBee: On the current limiting factor for scaling data-center capacity "GPU compute today is not fungible" — Brandon McBee: On why a liquid, standardized compute market may be hard to build

Implications: AI spending is shifting from experimentation to operational scale, making infrastructure, power, and financing the new choke points. If model routing and better efficiency improve, costs may stabilize; if not, demand for high-end compute could keep expanding faster than supply.