Anjney Midha's Plan to Radically Lower the Price of Compute

Episode Summary

Executive Summary: The episode explores how AI’s apparent “ephemerality” is constrained by very real physical and economic bottlenecks—compute, power, chips, talent, and infrastructure. Guest Anjane Mittal argues frontier AI is jagged across multiple domains, that verifiable feedback drives the fastest progress, and that his company AMP PBC can standardize and better allocate compute like a utility grid, reducing waste and enabling more efficient AI development.

Main Topics: Physical constraints behind AI progress (Priority: 5/5): The hosts frame AI as an abstract technology increasingly limited by tangible resources: energy, chips, data centers, real estate, and even space. They compare today’s AI buildout to the paperclip thought experiment, but driven by human investment rather than rogue AI. Jagged frontier and multi-frontier competition (Priority: 5/5): Mittal argues there is not one AI frontier but many—coding, chat, video, etc.—and different labs lead in different domains. This supports the view that models are not pure substitutes and that frontier leadership is fragmented. Verifiable feedback as the engine of AI improvement (Priority: 5/5): A major theme is that AI improves fastest where outputs can be checked objectively, such as software engineering and materials science. The guest contrasts these with subjective tasks like creative writing or therapy, where progress is slower and less reliable. AMP PBC’s compute standardization thesis (Priority: 5/5): Mittal explains AMP as a software layer that makes heterogeneous compute fungible, routing workloads across chips and clouds to create a compute grid. The goal is to reduce waste from fragmented leasing and improve utilization. Utilization, economics, and long-term leases (Priority: 4/5): The discussion highlights how AI labs often overpay due to low utilization and inflexible long-term compute contracts. AMP aims to reclaim stranded capacity, lower effective costs, and reallocate compute dynamically across training and inference. Technical literacy and AI governance (Priority: 4/5): Mittal argues leaders must understand how AI systems work rather than relying on simplistic sandbox or black-box assumptions. He warns that outsourcing understanding leads to poor deployment choices, overconfidence, and governance blind spots. Model choice, routing, and commodification (Priority: 4/5): The conversation ends on whether AI models are becoming commoditized and whether users will care about the underlying model. Mittal says most businesses will want outcomes, not model brands, and will increasingly route tasks to whichever model is cheapest and best suited.

Key Arguments: AI’s growth is bounded by real-world constraints, so the industry is competing for electricity, chips, compute, and talent, not just algorithms. There are multiple AI frontiers, not a single winner-take-all race; different companies may dominate different product domains. Progress is fastest when models receive verifiable feedback from the real world, especially in coding and physics-based tasks. Many AI workflows are wasteful because research demand is spiky and compute is leased in inflexible blocks, causing major idle capacity. AMP’s core idea is to standardize compute into a fungible utility, similar to how grids standardized electricity distribution. Software layers can raise compute utilization dramatically, improving economics without needing every lab to own custom silicon. Leaders need technical literacy to understand model limits, routing, and deployment risks; otherwise they outsource their understanding to the model. Most corporate users will not care which model powers a task as long as the work is done securely, cheaply, and well.

Data Points: Date referenced in discussion: June 4, 2026 - Hosts mention the frontier-model landscape as of that date. Anthropic initial raise target: $500 million - Mittal says he and the founders tried to raise this amount initially but failed. Anthropic early capital raised: ~$100 million - The first round was scraped together mostly from angels after failing to raise the full target. Anthropic compute and capital partnership: $4 billion - Amazon partnership became a major infrastructure and compute arrangement for Anthropic. Ubiquity Six funding: ~$47 million - Mittal says his prior company raised about this amount from benchmark-style investors. AMP / grid utilization improvement: From 50-60% to 95-96% - Mittal claims the software layer can improve utilization at incubated labs or on the grid. Google utilization benchmark: ~99% - He cites Google’s internal efficiency as the benchmark for good utilization. Average independent data center utilization: <70% - He says the broader independent ecosystem runs below this level. Colossus 2 node utilization: <60% - Mittal cites this as an example of low node utilization in a major cluster. Colossus 2 MFU: <11% - He distinguishes node utilization from model flop utilization within the chip. Compute economics spread: $2.50/hour marketed vs $25-$28 effective - He says wastage from low utilization creates a large gap between sticker price and actual effective cost. Long-term compute rentals: 2x increase over six months - He says compute rental prices for 2026 capacity have doubled from January to now. AMP staffing / scale claim: Fewer than 5,000 people at Anthropic - Used as a comparison to show how efficient a frontier lab can be relative to larger organizations. AMP compute market framing: 80 cents of every R&D dollar to NVIDIA - He argues labs spend the majority of R&D budget on chip providers.

Pivotal Quotes: "There are many frontiers to be conquered and pioneered. And it's not just one frontier." — Anjane Mittal: He explains why the AI market should be viewed as multiple overlapping competitive arenas rather than one model race. "Technical literacy should be non-negotiable." — Anjane Mittal: He makes the case that leaders cannot safely rely on black-box or sandbox-only thinking when deploying AI. "The model produces tokens. If the end user is only using tokens, then as long as... enough diversity in the end user base... things actually even out." — Anjane Mittal: He describes why AMP’s compute grid can balance workloads across training and inference at scale.

Implications: AI winners may be determined less by raw model quality and more by infrastructure efficiency, routing, and task-specific deployment. Expect greater commodification at the user layer, while compute allocation and utilization become strategic advantages.