Why experts writing AI evals is creating the fastest-growing companies in history | Brendan Foody (CEO of Mercor)

Episode Summary

Executive Summary: Brendan Foody argues that AI progress is increasingly bottlenecked by evals and post-training data, making expert labor marketplaces like Mercor central to model improvement. The conversation covers Mercor’s explosive growth, why labs need high-caliber professionals to define and measure success, how AI is reshaping hiring and work, and why he believes humans will remain essential for years as models still need rigorous evaluation and training environments.

Main Topics: Evals as the new bottleneck in AI (Priority: 5/5): Foody frames evals as the PRD for models: before labs can improve capabilities, they need reliable ways to measure success. He argues evals now function both as product requirements and marketing/sales collateral for demonstrating model capability. Mercor’s business model and market positioning (Priority: 5/5): Mercor matches AI labs with vetted experts—engineers, lawyers, doctors, screenwriters, and others—who create evals, rubrics, and post-training data. Foody says labs increasingly need a labor marketplace, not just a data company. Explosive growth and product-market fit (Priority: 5/5): Foody recounts Mercor’s origin, early pull from top labs, and how customer demand revealed a massive opportunity. He emphasizes leading indicators, customer obsession, and solving a painful, rapidly expanding problem as keys to hypergrowth. The future of work and labor markets (Priority: 4/5): He predicts AI will not simply eliminate jobs but create new categories of work centered on training, evaluation, and AI-assisted productivity. He believes labor markets will become more unified, more software-mediated, and more selective. Skills and roles that will endure (Priority: 4/5): Foody argues elastic industries like software, product management, operations, consulting, and AI-enabled management will expand as productivity rises. He advises younger people to learn how to use AI tools effectively rather than resist them. Hiring philosophy and company culture (Priority: 4/5): Mercor’s culture is built around can-do attitude, high standards, and intensity. Foody says early talent density, speed once demand is proven, and output-oriented ownership are central to building a category-defining company. AI’s trajectory and AGI skepticism (Priority: 3/5): While highly bullish on AI’s impact over the next decade, Foody thinks superintelligence is farther away than some claim. He believes post-training, RL, and better evals—not just more pretraining data—will drive much of the next wave of capability gains.

Key Arguments: AI progress is constrained less by model access and more by the ability to define, measure, and reward success in real-world tasks. Evals serve two roles: they tell researchers what to optimize and also provide evidence to customers that a model or product works. The labor market for AI training is shifting from low-skill crowdsourcing toward high-caliber experts who can assess nuanced, professional-grade outputs. Mercor’s advantage comes from sourcing exceptional people quickly and matching them to labs’ most valuable capability gaps. The future of work will likely include many new jobs that revolve around creating AI training environments, rubrics, and verification systems. Industries with elastic demand, especially software, will grow as AI increases productivity rather than shrinking. People who learn to use AI tools well will outperform those who resist them; AI literacy is becoming a core career skill. Post-training methods like RLHF/RLAIF and expert-defined evals are more important than raw pretraining scale for improving models. Mercor’s growth was driven by clear market pull, rapid iteration, and extreme focus on customer needs rather than sales-heavy expansion. High standards, speed, and intensity are necessary to win in a market where the best customers demand exceptional results.

Data Points: Revenue run rate growth: $1 to $400 million in 16 months - Foody describes Mercor’s rapid ascent from startup to massive revenue scale. Revenue run rate growth (alternate description in intro): $1 to $500 million in 17 months - Podcast introduction frames Mercor as the fastest-growing company in history based on a later milestone. Valuation: $2 billion - Foody says Mercor recently raised $100 million at a $2 billion valuation. Fundraise amount: $100 million - Recent funding round mentioned in the episode introduction. Net retention: Over 1600% - Podcast intro states Mercor’s customer expansion rate. Pay rate: Median $95/hour - Foody discusses compensation for experts in Mercor’s marketplace. Upper pay rate: Up to $500/hour - Used for especially deep expertise in the marketplace. Crowdsourcing benchmark pay: Around $30/hour - Foody contrasts Mercor’s higher-skilled labor model with legacy crowdsourcing firms. Hiring turnaround: Within 24 hours - He says Mercor can source experts like award-winning screenwriters extremely quickly. Initial scaling target: $50 million revenue run rate by end of year - Foody recalls promising Benchmark the company would hit this target. Initial team size milestone: First 10 hires - He emphasizes extreme patience and quality in the earliest hires. Market size/workforce: Tens of thousands at any given time; hundreds of thousands overall - Foody estimates the scale of people working on AI training and evaluation tasks. Customer concentration: No customer churn - Stated in the podcast intro as evidence of product-market fit. Operating status: Lifetime profitable - Foody says Mercor has remained capital efficient and never burned money.

Pivotal Quotes: "If the model is the product, then the eval is the product requirement document." — Brendan Foody: He explains why evals are foundational to model development and commercialization. "We were put on Earth to create reinforcement learning training data for labs." — Brendan Foody: He summarizes the role humans may play in the AI economy as model trainers and evaluators. "You can just do stuff." — Brendan Foody: His personal motto and advice to founders and builders to take initiative and ship.

Implications: For builders, eval literacy and AI fluency are becoming essential. For companies, the winners will be those that define measurable outcomes, embrace AI-enabled abundance, and recruit top experts to push model capability forward.