Ep 93 - From Reading Papers in the Gym to a Billion-Dollar AI Company | Cohere's Untold Story

Episode Summary

Executive Summary: Aidan Gomez traces Cohere’s path from rural Ontario and early AI research to building enterprise-focused LLMs. He argues AI adoption has shifted from novelty to implementation, with the main bottleneck now being integration, security, and access to enterprise data. He also says the next major breakthroughs are reasoning and continual learning, but the industry will likely optimize for ROI rather than frontier scale alone.

Main Topics: Aidan Gomez’s upbringing and path into AI (Priority: 5/5): He grew up in rural Ontario with limited internet, learned to code out of curiosity, and later entered Toronto’s AI ecosystem through UofT and mentorship from Geoff Hinton. The early research journey and Google/Transformer era (Priority: 5/5): Gomez describes an unconventional entry into Google, working across research ideas with Lukas and Noam Shazeer, and the environment that led to the transformer-era breakthroughs. Why Cohere is enterprise-first and cloud-agnostic (Priority: 5/5): Cohere builds LLMs for enterprises, emphasizing deployment flexibility across clouds and on-prem, data privacy, and deep model customization rather than consumer products. Enterprise AI adoption has shifted from curiosity to execution (Priority: 5/5): Customers now arrive with concrete workloads, benchmarks, and pricing questions; the market has moved beyond asking whether AI is useful and is now focused on implementation. Key enterprise use cases (Priority: 4/5): The biggest immediate use cases are customer support, knowledge-worker copilots, retrieval-augmented research, and broader workflow automation/RPA. Model limits, scaling, and the next frontier (Priority: 5/5): He argues that reasoning and continual learning are the next major missing capabilities, while frontier-model spending is increasingly hard to justify economically. Synthetic data, security, and integration as bottlenecks (Priority: 4/5): Synthetic data helps in verifiable domains like math and code, but enterprise adoption depends on secure access to data and systems plus lots of custom integration work.

Key Arguments: Aidan Gomez says his success was shaped by adaptability; Cohere survived by continually changing with the market and environment. He argues the enterprise opportunity is larger and more durable than consumer AI because businesses need infrastructure, privacy, and optional deployment. He believes cloud lock-in is a major risk for enterprises, so Cohere’s cloud-agnostic/on-prem deployment is a key advantage. He says the market has moved from "What can LLMs do?" to "Can you solve this specific workload, at this benchmark, for this price?" He claims the hardest problem now is not proving AI usefulness but integrating models into secure enterprise systems with the right data and tools. He argues reasoning and continual learning are the two biggest missing technical features in modern models. He says frontier spending has become economically inefficient; being slightly behind the frontier can yield better ROI. He predicts solutions integrators like Accenture and Deloitte will benefit because enterprise AI deployment requires broad refactoring and hands-on implementation. He says synthetic data is powerful in domains with verifiable correctness, especially math and code, but is risky in open-ended domains. He emphasizes that models cannot do human work unless they have access to the same tools, systems, and information humans use.

Data Points: Cohere company focus: Enterprise AI / large language models - Gomez describes Cohere as building LLMs for enterprise customers rather than consumers. Deployment model: All clouds + on-prem - He says Cohere is available across clouds and on-prem to avoid lock-in and fit regulated customers. Financial institutions on-prem share: 60% to 80% on-prem - He cites financial institutions as often running most workloads on-prem with some cloud usage. Time to read a paper early on: About 2 weeks per paper - He explains how slowly he initially learned research papers, reading them sequentially and looking up prerequisite concepts. Cost to build a GPT-4-like model: From hundreds of millions of dollars to maybe $10 million - He uses this to argue that model-building costs have dropped sharply and frontier investment is becoming less rational. Capital horizon for frontier spending: Incremental $5 billion - He frames the frontier ROI question as what an extra $5B actually buys at this stage. Research/internship timeline: Third-year undergrad - He says he was mistakenly treated as a PhD intern at Google and corrected the record on his last day. Horizon for integration work: Next half decade - He predicts the main enterprise AI challenge will be months-to-years of integration work across systems and teams. Number of major technical gaps named: 2 - He highlights reasoning and continual learning/lifelong memory as the two biggest missing capabilities.

Pivotal Quotes: ""The ability to adapt to those environments and change yourself and the company to fit, that's been the only way that we've survived."" — Aidan Gomez: He explains the founding lesson from building Cohere through rapidly changing AI market conditions. ""If I could capture every thought in your head for your entire, you know, hopefully 100 year lifespan, that would be my career."" — Aidan Gomez: He describes the kind of data that would most accelerate reasoning and continual learning in AI. ""I think we're definitely in the point where investors have just said, no, like, sorry, we're not going to give you that extra X billion dollars."" — Aidan Gomez: He argues frontier model spending is facing stronger capital discipline and weaker ROI justification.

Implications: Enterprise AI is moving from experimentation to deployment, and value will go to vendors that solve integration, security, and workflow fit. The next wave likely centers on reasoning, memory, and practical automation, not just bigger models.