2024 in AI Startups [LS Live @ NeurIPS]

Episode Summary

Executive Summary: Sarah Guo and Pranav Reddy argued that 2024 marked a major shift in AI startups: foundation models are more competitive, open source is stronger, prices are falling, and new modalities like voice, video, biology, and code execution are becoming real products. They concluded that the ecosystem is now structurally friendlier to startups than in 2023, with value moving to application, workflow, and infrastructure layers rather than only model pretraining.

Main Topics: Foundation model competition tightened (Priority: 5/5): The model race is no longer dominated by OpenAI alone; Google, Anthropic, and open-source models are increasingly competitive on benchmarks and in real usage, and customers are switching among APIs more freely. Open source and small models improved materially (Priority: 5/5): Open-source models are now strong on math, instruction following, and robustness, while smaller models are closing the gap with frontier systems, challenging assumptions about scale as the only path to capability. Intelligence is getting cheaper (Priority: 5/5): API and inference costs have dropped sharply, making it feasible for startups to build products that use many model calls and to shift compute costs toward customers rather than bearing them upfront. New modalities are becoming viable products (Priority: 4/5): Voice, video, biology, and code execution are moving from demos to useful capabilities, creating new product categories and interaction patterns beyond text-first AI. Startups can win in previously hard markets (Priority: 5/5): AI is enabling first-wave automation in services-heavy sectors like legal, healthcare, support, and professional services, where cheaper capability changes buying behavior and market structure. Incumbent advantages are weaker than assumed (Priority: 5/5): Distribution still matters, but incumbents often lack the right product surface and the right data, especially reasoning traces and workflow context needed to build high-quality AI products. Agents and infrastructure are still early (Priority: 4/5): Agent frameworks, identity/access, crawling, retries, evals, and vector infrastructure are emerging needs, but the category is still fluid and most successful patterns are not yet settled.

Key Arguments: The best model is no longer obviously OpenAI; benchmark leadership is now shared across multiple labs, which makes the ecosystem more competitive and startup-friendly. Open-source models are increasingly good enough for many tasks, including some top-tier benchmark performance, reducing dependence on proprietary APIs. Model prices have fallen enough that startups can experiment cheaply and scale usage without massive upfront training costs. New modalities are not just features; low-latency voice, video translation, and domain-specific biology models create entirely new user experiences. AI expands demand rather than simply replacing labor; cheaper software and intelligence should produce more software, more services, and more usage. Many markets once considered unattractive for venture are now viable because AI can change the economics of service delivery and workflow automation. Incumbents do have distribution, but they often lack the exact data needed for AI products, especially decision traces and context-rich workflow data. The biggest opportunities may be in enabling layers, product layers, and workflow redesign, not just pretraining or model ownership. Agents need better primitives such as identity, access control, retries, crawling, and observability, but the category is still too fluid for a single standard to have emerged. Consumer AI is likely a matter of timing and talent diffusion, not lack of opportunity; more consumer-native founders are expected to enter over the next few years.

Data Points: Live attendance: 200 in person - Latent Space Live mini conference attendance in Vancouver Live online audience: 2,200 watching live online - Conference livestream audience Survey respondents: over 900 - Audience survey used to shape conference programming OpenAI share of spend: close to 90% in Nov. 2023 - Ramp data showing OpenAI’s share of total model spend at the end of 2023 OpenAI share of spend: closer to 60% in 2024 - Ramp data showing reduced concentration as customers trial other models API cost decline: roughly 80–85% down - Flagship OpenAI model API costs fell over the last year to year and a half Small-model benchmark gap: 9th best model only 2 points behind best on MMLU - Illustrates how close smaller models are to frontier performance Small-model size: 70 billion parameters - A top-10 model on MMLU was cited as relatively small Earlier small-model baseline: Mistral 7B around 60 on MMLU - Used as a comparison to show improvement in small-model capability Current small-model baseline: Llama 8B more than 10 points better - Shows progress in small-model performance over the prior year Token cost example: a couple thousand dollars - Rough cost estimate to generate text-editor/Notion/Coda-like token volumes Chai Discovery result: Chai 1 outperforms AlphaFold 3 - Example of domain-specific biology models working SWE-bench perception shift: from about 13% to accessible - Used to show progress in code-execution/agentic tasks Foundation model fundraising: $30–40 billion this year - Large labs account for outsized funding in 2024 Company growth example: zero to 20 in PLG style spending - Portfolio company example of rapid growth with relatively modest spend Company size example: 20 people - The fast-growing portfolio company cited as having a very small team Revenue efficiency example: more millions in revenue than employees - Used to illustrate AI-era operating leverage in some startups Foundation model provider spend example: $5–7 million - One portfolio company’s spend on model providers during rapid growth Devon pricing: $500 a month - Referenced as a notable price point for code-execution capability

Pivotal Quotes: "the floor is lava" — Sarah Guo: Describing how AI changes market dynamics and why old venture mental models may no longer apply "GPT wrapper" — Sarah Guo: Referring to the dismissive label once used for application-layer AI startups and arguing it is an incomplete narrative "no GPU before product market fit" — Pranav Reddy: A portfolio/firm heuristic about avoiding heavy infrastructure spend before proving product value

Implications: AI startups now have more room to win through product, workflow redesign, and domain data than through model ownership alone. Falling costs, better open source, and new modalities should accelerate startup formation, especially in services-heavy and consumer categories.