High Agency Pydantic > VC Backed Frameworks — with Jason Liu of Instructor

Episode Summary

Executive Summary: Jason Liu traces his path from skeptical ML engineer at Stitch Fix to creator of Instructor, arguing that LLMs are most valuable when they return typed data structures, not free-form text. The conversation covers recommendation systems, RAG as a form of retrieval/recommendation, why workflows and DAGs may beat agentic loops, and why he prefers a lean, developer-first library plus consulting over a venture-scale startup.

Main Topics: Stitch Fix: recommendations, observability, and early LLM skepticism (Priority: 5/5): Jason explains how he built Flight, a standardized recommendation framework at Stitch Fix, to replace bespoke team systems and enable observability across pipelines. He also describes his long skepticism toward language models until ChatGPT changed his view. Instructor’s origin and design philosophy (Priority: 5/5): Instructor emerged from Jason’s desire to make structured outputs from LLMs feel as natural as requests in Python: simple, developer-friendly, and centered on typed schemas, validation, and minimal abstraction. Function calling, JSON mode, and typed outputs (Priority: 5/5): The discussion compares function calling, JSON mode, and structured outputs, with Jason emphasizing that the real value is typed responses, autocomplete, validation, and schema separation rather than just getting JSON back. RAG, retrieval, and recommendation systems (Priority: 4/5): Jason argues that RAG is essentially a modern form of retrieval/recommendation and that many LLM problems should be reframed as ranking, retrieval, and structured data transformation rather than pure generation. Workflows vs agent loops (Priority: 4/5): He advocates for workflow/DAG-based systems over endless ReAct-style loops, saying plans should be explicit, inspectable, and modifiable, especially for complex tasks and production systems. Business model, consulting, and staying lean (Priority: 4/5): Jason explains why Instructor is intentionally not a venture-backed startup: he prefers consulting, small teams, and direct access to interesting production problems over building a billion-dollar platform. Career advice, agency, and timing (Priority: 3/5): The conversation closes with advice on courage, starting now, documenting negative results, and focusing on process metrics and experimentation rather than waiting for the perfect moment or outcome.

Key Arguments: Standardizing ML infrastructure at Stitch Fix reduced duplicated work, improved observability, and let one person maintain multiple systems instead of many teams each owning bespoke code. LLMs are most useful when they produce typed data structures that developers can reason about, validate, and autocomplete against; JSON alone is not enough. Function calling is valuable because it separates schema from instructions and supports richer validation than plain JSON mode. RAG should be understood as retrieval/recommendation; many “LLM” problems are actually ranking and structured query understanding problems. For many production use cases, workflows/DAGs are better than agentic loops because they are inspectable, debuggable, and easier to optimize. Tool selection should often be handled by rankers or retrieval over tool descriptions rather than stuffing dozens or hundreds of tools into one prompt. Instructor is intentionally lightweight and developer-centric, more like requests than Django, because most users want boring plumbing handled without losing control. Jason believes many AI startups over-index on venture-scale ambitions when a profitable, small, consulting-oriented business may be the better fit. Career progress comes from courage and repeated experimentation; waiting for the perfect time or perfect skill level is usually just avoidance. Negative experimental results should be documented with conditions for when to revisit them, since model and data improvements can change outcomes later.

Data Points: Flight adoption: over 80% - Jason’s recommendation framework at Stitch Fix reportedly achieved broad internal adoption. Request volume: 350 million requests/day - Scale of the Stitch Fix recommendation framework Flight. Annual revenue impact: over $50 million - Jason attributes this level of revenue to the similarity search / embedding-driven merchandising system at Stitch Fix. Inventory size: 3 million pieces - Approximate inventory scale used to justify why off-the-shelf embeddings were insufficient. Interaction volume: over a billion interactions - User-clothing interaction data used for model training and recommendation quality. Hand injury downtime: about 1–2 years - Jason says a hand injury limited coding and shifted him toward tech lead work and later exploration. Sabbatical exploration: about 5 months - Time spent prompting and experimenting after ChatGPT before Instructor became full-time work. Anthropic support delay: about 2 weeks - Jason says he got commercial access after reaching out to sales and waiting roughly two weeks. Cost reduction: 70% - He says Anthropic’s Sonnet can be dramatically cheaper than prior options, making consumer apps more viable. Tool count guidance: 6 or 7 - Jason says he usually hasn’t needed more than this many functions in a single call. Potential tool capacity: 250+ tools - Referenced as a capability from a recent Anthropic function-calling release. Connector scale: 8,000–9,000 connectors - Zapier was cited as an example of a platform with far too many tools to fit in a single prompt. Founder success stat: ~30% - Jason cites a rough statistic that around 30% of companies reaching Series B are solo-founded. Startup revenue example: 70,000 MRR - He mentions profitable small startups as a desirable alternative to venture-scale growth.

Pivotal Quotes: "RAG is Rexis." — Jason Liu: He reframes retrieval-augmented generation as a form of recommendation/retrieval systems, connecting LLM work to classical ML infrastructure. "LLMs give you strings and Instructure gives you data structures." — Jason Liu: Core summary of Instructor’s purpose: turning model outputs into typed, usable objects for developers. "What we lack is not money or skill. What we lack is courage." — Jason Liu: Career advice segment about starting now, taking risks, and not waiting for perfect conditions.

Implications: The episode suggests the AI stack is maturing toward typed outputs, retrieval/ranking, and workflow orchestration. For builders, the winning pattern may be less “agent magic” and more disciplined data structures, observability, and developer control.