Best of: The future of AI coaching

Episode Summary

Executive Summary: Stanford’s James Landay argues AI is moving beyond productivity into coaching and tutoring. He describes AI fitness coaching that uses motivational interviewing, sensor data, and specialized agents, plus personalized outdoor learning systems for elementary students that improved engagement, writing, and learning. He also defines human-centered AI as designing for users, affected communities, and society.

Main Topics: AI as a personal fitness coach (Priority: 5/5): Landay explains how large language models can support health and fitness goals by handling qualitative barriers, not just quantitative tracking, and by simulating a coach-like conversation. Building GPT Coach with specialized agents (Priority: 5/5): Rather than using off-the-shelf ChatGPT, the team splits tasks across agents, uses motivational interviewing, and incorporates watch/phone data to keep coaching focused and personalized. Interdisciplinary health design and personalization (Priority: 4/5): The health work is shaped by collaboration with public health and medicine experts, whose coaching manuals and methods help define the system’s behavior and style. Smart Primer: AI-powered elementary education (Priority: 5/5): Landay’s education research uses narrative, augmented reality, and LLMs to create personalized outdoor learning experiences about planets, sustainability, and local ecosystems. LLMs improving student writing and engagement (Priority: 5/5): In the Moon Story and Acorn projects, the LLM provides feedback in-character, prompting children to write more and showing learning gains beyond traditional activities. Human-centered AI beyond the individual user (Priority: 4/5): Landay defines human-centered AI as considering users, impacted communities, and society, because AI systems can create side effects far beyond the direct user.

Key Arguments: AI can function as a coach or tutor, not just a productivity tool, because it can interpret qualitative context and adapt advice to real-life constraints. Off-the-shelf chatbots are insufficient for coaching; useful systems require structured agents, guardrails, and a coaching method such as motivational interviewing. Existing fitness apps are too quantitative and often fail to address schedule conflicts, barriers, and motivation, which is where LLMs add value. Expert knowledge from clinicians and coaches can be encoded through manuals and collaboration to shape a more credible coaching style. Ambience matters: small, glanceable displays can create ongoing awareness of progress without requiring users to open an app. Educational AI should move children into the real world, using narrative and AR to support learners who do not thrive in factory-style classrooms. LLMs can enhance, not replace, writing by giving personalized feedback and prompting more reflection and longer responses. Human-centered AI must include community-centered and society-centered design because AI affects people who are not necessarily the primary users.

Data Points: Initial coaching study duration: 30-60 minutes - Landay says GPT Coach replicates the first meeting with a coach. Planned short-term study duration: 3 weeks - Used to debug the next version of the fitness coach interface. Planned longitudinal study duration: 3-6 months - Target duration to test whether users hit goals better than a control group. Prior user data used for coaching: 3 months - The system can use three months of historical watch/phone data to inform the conversation. Moon Story learning assessments: Pre-test, post-test, and follow-up a few weeks later - Used to measure educational gains from the Moon Story project. Acorn study effect: Large effect sizes - Landay says the Acorn study produced the biggest learning gains of any education study he has done with LLMs. Institute age: A little over 5 years - Landay references the Stanford Institute for Human-Centered AI's age.

Pivotal Quotes: "Can we use this AI as part of a coach or a tutor to help us get from where we are to where we want to be?" — Russ Ulman: Framing the episode’s core question about AI’s next major use case. "We can use that type of model to get people's real needs for coaching." — James Landay: Explaining why large language models are useful for coaching beyond numeric fitness tracking. "Human-centered AI means we need to design at the user level, but also community level and society level." — James Landay: Defining his broader framework for responsible AI design.

Implications: The episode suggests AI’s biggest near-term value may be adaptive coaching and tutoring. For industry, success will depend on domain expertise, personalization, and responsible design. For listeners, AI can become a practical partner in health and learning, not just a chatbot.