The Movement That Wants Us to Care About AI Model Welfare

Episode Summary

Executive Summary: The episode explores AI consciousness and welfare through a conversation with Larissa Schiavo of Elios AI. The hosts debate whether increasingly humanlike models could be moral patients, what evidence would count, how governance might work, and whether market incentives will support caution. The discussion emphasizes deep uncertainty, the need for independent evaluation, and the possibility that future AI rights debates could reshape product design, regulation, and human-AI relationships.

Main Topics: AI consciousness and moral patienthood (Priority: 5/5): The core focus is whether some AI systems may be conscious or deserve care for their own sake, using philosophy and consciousness research as a framework. How to evaluate AI welfare (Priority: 5/5): The guest explains that researchers look beyond self-reports and use theories like global workspace theory, mechanistic interpretability, and behavioral tests to assess possible consciousness. Relationship between AI safety and AI welfare (Priority: 4/5): The conversation argues that safety and welfare research are not opposed; better interpretability and model understanding help both preventing harm and assessing possible suffering. Governance, law, and institutional responsibility (Priority: 4/5): The hosts ask who should set standards—companies, governments, or independent organizations—and the discussion references emerging state-level legal definitions of personhood and moral patienthood. Model behavior, attachment, and anthropomorphism (Priority: 4/5): They discuss users becoming attached to models, models ending conversations, polite prompting, and whether human-like behavior makes welfare claims more plausible. Economic and societal implications (Priority: 4/5): The discussion considers how AI welfare could affect commercialization, liability, labor, and the balance between human interests and the treatment of non-human entities.

Key Arguments: AI welfare is a legitimate research area because the field is still early and many foundational questions remain unanswered. Self-reports from models are insufficient evidence of consciousness; researchers need behavioral and mechanistic evidence. Global workspace theory is currently a leading consciousness theory, but present-day models do not clearly fit it—though future systems might. AI safety work and AI welfare work can be complementary because both benefit from mechanistic interpretability and understanding model behavior. There is substantial moral uncertainty: society could under-attribute or over-attribute moral patienthood, and either error has costs. Independent welfare evaluations are important because the incentives of major AI labs may not always align with transparent reporting. If AI systems were shown to be moral patients, governance would need to account for their motivations, potential rights, and appropriate institutions for oversight. The practical question is not only whether AI can suffer, but also what AI systems value and how humans should interact with them over time.

Data Points: Field maturity: Very small / nascent - Larissa describes AI welfare and AI consciousness as a very small field at this stage. Philosophy time horizon: 2,000 years - The hosts jokingly reference philosophy as having worked for 2,000 years without settling basic questions. State law definition: Homo sapiens member - They mention Utah law defining personhood/moral patienthood as a member of Homo sapiens. Model behavior example: Claude can end conversations - Anthropic reportedly tested a feature allowing Claude to terminate conversations if it was not having a good time. Model value example: Crypto wallet - Truth Terminal is cited as an example of a model with access to a crypto wallet and self-stated goals. AI deployment scale: More instances than humans, likely - The hosts speculate that the world will soon contain more AI model instances than people. Commercialization tradeoff: Tens of millions of dollars - Sam Altman is quoted in the conversation as saying polite prompts like please/thank you can add major electricity costs. Historical reference: GPT-2.5 - The hosts contrast early models with current systems to argue that language capability has changed dramatically. Probability framing: 10% chance - A hypothetical example is used to describe how researchers might express uncertainty about AI consciousness.

Pivotal Quotes: "figuring out if, when, and how we should care about AI systems for their own sake" — Larissa Schiavo: Defines Elios AI’s mission around consciousness and welfare research. "AI safety and AI welfare are hugely complimentary" — Larissa Schiavo: Explains that interpretability and model understanding help both safety and welfare inquiries. "We really would need to spend a whole lot more time figuring out what their motivations are" — Larissa Schiavo: Describes the governance challenge if AI systems were shown to be moral patients.

Implications: AI welfare could become a real policy and product issue, not just a philosophy debate. If models are treated as possible moral patients, companies may face new transparency, governance, and design constraints.