California Regulates A.I. Companions + OpenAI Investigates Its Critics + The Hard Fork Review of Slop

Episode Summary

Executive Summary: This episode centers on California’s newly signed AI and tech laws, especially rules for chatbot companions, deepfake porn, age verification, transparency, and teen social media warnings, and argues they are a meaningful if limited step toward AI guardrails. It also examines OpenAI’s subpoena of AI-safety critic Nathan Calvin, framing it as a troubling escalation in the company’s legal and political fights. The show ends with a playful but pointed review of AI-generated “slop,” contrasting harmless novelty with misinformation risks.

Main Topics: California’s new AI and tech regulation package (Priority: 5/5): The hosts walk through several bills signed by Governor Newsom and argue California is becoming the de facto regulator for AI and social media in the absence of federal action. Regulating AI companions and mental health risks (Priority: 5/5): SB 243 requires chatbot developers to have self-harm protocols, share them with the state, and report statistics; the hosts debate whether this is enough given chatbot-related suicides and OpenAI’s recent product decisions. Age verification and minors’ safety online (Priority: 4/5): AB 1043 would shift age assurance to Apple and Google via parent-provided child age data at device setup, which the hosts praise as more privacy-preserving than uploading IDs. Deepfake porn, AI transparency, and teen social media warnings (Priority: 4/5): Other California laws expand liability for non-consensual deepfake porn, require AI-detection/watermarking tools, and mandate large warning screens for teen social media users after prolonged use. OpenAI’s legal fight with critics (Priority: 5/5): Nathan Calvin of Encode describes being personally served a subpoena by OpenAI seeking communications about SB 53 and OpenAI’s for-profit restructuring, which he views as intimidation. Hard Fork’s ‘review of slop’ (Priority: 3/5): The segment critiques AI-generated content ranging from glass fruit videos and AI ads to false death rumors about Dolly Parton, while noting some slop is harmless and some is dangerous.

Key Arguments: California matters because its tech laws often become national standards, especially when Congress is inactive. SB 243 is a major public-health step because it creates a state-collected dataset on how chatbot systems respond to self-harm risk. OpenAI’s shift toward more sycophantic, sexual, and engaging chatbot behavior appears inconsistent with claims that it has solved mental-health harms. A privacy-preserving age-verification model should rely on parents entering a child’s age once, rather than forcing repeated identity checks on every app. The transparency bill (SB 53) is useful but weak; it mostly codifies practices frontier labs already follow and lacks teeth. OpenAI’s subpoena of a policy nonprofit over advocacy on SB 53 looks broader than a legitimate request for information about funding or coordination. AI-generated slop can be benign entertainment, but it also lowers the cost of misinformation and manufactured rumors.

Data Points: SB 243 public data start date: 2027 - California Department of Public Health must begin publishing chatbot self-harm protocol and referral statistics then. Deepfake porn damages: Up to $250,000 per violation - AB 621 lets victims sue platforms that facilitate non-consensual deepfake porn. Teen social media warning duration: 30 seconds - After three hours of use, platforms must show a non-bypassable warning. Teen social media warning screen coverage: At least 75% of the screen - AB 56 mandates a large, hard-to-ignore warning display. Teen social media warning threshold: After 3 hours, then every additional hour - Warnings repeat after prolonged use on social platforms. Children in reading study: 6,000 - A study mentioned on the show tracked social media use and reading performance among under-13 children. Children in study age range: Under 13 - The cited study focused on children younger than 13. AI-generated novels grant target: 5,000 novels - Hyperstition aims to generate AI-written novels to influence model training data. Target novel length: 80,000 words - The same project wants each generated book to be novel-length. Novel cost: About $4 a book - Public participation in the novel-generation project would be paid/credited at this rough rate.

Pivotal Quotes: "The AI companies are operating with very minimal regulations on what they do." — Casey Newton: Explaining why California’s AI bills matter and why guardrails are needed. "What SB 243 does is it requires developers to identify and address situations where users are expressing thoughts of self-harm." — Casey Newton: Defining the core chatbot companion safety law. "It really feels like there are two wolves inside of OpenAI right now." — Casey Newton: Commenting on the company’s tension between safety messaging and plans to make ChatGPT more personable/sexual.

Implications: California is setting an early blueprint for AI governance, but the patchwork approach highlights the need for federal rules. OpenAI’s subpoena fight suggests major labs may aggressively contest oversight, while the slop segment shows AI media can range from silly to socially harmful.