Can California Regulate A.I.? + Silicon Valley’s Super Babies + System Update!

Episode Summary

Executive Summary: The episode covers three major tech stories: California’s wave of AI regulation, the rise of fertility tech in Silicon Valley, and a fast-moving news roundup including OpenAI’s record fundraise, Reddit’s protest limits, a deepfake scam targeting a senator, and Sonos’s app backlash. The hosts argue regulation is inevitable, but the form it takes will shape how burdensome and effective it becomes.

Main Topics: California’s AI laws and SB 1047 veto (Priority: 5/5): The hosts explain the 18 AI-related bills Governor Newsom signed into law and the veto of SB 1047, a controversial bill aimed at regulating large model developers directly. The discussion contrasts use-based regulation with model-level regulation and debates whether the veto was a setback or a strategic mistake by industry. Why state-level AI regulation matters (Priority: 4/5): They argue that California’s laws matter far beyond the state because many major AI firms are based there, operate there, or sell into its market. In the absence of federal action, California can effectively set national norms, much like it has with vehicle emissions standards. Fertility tech and Silicon Valley pronatalism (Priority: 5/5): Julia Black joins to discuss the surge of interest in fertility technology, from embryo screening to speculative tools like artificial wombs and IVG. The conversation explores pronatalism, elite investment in reproduction, and the ethical tension between helping families and enabling eugenics-like selection. OpenAI’s massive fundraising round (Priority: 4/5): In System Update, the hosts cover OpenAI’s $6.6 billion raise at a $157 billion valuation, led by Thrive Capital and joined by major investors. They question whether the company’s revenue growth can offset its heavy cash burn and note the strategic importance of possible employee liquidity. Platform power, protest, and moderation on Reddit (Priority: 3/5): Reddit changes its rules so moderators can no longer easily flip communities private without admin approval, reducing a major tool of protest. The hosts frame this as a platform response to prior moderation revolts and a broader example of companies restricting user dissent. Deepfakes, scams, and trust in communication (Priority: 4/5): A real-world deepfake incident involving Senator Benjamin Cardin underscores how convincing synthetic audio/video can be in political or business settings. The hosts suggest code words and other verification practices may soon be necessary. Sonos’s app debacle and corporate accountability (Priority: 3/5): Sonos responds to its widely criticized app rewrite with a seven-point trust-repair plan, including better testing, humility, customer experience focus, and a new quality ombuds person. The hosts mock the vagueness of the promises while noting the company is tying executive bonuses to app quality.

Key Arguments: California is acting as an AI regulator because the federal government has largely failed to pass meaningful tech rules, making state-level law the de facto national standard. Most of the laws Newsom signed focus on disclosure and misuse prevention—e.g., deepfakes, CSAM, voice cloning, AI robocalls, election content—rather than banning AI outright. SB 1047 would have regulated frontier model developers through safety testing and liability for catastrophic harms, but it was vetoed despite being watered down from earlier versions. The hosts argue the AI industry may regret killing SB 1047 because a patchwork of use-based regulations can be more expensive and annoying than a few broad rules. Fertility tech is being driven not only by consumer infertility needs but also by Silicon Valley pronatalism, crypto/decentralized-science ideology, and status signaling among elites. Embryo screening technologies can reduce risk from some serious genetic disorders, but the field risks sliding toward controversial trait selection and eugenics-adjacent uses. OpenAI’s huge fundraise shows extraordinary market demand, but also highlights the company’s enormous cash burn and dependence on continued investor confidence. Deepfake capabilities are moving from hypothetical to operational, making identity verification and authentication increasingly important in both politics and business. Platform companies like Reddit and Sonos are using product and policy changes to regain control, but these often provoke backlash when users feel their agency is being reduced.

Data Points: California AI bills signed: 18 - Number of AI-related laws Governor Gavin Newsom signed into law. SB 1047 veto date: September 29 - Date Newsom vetoed the controversial AI model-liability bill. Training-data disclosure start date: January 1, 2026 - Companies publicly releasing AI systems in California must disclose training data sources starting on this date. SB 1047 applicability threshold: north of $100 million - Bill would have applied to models with development costs above this level and massive compute usage. Catastrophic harm threshold: more than $500 million - SB 1047 defined qualifying harms partly by damages above this amount. OpenAI fundraising: $6.6 billion - OpenAI’s new funding round announced in System Update. OpenAI valuation: $157 billion - Valuation reached after the new fundraising round. OpenAI expected 2024 sales: $3.7 billion - Projected sales mentioned by the hosts. OpenAI expected 2025 sales: $11.6 billion - Projected sales mentioned by the hosts. OpenAI projected loss this year: about $5 billion - Estimated cash burn discussed in relation to the company’s growth. Orchid embryo testing cost: $2,500 per embryo - Price for pre-implantation polygenic testing discussed during the fertility-tech interview. Fertility companies’ investment level: hundreds of millions of dollars - Approximate amount investors have poured into fertility technology startups. Deepfake legislative context: 2024 election year - Why AI misinformation and election-content regulation was prioritized in California.

Pivotal Quotes: "If you want to support the work we do and the work that my colleagues at The New York Times do on all of our great shows, you can go to nytimes.com slash podcasts to learn more about this new audio subscription." — Kevin Roose: Explaining The New York Times’ new audio subscription model at the top of the episode. "The main requirements of this bill, Kevin, were that one, it required safety tests for models that had a development cost north of $100 million and used a ton of computing power." — Casey Newton: Summarizing SB 1047 before discussing Newsom’s veto. "I think what the AI industry had in SB 1047 was a chance to create regulation and rules for a new technology when there was no crisis that they were dealing with." — Kevin Roose: Arguing that the tech industry may regret blocking pre-crisis regulation.

Implications: AI regulation is coming, but the balance between model-level and use-level rules will determine costs, compliance burdens, and safety outcomes. Fertility tech is advancing quickly, raising both hope and ethical risk. Deepfakes and platform control issues show trust and verification are now central tech problems.