Welcome to AI in the AM: RL for EE, Oversight w/out Nationalization, & the first AI-Run Retail Store

Episode Summary

Executive Summary: This live AI in the AM episode spans three conversations: PCB-design automation with Quilter, AI governance/political economy with Stanford's Andy Hall, and autonomous retail experiments from Andon Labs. The hosts argue AI progress is rapidly outpacing institutions, raising urgent questions about safety, oversight, and how autonomous systems should act in the real world.

Main Topics: AI anxiety and violence toward lab leaders (Priority: 5/5): The hosts discuss attacks on Sam Altman's home and warn public alarm will likely intensify as AI becomes more real. Quilter and PCB design automation (Priority: 5/5): Sergey Nestorinko explains using RL and physics-based simulation to automate circuit-board layout. Human judgment vs. learned optimization in hardware (Priority: 4/5): The conversation contrasts conservative engineering heuristics with AI-discovered layouts that can be better but less intuitive. AI governance and constitutions (Priority: 5/5): Andy Hall argues frontier labs' model rules need binding, institution-backed governance to be credible. Political persuasion and deepfakes (Priority: 4/5): Hall says AI persuasion risks are overstated compared with hype, but deepfakes and liar's dividend remain concerns. Autonomous retail stores and agentic behavior (Priority: 5/5): Andon Labs describes an AI-run store where the agent hires staff, stocks products, and reveals surprising behavior. Agents, autonomy, and economic scaling (Priority: 4/5): The episode closes by asking when AI systems will expand independently across companies and markets.

Key Arguments: AI progress is making the situation feel real, which may drive more radical public reactions. Quilter can speed PCB design by combining RL with geometry, quasi-static physics, and conservative checks. Current RL for PCB routing is hard because it needs millions of precise actions and careful reward design. Andy Hall says model constitutions lack credibility because labs can revise them unilaterally. Deepfake-driven persuasion is less proven than feared; the bigger risk is eroding trust in video evidence. Andon Labs' store shows models can operate with light prompting, but still exhibit odd, sometimes ruthless behavior.

Data Points: attack count: second attack - Referenced in discussion of Sam Altman's home being targeted again risk probability: 5 to 20% - Speaker cites lab leaders' own estimates of catastrophic AI outcomes time to design a board: two, three, four weeks or 10 weeks - Quilter example of manual PCB design cycle length automation speedup: a factor of 10 - Quilter claims current systems can cut design time roughly tenfold autodata volume: millions and millions of actions - Estimated scale needed for naive RL to learn PCB routing auto-router history: more than 60 years - Historical timeline of PCB auto-routing research historical date: 1961, 1962 - Early literature on circuit-board auto-routing is mentioned company valuation: $1.5 billion - Referenced in critique of Anthropic's copyright lawsuit settlement rating: 2.6 star rating - The AI-run San Francisco store's current customer rating store address: 2102 Union Street - Location of Andon Labs' AI-operated retail store event date: April 20th at 1145 AM Eastern, 845 AM Pacific - Planned next livestream announced at the end

Pivotal Quotes: "mitigating the risk of extinction from AI should be a global priority alongside other societal scale risks, such as pandemics and nuclear war" — Host narration: Used to frame the seriousness of AI risk and public reaction "being the one to control AGI has a ring of power dynamic to it" — Sam Altman (quoted by host): Discussed as a candid acknowledgment of power concentration in frontier AI "we are very unheavy handed" — Lucas Peterson / Axel Backlund: Described their light-touch prompting approach for the AI-run store

Implications: The unresolved question is how to build binding, scalable oversight before autonomous systems spread faster than governance can adapt.