The RAISE Act: Minimum Standards for Frontier AI Development, with NY Assembly Member Alex Bores

Episode Summary

Executive Summary: The episode centers on New York Assemblymember Alex Boris’s RAISE Act, a narrowly targeted AI safety bill for frontier developers. Boris argues that large AI labs should publish safety plans, undergo third-party audits, disclose critical incidents, and protect whistleblowers, while leaving room for open source and industry flexibility. The conversation probes definitions, enforcement, state-vs-federal coordination, and how the bill could shape frontier model deployment.

Main Topics: RAISE Act overview and policy goal (Priority: 5/5): The bill seeks to establish a minimum safety floor for a very small number of frontier AI developers by requiring safety planning, auditing, incident disclosure, and whistleblower protections. Who the bill applies to (Priority: 5/5): Boris explains the bill is designed to cover only high-resource frontier labs, using financial and technical thresholds intended to exclude startups and academia while capturing the largest model developers and distillation-based successors. Risk categories and safety standards (Priority: 5/5): The discussion focuses on the bill’s narrow risk scope: CBRN-related harms and automated crime, plus requirements for reasonable care, internal testing, and disclosure when risks rise to catastrophic levels. Open source vs. closed source (Priority: 4/5): Boris defends the bill as agnostic to release model, arguing it imposes a risk-based standard rather than a blanket restriction on open source, while acknowledging open source can increase downstream safety challenges. Whistleblowers, audits, and enforcement (Priority: 4/5): The conversation examines how third-party auditors, legal protections, and modest penalties would work in practice, with concerns raised about capture, enforcement strength, and the meaning of 'reasonable' safety practices. AI, labor displacement, and public-sector use (Priority: 3/5): The host and Boris discuss broader labor and government implications of AI, including automation in call centers, worker displacement, retraining limits, and whether a new social contract may eventually be needed. Legislative process and multi-state coordination (Priority: 3/5): Boris describes outreach to industry, the possibility of amendments, and efforts to avoid a fragmented patchwork of state AI laws through reciprocity or eventual federal action.

Key Arguments: The RAISE Act is intended as a light-touch safety floor for frontier AI, not a broad restriction on innovation. The bill targets only a small number of the most well-resourced developers, using spending and model-scale thresholds to avoid sweeping in startups or academic research. Companies should be allowed wide latitude to design their own safety plans, but they should not release models when their own testing indicates catastrophic risk. Third-party audits, incident disclosure, and whistleblower protections are essential because companies can face incentives to cut corners under competitive pressure. The bill is intentionally neutral toward open source; it does not ban open releases, but it expects developers to account for the risks those releases create. Knowledge distillation is included to catch deepseek-like successors that may fall below the main frontier-model threshold but still pose similar risks. AI regulation should start with basic guardrails now to reduce catastrophic risk and avoid a future public panic that could trigger worse, more reactionary policy later. Most government work is noncontroversial, and AI policy should be approached through collaboration with industry rather than as an ideological fight. AI-related worker displacement is real, but public-sector adoption may be easier to manage because government already has vacancies and inefficiencies that AI can help reduce. If federal action doesn’t arrive soon, states can still move first, and a compact or shared standard could later reduce fragmentation.

Data Points: Legislative scope: High single-digit or low double-digit number of companies - Host and Boris describe the bill as applying only to a very small set of frontier AI developers. Company threshold: $100 million - Used as the cumulative training-spend threshold for a large AI developer under the bill. Per-model threshold: $5 million - Used in the knowledge-distillation pathway and for certain model-specific spending requirements. Frontier model technical threshold: 10^26 parameters - One route for a model to qualify as a frontier model under the bill. Critical harm threshold: 100 deaths or $1 billion in damages - The bill’s trigger for catastrophic-risk concerns tied to CBRN or automated crime risks. Whistleblower penalty: $10,000 per employee per retaliation - Penalty for retaliating against whistleblowers under the bill. Repeat offense penalty: $30 million maximum fine - Discussed in the red-teaming of the bill’s enforcement provisions. New judges in NY: 20 - Boris says he passed a bill creating 20 new judges statewide. Vacant-land revenue estimate: $800 million per year - Boris says taxing vacant land at market value in NYC could raise this much for the city budget. Bill length: About 15 pages - Boris contrasts the bill’s brevity with much longer federal legislation. State agencies/legislative throughput: ~800 bills sent to the governor; ~600 signed - Boris uses this to argue most legislative work is routine and noncontroversial. District rent burden: 40% of renters - Boris says this share of renters in his district pay more than one-third of income on rent.

Pivotal Quotes: "We are largely letting the companies grade their own homework." — Alex Boris: Boris explains the bill’s philosophy: companies write safety plans, but audits and disclosure are meant to check them. "This is meant to say that if your own testing, if your own research is saying this is a massive risk... you shouldn't be releasing that model." — Alex Boris: Boris clarifies the bill’s core no-release standard for catastrophic-risk findings. "If you are not 100% on board with every clause one way or the other, you can still participate." — Alex Boris: Boris encourages stakeholders to support or amend the bill rather than treating it as an all-or-nothing proposal.

Implications: The episode signals that frontier AI regulation may begin with modest, risk-based state rules rather than sweeping federal controls. If enacted, the RAISE Act could normalize safety planning, audits, and whistleblower reporting for the largest labs and push the industry toward stronger pre-deployment safety cases.