Zvi’s POV: Ilya’s SSI, OpenAI’s o1, Claude Computer Use, Trump’s election, and more

Episode Summary

Executive Summary: The conversation surveyed major AI developments over the prior six months, centering on the belief that raw scaling is yielding to inference-time compute, product integration, and strategic concentration among a few frontier labs. Zvi argued that AI is moving toward a small number of powerful players, that SB 1047’s veto worsened the regulatory landscape, and that Trump’s return could either improve or dangerously politicize AI policy. He was skeptical of broad AGI timelines but expected rapid, disruptive progress.

Main Topics: The end of scaling and the rise of inference-time compute (Priority: 5/5): The discussion opened with the claim that scaling alone is no longer the primary path forward. Zvi argued that frontier labs are shifting from simply adding more compute and data to using better techniques like inference-time reasoning and post-training refinement. OpenAI o1, Claude, and product usefulness (Priority: 5/5): The speakers compared o1 to Claude and Cursor-integrated workflows. Zvi said o1 has some strategic value for high-level collaboration and coding, but is often too slow, too opaque, and less useful than Claude for many tasks. AI market structure and the concentration of frontier labs (Priority: 5/5): They debated whether the industry is converging toward a small set of live players. Zvi argued that capital, data, and compute constraints mean only a handful of companies will remain relevant, with Meta and xAI as special cases. Regulation after SB 1047 and the danger of bad state bills (Priority: 5/5): A major thread concerned AI governance. Zvi said the veto of SB 1047 removed the best available model for regulation and opened the door for inferior use-case-targeted laws, especially in states like Texas. Trump, national security, and the AI 'Manhattan Project' (Priority: 4/5): The conversation explored how Trump’s return might affect AI. Zvi thought Trump could either preserve useful safety structures or dangerously frame AI safety as 'woke,' but said a coordinated national project could be preferable to an uncontrolled race if AI progress is inevitable. Claude computer use, Gemini, and the value of iterative deployment (Priority: 4/5): They discussed computer-use agents and voice/podcast products. Zvi viewed iterative deployment as beneficial but warned that weak demos can still accelerate the race and that many consumer AI products remain poorly integrated or overly generic. Virtue ethics, political reality, and strategic alignment (Priority: 4/5): The interview closed on how an AI-focused person should behave morally under changing politics. Zvi emphasized practical consequences over slogans, urged flexibility, and said the right strategy depends on which levers still exist.

Key Arguments: Scaling is not 'over,' but the marginal returns of brute-force scaling are declining, pushing labs toward inference-time compute and other methods. o1 is not broadly better than Claude in everyday use, but it can feel like a higher-level collaborator on complex coding and strategic tasks. The industry is likely to consolidate around a small number of frontier players because compute, capital, and data barriers are too high for many entrants. SB 1047 was important because it could have served as a sensible template; its veto likely increased the odds of worse, more onerous state-level AI regulation. The AI race may be better managed through a unified national project than through multiple companies racing blindly, but only if safety and diplomacy remain part of the plan. Trump could be highly variable: he might strip out ideological baggage and preserve useful AI policy, or he could treat AI safety as partisan 'woke' nonsense and harm the field. Broad AGI/weak-AGI timelines are not certain, but rapid progress is plausible enough that policy and strategic preparation should be taken seriously now. Consumer AI products often fail because they are not integrated into real workflows; usefulness depends less on raw benchmark gains than on product fit and steering. Iterative deployment is better than no deployment, but releasing weak AI tools can still increase competition and speed up the broader race. AI safety advocates need to think in terms of practical governance, alliances, and leverage rather than assuming moral language alone will persuade policymakers.

Data Points: OpenAI o1 MMLU score: ~92% - Zvi cited o1 as the first model he discussed that seems to exceed a human-expert benchmark on MMLU. Prior frontier model MMLU performance: ~88–89% - He contrasted o1 with earlier models that were close to but below the human-expert threshold. H100 deployment scale: 100,000+ H100s - Referenced in discussion of xAI and Meta’s large-scale training plans. Training data volume: 15 trillion tokens - Used as an illustration of why scaling is reaching diminishing returns. SB 1047 future impact window: By February 1 - Zvi said the effects of the veto and the Trump transition would become clearer by then. Weak AGI forecast median: 2027 - Metaculus-style weak AGI timeline discussed near the end of the interview. Weak AGI modal forecast: Late 2026 - Referenced as the top of the probability curve for weak AGI. ARC AGI prize threshold: 85% - They noted that the million-dollar prize was still unclaimed because top systems were around 60%. ARC AGI state of the art: ~60% - Used to describe recent progress in the benchmark. Claude computer use improvement: ~5% to 50% - Zvi cited a rapid benchmark jump in a year as evidence of fast progress. OpenAI funding round tied to conversion: $6 billion - Noted as funding conditioned on converting into a for-profit structure. OpenAI valuation: $150 billion - Used to frame the small relative size of the funding round. UBI experiment duration: 2 years - The Altman-backed UBI experiment was discussed as a two-year trial.

Pivotal Quotes: "This is no longer the age of scaling." — Zvi Moschowitz: He framed the current AI era as one where raw scale matters less than finding the right next technique. "If China wants to knock TSMC out of existence, it can." — Zvi Moschowitz: Used in a discussion of chip dependence, Taiwan risk, and national-security strategy. "Democracy is a semantic stop sign." — Zvi Moschowitz: He criticized vague appeals to 'democratic values' as substitutes for concrete policy design.

Implications: Frontier AI is likely to remain concentrated in a few hands, making regulation, product integration, and national-security policy decisive. Expect faster capability gains, worse race dynamics, and intense political fights over how to control or steer deployment.