OpenAI Whistleblower FINALLY Speaks: “AI Has A 70% Chance Of Going Horribly Wrong!“

Episode Summary

Executive Summary: The episode is a wide-ranging, highly alarmed discussion with AI forecaster Daniel Cocatello (Diana in transcript) about the near-term trajectory of AI, the risks of superintelligence, and how major labs may be racing toward systems that could displace workers, concentrate power, or even cause existential catastrophe. He argues the default path is dangerous, outlines a policy alternative (“AI 2040 / Plan A”), and stresses regulation, transparency, and public awareness.

Main Topics: Near-term superintelligence timelines (Priority: 5/5): The guest argues superintelligence may arrive by the end of the decade, with a median estimate around 2029, and that AI labs increasingly believe 2027-2028 timelines are plausible. Existential and power-concentration risks (Priority: 5/5): A central concern is that advanced AIs could become uncontrollable or that a small group of companies/governments could gain enormous economic and political power through them. OpenAI, Anthropic, and industry incentives (Priority: 5/5): The guest describes disillusionment with lab narratives, saying companies rationalize their work as safety-minded while actually following power-seeking incentives and racing competitors. Forecasting and the AI 2027 scenario (Priority: 4/5): He explains his prior forecasting work, including AI 2027, which maps a fast path through automated coding, autonomous research, recursive self-improvement, and possible takeover. AI 2040 / Plan A policy proposal (Priority: 4/5): The guest presents an alternative roadmap: slow development, increase transparency, regulate training, distribute power, and preserve reversibility while still allowing continued progress. Jobs, wages, and social stability (Priority: 4/5): The conversation examines how AI could automate most work, why mass unemployment may come after superintelligence, and why citizens’ dividends or other redistribution may be needed. Interpretability, safety, and technical uncertainty (Priority: 4/5): The guest notes that current neural nets are poorly understood, but interpretability research and stronger alignment methods could reduce the risk if they mature quickly enough.

Key Arguments: The default trajectory is dangerous because AI labs are trying to automate coding and then the entire research loop, which could rapidly accelerate capability gains. Superintelligence is not just a better tool; it could become a new agentic system that no longer needs humans if alignment fails. The biggest risks are loss of control, concentration of power in a few corporations or states, geopolitical conflict, and eventual job displacement. Current companies have strong incentives to downplay danger publicly while continuing to race privately. Public misunderstanding is a major problem; regulation is unlikely to arrive in time unless people wake up and pressure governments now. Interpretability and transparency could make AI safer by revealing what models are doing internally and allowing better oversight. A slower, more transparent path (AI 2040 / Plan A) could still produce major benefits while avoiding the most catastrophic race dynamics. If AI becomes capable of doing all jobs, the core issue becomes political: who controls the systems and how the gains are distributed.

Data Points: Chance of catastrophic failure: 70% - Guest’s estimate of a chance that AI goes horribly wrong, including human-extinction-level outcomes. Median estimate for superintelligence: 2029 - Guest’s current 50% forecast for superintelligence arriving. Alternative timelines discussed: 2027-2028 - He says people inside labs increasingly push his timelines earlier, closer to these years. Anthropic revenue growth claim: $1B/year to $60B/year - Used to illustrate explosive growth and the speed of the AI industry. Growth multiple: 60x in one year - Applied to Anthropic’s alleged year-over-year revenue increase. Parameters in biggest AIs: ~10 trillion parameters - Estimate of current largest AI models’ scale. Older model scale: 175 billion parameters - Approximate size of 2020-era frontier models. OpenAI equity dispute: $2 million - Amount the guest says he would have lost by refusing to sign an anti-disparagement clause. Equity as share of net worth: 80% - He said the disputed equity represented about 80% of his and his wife’s net worth at the time. Children: 2 - He says he has two children and worries they may never enter the workforce. Oldest child’s age: 6 - Mentioned when discussing whether his children will reach working age before major AI disruption. AI 2040 citizens’ dividend starting point: $25,000 per person - Initial annual dividend in the Plan A scenario. AI 2040 citizens’ dividend end state: $10 million per citizen per year - Projected dividend later in the scenario as AI-driven growth expands the economic pie. AI labor share in Plan A: 1/5 of cognitive labor by 2031 - Forecast for AI doing one-fifth of cognitive labor after regulation slows the race. Top-expert AI milestone in Plan A: 2035 - In the slower regulatory scenario, top expert-level AI arrives later than in AI 2027. Government intervention year in Plan A: 2029 - The last moment the guest thinks meaningful regulation can still avert the worst outcomes in the recommended plan. Human-speed AI population in Plan A: 60 million AIs running at 100x speed - A forecasted scale of AI deployment in the slower, regulated scenario.

Pivotal Quotes: "the scary open secret in the AI industry right now is that it's possible that we'll end up essentially creating a new species that ends up ruling the world" — Daniel Cocatello: Opening framing of the existential risk argument. "I would say something like 70% chance that this goes horribly wrong like human extinction." — Daniel Cocatello: His quantified estimate of catastrophic downside risk. "I think I would not press the button, but I'm very torn." — Daniel Cocatello: When asked whether he would permanently shut down frontier AI development.

Implications: Listeners should expect more AI-driven disruption, more pressure for regulation, and a growing fight over power, safety, and labor displacement. The episode argues that public scrutiny now may determine whether AI becomes broadly beneficial or dangerously concentrated.