🔬“We have foundation models for language, not for physics” — Anima Anandkumar, Bren Professor of Computing

Episode Summary

Executive Summary: Anima Anankumar explains how neural operators let AI model physical systems at scale, with weather as the headline example: data-driven models can match traditional forecasting accuracy while running vastly faster. The conversation covers TorchLean for formal verification, the limits of physics-informed neural nets, why spherical geometry matters, and how these methods extend to climate, fusion, and inverse design.

Main Topics: AI for science and physical-world modeling: Anankumar frames her work as building AI systems that understand and reliably operate in the physical world, not just generate text or images. TorchLean and formal verification: She discusses TorchLean as a PyTorch-like framework in Lean for formally verifying neural networks, robustness bounds, and properties relevant to control and safety-critical systems. Why neural operators outperform standard neural nets for PDEs: Neural operators learn mappings between function spaces, enabling variable-resolution inputs/outputs and better handling of multiscale, non-local physical phenomena than fixed-grid models. Weather forecasting breakthrough: The team trained on open weather data and showed neural-operator models could be nearly as accurate as traditional methods but tens of thousands of times faster, changing the field’s assumptions. From weather to climate via geometry and ensembles: ForecastNet evolved from short-term weather to longer-term climate use by incorporating spherical Earth geometry and ensemble calibration for probabilistic forecasting. Fusion, materials, and inverse design: The same modeling approach is applied to plasma simulation, carbon sequestration, aerodynamics, photonics, and other inverse design problems where fast simulation and optimization matter.

Key Arguments: AI for science needs more than idea generation; it must test, verify, simulate, and control systems in the real world. TorchLean helps bring formal verification into neural-network workflows, enabling certified robustness and bounds useful for safety-critical deployment. Physics-informed neural nets alone often fail because solving PDEs from scratch is an unstable optimization problem, especially for time-dependent and chaotic systems. Neural operators work better because they learn from data across many instances, then generalize to new resolutions and scenarios while still allowing physical constraints. The physical world is inherently multiscale and non-local, so fixed-resolution models are a poor fit; function-space models are a better inductive bias. Weather modeling showed that open data plus the right architecture can overturn skepticism: AI can be accurate and dramatically faster than supercomputer-based simulations. Adding spherical geometry is critical for long rollouts and climate modeling because the Earth is not a rectangle; geometry-aware models are more stable. Ensembles are essential for extreme weather and climate because probabilistic prediction provides risk estimates rather than a single deterministic trajectory. The same framework can accelerate many domains where data is scarce and simulations are expensive, including fusion, carbon storage, and photonics. Classical mathematical ideas still matter, but as enablers for principled deep learning rather than as replacements for modern feature learning and scale. AI should not be regulated as if all use cases are language models; AI for science has distinct benefits and deserves dedicated policy treatment.

Data Points: Years of AI experience: More than two decades - Anankumar describes her background in AI research spanning pre-deep-learning probabilistic models to modern deep learning. Weather training samples: About 50,000 samples - She cites the size of the global weather dataset used for the early neural-operator weather model. Speedup vs traditional weather models: Tens of thousands of times faster - She says the neural-operator weather model matched traditional forecasting accuracy closely while being vastly faster. Compute requirement: Consumer-grade GPU - The weather model ran on a small model rather than a big supercomputer. ForecastNet 3 launch: Fall 2023 - She says ECMWF made the AI weather model publicly available around this time. Weather prediction timestep: Every 6 hours - The autoregressive forecasting setup predicts the next six-hour step repeatedly. Data resolution: 0.25 degrees - She notes the weather data available for training is approximately quarter-degree global resolution. Long-context scale estimate: Hundreds of billions to a trillion context length - She uses this estimate to argue transformers are impractical for high-resolution physical simulations. Fusion simulation speedup: A million times faster - She cites plasma modeling for tokamak disruption prediction as an example of dramatic acceleration over traditional simulation. Climate/weather rollout horizon: Several months to years - She describes extending weather models to climate-scale probabilistic rollouts.

Pivotal Quotes: "It's not only accurate, it's almost as close to what the traditional weather models can do accurately, but also tens of thousands of times faster." — Anima Anankumar: Summarizing the breakthrough result that made AI weather forecasting credible. "Ideas are not enough, right? So you can have a lot of ideas. The bottleneck is going testing and verifying that they work in the real world." — Anima Anankumar: Explaining why AI for science needs verification and formal methods, not just language-model-style brainstorming. "The world is latent." — Anima Anankumar: Her concise framing of physical reality as something that must be learned through latent representations and function-space models.

Implications: The discussion suggests a shift from narrow, data-hungry AI toward principled, geometry-aware models for science. For listeners and industry, neural operators could enable faster forecasting, safer control, and cheaper simulation/design across many physical domains.