Episode Summary
Executive Summary: Anima Anandkumar explains how AI for science is moving from structure prediction to function discovery, using generative models, diffusion, and neural operators to tackle biology, weather, climate, and materials science. She argues that scientific AI must handle sparse data, extrapolation, uncertainty, and multiple resolutions, and highlights major speedups and open-sourced tools that could transform simulation and discovery.
Main Topics: From early ML theory to generative AI: Anima traces her career from statistics, signal processing, unsupervised learning, and tensor methods to today’s generative AI, framing diffusion and neural networks as a continuation of a long move beyond linear models. Why AI for science is different: Scientific problems often lack large labeled datasets and require extrapolation beyond training data, unlike image/text tasks. She emphasizes zero-shot generalization, novelty, and discovery as central challenges. Protein folding, binding, and biological function: The conversation moves from protein structure prediction to protein-ligand interactions and then to function, with the argument that understanding DNA/genome-level patterns is necessary to generate functionally meaningful proteins. Genome language models for biology: Anima describes a 25B-parameter biological language model trained on over 110 million genome sequences to learn the 'language of the genome,' generate new sequences, and predict held-out coronavirus variants. Diffusion models beyond images: She compares stable diffusion’s image generation to scientific generation tasks, arguing diffusion can model complex probability distributions for genome sequences and other scientific phenomena, though sampling can be slow. Neural operators for PDEs and simulation: A major theme is neural operators as resolution-agnostic function-space models that learn mappings across grids and can approximate operators in physics, enabling fast simulation of fluids, weather, carbon capture, and materials. Foundation models for decision making: The discussion ends with MindDojo, a Minecraft benchmark designed to push foundation models toward open-ended decision-making, combining text, video, and wiki data for thousands of tasks.
Key Arguments: AI for science needs models that generalize beyond training data because scientific discovery is fundamentally about extrapolation and novelty. Generative AI makes scientific generation more feasible, e.g. designing new molecules, genes, and proteins rather than only recognizing known patterns. Protein folding is important, but structure alone is insufficient; function emerges from broader biological context, especially genome-level relationships. Genome-scale language models can capture latent relationships among genes and help generate biologically meaningful new sequences. Diffusion is not just for images; it is a general generative framework that can learn complex distributions in scientific domains. Neural operators are better suited than standard neural networks for PDE-driven science because they can operate across resolutions and preserve fine-scale information. Hybrid AI-HPC approaches should integrate numerical methods and machine learning at the algorithmic level rather than simply bolting AI onto traditional solvers. AI can dramatically accelerate simulation while preserving useful accuracy, making large-scale science problems more tractable. Uncertainty quantification is essential in applied domains like weather forecasting, where practical decisions depend on probabilities, not just point predictions. Foundation models will likely become the starting point for future reinforcement learning and open-ended decision systems in science and robotics-like environments. There is a need for broad, open scientific benchmarks and data-sharing ecosystems to let the community build toward general-purpose scientific models.
Data Points: AI for Science initiative founded: 2018 - Caltech launched a campus-wide AI for Science initiative before the area became mainstream. Genome language model size: ~25 billion parameters - Anima described the largest biological language model her group built. Training corpus size: 110+ million genome sequences - The genome model was trained on long sequences from bacteria and viruses. Weather speedup: ~45,000x faster - Neural-operator-based weather prediction matched current numerical weather models for up to two weeks while running much faster. Weather inference time: ~0.25 seconds on a single GPU - Reported runtime for the neural weather model versus hours on CPU clusters for traditional models. Carbon capture speedup: up to ~700,000x faster - Neural operators modeled underground CO2-water interactions far faster than numerical solvers. Materials/deformation speedup: hundreds of thousands of times faster - Reported for modeling deformation/plasticity in materials. Weather data resolution used in training: 25 kilometers - Historical weather model training used global data at 25 km spatial resolution. Desired climate-model resolution: ~1 meter spatial resolution and 1 second temporal resolution - Used to illustrate the enormous compute gap for high-fidelity climate simulation. Additional compute requirement: ~100 billion times more computing - A colleague’s estimate for resolving key climate turbulence phenomena at the needed resolution. Ensemble members in traditional weather models: ~50 - Used to contrast with the ability of AI models to run thousands of ensembles cheaply. Benchmark scale for MindDojo: thousands of Minecraft tasks - The decision-making benchmark is built around open-ended task solving in Minecraft.
Pivotal Quotes: "We can now learn the probability distribution of how do we create new genome sequences." — Anima Anandkumar: Explaining how diffusion models extend from image generation to genome design. "We need to understand the language of the genome and not only learn those relationships between genes because they map to proteins, but also potentially generate new gene sequences." — Anima Anandkumar: Describing the shift from protein structure to genome-level function and generation. "I don't want to get into the debate of what was the numerical method, what was AI. That I org-like baby, you shouldn't be able to tell them apart." — Anima Anandkumar: Arguing that future HPC and AI systems should be seamlessly integrated rather than treated as separate approaches.
Implications: AI is becoming a core scientific instrument, not just a digital product technology. Expect faster simulation, better uncertainty estimates, and new discovery pipelines in biology, climate, and engineering—especially where physics-informed, resolution-flexible models replace brute-force computation.