Where Is All the A.I.-Driven Scientific Progress?

Episode Summary

Executive Summary: This Hard Fork episode examines the real-world state of AI for science with Sam Rodriguez of Future House/Edison Scientific. The discussion separates hype from current capability: AI can already accelerate literature review, code, and hypothesis generation, and Cosmos can sometimes replicate months of scientific work in a single run. But Rodriguez argues clinical trials, manufacturing, patient recruitment, and validation remain the true bottlenecks, making near-term “cure all diseases” claims unrealistic.

Main Topics: AI scientist Cosmos and the promise of automated discovery (Priority: 5/5): Rodriguez explains Cosmos, an AI system that takes a research objective, runs for hours, and returns scientific insights. He frames it as an early “AI scientist” that can contribute meaningfully to research, sometimes finding new results rather than merely summarizing existing work. What AI can and cannot speed up in science (Priority: 5/5): The conversation distinguishes between discovery work AI can accelerate now—analysis, literature search, hypothesis generation, and some reasoning over data—and the slower parts of science that still require humans, including experimental design, wet-lab validation, and clinical trials. The limits of hype around curing disease (Priority: 5/5): The hosts press Rodriguez on claims from AI leaders that AI will cure major diseases within a decade. He pushes back, arguing that clinical timelines, biological complexity, and incomplete knowledge make that timeline implausible, though major progress over 30 years is plausible. AI for science vs. models of the natural world (Priority: 4/5): Rodriguez draws a distinction between AI that models the process of doing science and AI that models nature itself, such as protein structure, antibody design, or organism generation. He sees generative models as one of the most important current breakthroughs. Benchmarks, validation, and reliability (Priority: 4/5): The episode addresses concerns about AI mistakes and the need for validation. Rodriguez emphasizes that outputs must be checked like any scientific result, and that the best standard is performance comparable to a competent human, not perfection. What is overhyped and underhyped in AI science (Priority: 3/5): In a lightning round, Rodriguez characterizes vibe-proven math, quantum computing, and brain-computer interfaces as overhyped; robotics for lab automation as appropriately hyped; and AlphaFold-style protein models as underhyped.

Key Arguments: AI is already useful for steps of science where there is enough existing data to extract insights, especially analysis, literature review, and hypothesis generation. Cosmos’ reported ability to reproduce work that took humans 3-6 months in a single run suggests AI can meaningfully compress research cycles. The real bottleneck in medicine is not only discovering candidates but running expensive, slow clinical trials, making “cure all diseases in 10 years” unrealistic. Even if AI improves discovery, humans still must validate findings experimentally before publishing or pursuing drug development. Generative models that create proteins, antibodies, or organisms from scratch are among the most consequential current AI-for-science advances. Scientists are conservative adopters, so widespread transformation will happen gradually, first in coding and literature search, then in deeper research workflows. Some hype is justified because benchmarks are improving, but claims should be judged against the constraints of biology, manufacturing, and regulatory validation.

Data Points: Cosmos pricing: $200 per prompt - Rodriguez says this is a promotional price for the AI scientist system. Cosmos runtime: About 12 hours per run - He describes Cosmos as not being a chatbot, but a system that works for hours before returning findings. Code generation: 42,000 lines of code - Rodriguez says an individual Cosmos run can write this much code on average. Literature reviewed: 1,500 research papers on average - He uses this to illustrate the compute and scale involved in each Cosmos run. Accuracy claim: About 80% right - Rodriguez says Cosmos returns deep insights that are “sometimes wrong,” but around 80% of the time correct. Scientific replication test: 3 months to 6 months of human work - The team estimated that Cosmos reproduced findings that had originally taken researchers this long to discover. Research conclusions: 7 total conclusions; 4 net new, 3 replications - Rodriguez says the Cosmos paper included both replications and new contributions. Disease discovery timeline: 30 years - He says a huge leap in AI-enabled science is plausible over this horizon, but not necessarily in 10 years. Near-term cure claim: 10 years is “crazy” - Rodriguez rejects claims that AI will cure all diseases within a decade.

Pivotal Quotes: "“The thing that happened with Cosmos that is pretty cool is Cosmos is like the first thing that I think we’ve made that actually really feels like an AI scientist.”" — Sam Rodriguez: Describing his company’s AI system and why it stands out from ordinary chatbots or copilots. "“A decade is crazy.”" — Sam Rodriguez: Rejecting the idea that AI will cure all diseases within 10 years because clinical trials and biological validation take too long. "“What AI will allow us to do is it will allow us to discover a lot of things where we already have the information to discover it, we just haven’t figured that out yet.”" — Sam Rodriguez: Summarizing his view of AI’s immediate value in science.

Implications: AI is likely to reshape scientific workflows first by speeding analysis, search, and hypothesis generation, not by instantly solving biology. The biggest gains will come when AI helps scientists run better experiments and prioritize stronger leads, while validation and trials still set the pace.