Science Friday
Science Friday

Move over, vibe-coding. Vibe-proving is here for math

A few years ago, ChatGPT couldn’t do simple arithmetic. Now, some experts say that AI could make mathematicians obsolete.

Topics Discussed

Episode Summary

Executive Summary: The conversation examines whether AI is truly transforming mathematics or simply becoming a stronger assistant. The guests agree models are improving quickly—from arithmetic to contest and some research-level proofs—but they say mathematics still depends on human judgment, problem selection, explanation, and trust. They also warn about AI-generated “slop” and emphasize formal proof assistants as a promising way to verify machine-generated work.

Main Topics: AI’s rapid progress in mathematical ability (Priority: 5/5): The guests trace AI’s evolution from failing basic arithmetic to handling contest-style proofs and some research-level problems, describing it as a major but still incomplete shift. Is AI revolutionizing mathematics? (Priority: 5/5): Both mathematicians reject the idea that math has already been revolutionized, though they see clear signs that AI will significantly affect how research is done. Human understanding vs. AI “oracles” (Priority: 5/5): The discussion focuses on whether AI could produce correct but unintelligible proofs, and whether mathematicians would accept truths they cannot explain. Formal proof assistants and verification (Priority: 5/5): The guests argue that formal proof systems can help verify AI-generated mathematics and reduce the risk of hallucinated or sloppy proofs. Vibe proving and AI slop (Priority: 4/5): They compare casual AI-assisted proving to vibe coding, noting that novices may overestimate results and that mathematical preprint servers are already seeing nonsense generated by models. How AI may change mathematician skills (Priority: 4/5): AI may automate literature review and routine learning, but it could also shift the profession toward higher-level judgment, creativity, and interpretation. What makes a great mathematician (Priority: 4/5): The guests emphasize that great math depends on time, dedication, curiosity, philosophical insight, and collaborative creativity—not just technical problem solving.

Key Arguments: AI has moved from poor performance on basic math to success on contest-level proofs and some research-level problems. The field has not yet been revolutionized, because AI still lacks the ability to choose interesting problems and develop mathematical direction. Mathematics is not only about solving problems; it is also about formulating the right questions and inventing new structures. If AI produces proofs that humans cannot understand, mathematicians may still insist on a human-understandable or formalized proof rather than accepting an opaque oracle. Formal proof assistants are a promising middle ground because they can verify line-by-line logic and reduce dependence on natural-language trust. AI-generated proofs in natural language can produce misleading or nonsensical output, so human oversight remains essential. AI tools are already useful for learning, literature navigation, and assisting research workflows, even if they are not yet replacing mathematicians. The most influential mathematical work often resembles philosophy and creativity as much as formal computation. Great mathematicians are shaped by long-term dedication, intuition developed through failure, and collaboration with other people. AI may automate some existing skills while opening new possibilities, but it is too early to know exactly which skills will matter most.

Data Points: International Mathematical Olympiad medals: gold medals - Google and OpenAI models reportedly earned gold medals at the IMO last year. Hodge conjecture papers on arXiv: about 12 total - Daniel Litt said he counted papers with “Hodge conjecture” in the title or abstract. Likely AI-generated Hodge conjecture papers: 11 of 12 - He estimated 11 were nonsense generated by AI tools. Erdős problems solved autonomously by AI: about 3 - Litt said he is aware of roughly three Erdős problems solved fully autonomously. Erdős problems solved with human + AI help: about 6 or 7 - He also cited another six or seven solved with AI assistance and no prior literature solution. Emily Real’s age: 41 - She mentioned this while discussing that mathematicians can improve substantially over time. Fields Medal eligibility: no longer eligible at 41 - Real noted that her age puts her beyond the age cutoff for the Fields Medal.

Pivotal Quotes: "“The specific claim that AI tools are revolutionizing mathematics definitely has not come true yet.”" — Daniel Litt: He answered the question of whether AI has already transformed the field. "“What makes a great mathematician is time and dedication to mathematics.”" — Emily Real: She explained how expertise develops beyond early technical talent. "“The most important parts of mathematics are actually closer to philosophy, I think, than science.”" — Daniel Litt: He described the deeper, conceptual side of mathematical work.

Implications: AI will likely become a powerful mathematical assistant, but not a replacement for human mathematicians. The future likely centers on formal verification, better collaboration, and higher standards for trust and explanation.

🔓 Sign Up for Unlimited Episode Search

About Science Friday

View all episodes from Science Friday