Is innovation slowing down? With Matt Clancy

Episode Summary

Executive Summary: John Byrne Murdoch and Matt Clancy debate whether progress has become culturally less valued and whether innovation is genuinely slowing. They discuss historical “progress” narratives, evidence from text analysis and productivity data, causes like burden of knowledge, aging scientific workforces, competition, and incentives, and possible reforms to make science more experimental, risk-tolerant, and effective.

Main Topics: Culture, progress, and the Industrial Revolution (Priority: 5/5): The conversation contrasts economic explanations of the Industrial Revolution with theories that cultural beliefs in progress and human perfectibility helped catalyze sustained innovation. Text analysis as a proxy for cultural change (Priority: 5/5): They discuss a 2023 paper using large book corpora to track progress-oriented language, noting growth in scientific and political-economy writing and a possible shift away from progress language in more recent decades, while also questioning whether word frequencies truly capture culture. Whether innovation productivity is slowing (Priority: 5/5): Clancy argues that measures like productivity growth, health gains, and research output per scientist suggest diminishing returns: society is investing more in R&D, yet getting less output per unit of effort. Burden of knowledge and rising complexity (Priority: 5/5): A major explanation for slower innovation is that new discoveries require more prior knowledge, larger teams, deeper specialization, and longer training, making breakthroughs harder to achieve. Demographics, turnover, and field renewal (Priority: 4/5): The discussion explores how aging populations and lower turnover in scientific fields may reduce novelty, with new people and changing keywords historically helping fields evolve. War, geopolitics, and incentives (Priority: 4/5): The hosts consider whether wartime urgency and great-power rivalry spur innovation, but note the tradeoff: war can accelerate applied work while crowding out basic research. Reforming science and innovation policy (Priority: 5/5): Clancy argues for a more experimental, evidence-based science system that tests new funding and evaluation methods, borrowing practices from venture capital and meta-science efforts.

Key Arguments: Belief in progress may have been a real cultural force in the run-up to industrialization, not just an economic byproduct. Textual analysis can reveal broad shifts in language and ideas, but word frequencies can be misleading because they are affected by topic mix, influential books, and domain-specific usage. Productivity growth has declined since roughly the 1960s, even as R&D investment has risen substantially, implying lower output per researcher. Innovation is getting harder because of the burden of knowledge: each new advance requires more existing knowledge, larger teams, and longer training. Aging scientific communities may reduce turnover and the inflow of new ideas, although this effect is less studied than individual career productivity. Large teams may produce less disruptive ideas than small teams, suggesting communication and siloing problems in modern research. AI and large language models could help overcome knowledge burden and coordination frictions, though it is too early to know how much. Science funding and evaluation may be too consensus-driven and conservative; more high-variance, experimental selection mechanisms could improve outcomes. Basic science has increasingly moved out of corporate labs and into universities/nonprofits because firms benefit less from undirected research that can help competitors. The best path forward is to make science itself more evidence-based, by experimenting with different funding and institutional designs.

Data Points: Industrial Revolution interpretation: Robert Allen vs. Joel Mokyr - Allen is presented as the classical economics-based view; Mokyr is cited for the progress-culture theory. Corpus timeframe: 1500s–1700s and beyond - The 2023 text analysis paper studies books published across this period. Paper title: Enlightenment Ideals and Belief in Progress in the Run-Up to the Industrial Revolution: A Textual Analysis - The study discussed as evidence for changing progress language. Decline in productivity growth: Since the 1960s - Clancy says productivity growth has declined from around this period. US R&D spend: About 3% of GDP - The US has invested roughly this share consistently for the last 50–60 years. Research effort growth: Enormously higher than in the past - Effort in research has risen far faster than output such as productivity or life expectancy. Productivity per scientist: Declining - Clancy argues that output per researcher has fallen over time. US economy productivity relative to researchers: Declining since the 1930s - One long-run measure of productivity relative to research effort. Nobel Prize discovery age: Higher than before - Age at first major discovery has risen over time. First paper age: Higher across many fields - Researchers are publishing their first papers later than in the past. Research career high-impact period: First 25 years - Clancy cites evidence that high-impact work tends to be produced within roughly the first 25 years of a career. Open Philanthropy origin: Originally to give away one wealthy person’s money - Clancy explains the organization’s history before it expanded to multiple grantmaking activities.

Pivotal Quotes: "We're running faster to stay in the same place." — Matt Clancy: Describing rising research effort without corresponding gains in productivity or life expectancy. "You want science to be more experimental and evidence-based, but about itself." — Matt Clancy: Summarizing his preferred reform agenda for science funding and institutions. "The theory I was setting out was that a lot of what did determines how economies function and what they do and how they grow is how we think about or talk about innovation and progress on a societal level." — John Byrne Murdoch: Introducing the central premise that culture around progress can shape economic outcomes.

Implications: The episode suggests innovation policy should focus less on assuming automatic progress and more on fixing incentives, reducing knowledge bottlenecks, and testing new scientific institutions. AI and better meta-science may help, but cultural support for risk-taking also matters.