Episode Summary
Executive Summary: Carissa Véliz argues that prediction, especially in AI, is not neutral description but a powerful social act that shapes behavior, incentives, and power. Drawing on ancient oracles, Popper, Arendt, and modern machine learning, she warns that predictive systems can erode autonomy, democracy, and justice when treated as facts rather than contested claims. She urges epistemic humility, curiosity, and active resistance to harmful futures.
Main Topics: Prediction as a social act, not a fact (Priority: 5/5): Véliz argues that predictions are not factual descriptions of the future but speech acts that can influence behavior, set norms, and push the world toward a particular outcome. AI as a prediction machine (Priority: 5/5): Machine learning is framed as projecting from existing data to unseen data, making AI powerful but also limited, value-laden, and often overtrusted in consequential settings. Ancient divination and modern power (Priority: 4/5): The Oracle of Delphi and Roman divination are used to show that prediction has always been entangled with commerce, politics, and manipulation, not just truth-seeking. Democracy, agency, and predictive control (Priority: 5/5): The conversation stresses that predictive systems used by companies and governments can narrow human autonomy by steering decisions in hiring, housing, justice, and governance. Critique of long-termist and utilitarian reasoning (Priority: 4/5): Véliz questions overconfident cost-benefit calculations about distant futures, arguing that uncertainty, blind spots, and justice concerns make purely quantitative ethics unreliable. Constructive responses: humility, curiosity, and resistance (Priority: 4/5): Rather than rejecting prediction entirely, she advocates epistemic virtues, experimental skepticism, and active efforts to build futures aligned with democratic values.
Key Arguments: Predictions are not facts; they are assertions about the future that often function as commands or incentives. AI/ML systems are prediction engines that extend patterns from past data, so their outputs inherit the biases and limits of that data. News and public discourse often treat predictions as if they were objective facts, obscuring the interests of the predictor. Historical divination shows that prediction has always been a business and a tool of power, not a purely spiritual or neutral practice. People should ask who is making a prediction, what data they used, what incentives they have, and who benefits if the prediction becomes accepted. The future is unwritten, so predictions should be read partly as evidence of present ambitions and power structures rather than as destiny. Epistemic humility is essential because distant-future forecasts are highly unreliable and often based on many questionable assumptions. Effective altruism and existential-risk thinking can become overconfident when they extend prediction too far into the future. Justice-oriented action can be worthwhile even when statistically unlikely to succeed, because principles and democratic progress matter beyond expected value calculations. Large language models are best understood as systems optimized for plausibility and engagement rather than truth, making them a form of 'bullshit' in the Frankfurt sense. Using predictions uncritically can legitimize surveillance, discrimination, and reduced human agency across many parts of life. A healthier response is not fatalism but active participation in shaping desired futures through curiosity, playfulness, and engagement with diverse perspectives.
Data Points: Effective altruism time horizon: thousands of years - Described as the extended future some longtermist arguments focus on, which Véliz criticizes as overly speculative. Current human population scale: billions - Used in contrast to imagined future trillions in longtermist calculations about moral priority. Existential risk estimate (Jeff Hinton): 20–30% - Cited as one estimate for AI existential risk, illustrating uncertainty and disagreement. Existential risk estimate (super forecasters): less than 1% - Contrasted with Hinton’s estimate to show how forecasting varies and reveals ignorance. Randomized controlled trial chance threshold: about 20% - Mentioned in discussion of how some RCT outcomes may arise from chance alone. Probability reduction example: a billionth of a percentile - Used to illustrate the extreme precision and speculative nature of some longtermist AI-risk arguments.
Pivotal Quotes: "Predictions are not facts. Facts belong to the present and the past. An assertion about the future can be many things, but never a fact." — Carissa Véliz: Central thesis from the book, discussed early in the interview. "Predictions are commands disguised as descriptions." — Carissa Véliz: Used to explain how forecasts can steer behavior and concentrate power. "They represent the force of bullshit." — Carissa Véliz: Her characterization of large language models in the Frankfurtian sense of persuasion without truth-tracking.
Implications: Listeners are urged to treat forecasts as power-laden claims, not destiny. For AI, policy, and everyday decisions, the takeaway is to question incentives, defend agency, and favor democratic, justice-oriented action over passive acceptance of predictive narratives.