Episode Summary
Executive Summary: Ajaya Kotra discussed how Open Philanthropy allocates billions across competing worldviews, why long-termist arguments matter but shouldn’t dominate uncritically, and how fairness, tractability, and declining returns shape grantmaking. She also walked through her work forecasting transformative AI using biological anchors, concluding AI could be transformative within decades, and reflected on the emotional and methodological challenges of doing high-stakes, open-ended research.
Main Topics: Open Philanthropy’s worldview diversification problem: Ajaya explains how OpenPhil divides funding among near-term human welfare, animal welfare, and long-termist risk reduction, while also considering smaller candidate worldviews like science, growth, civic institutions, and democracy. Fairness agreements vs. credence-based allocation: She describes competing methods for splitting money across worldviews: allocate by moral credence, by the worldview with the most at stake, or by fairness-style bargains that mimic what worldviews would have agreed to under uncertainty. The case for and limits of long-termism: The conversation examines astronomical waste, existential risk reduction, doomsday/simulation arguments, and why Ajaya is sympathetic to long-termism but wary of pushing philosophy to overly weird or fanatical conclusions. Biological anchors and AI timelines: Ajaya details her forecasting model for transformative AI, combining estimates of training compute needs, algorithmic progress, hardware cost declines, and rising spending. She lands on roughly 12–15% by 2036 and a median around 2055. Space colonization and the far future: She argues that biological interstellar colonization is much harder to defend than computer-based or uploaded-human scenarios, making the far-future case hinge more on digital minds and software than on human bodies in space. Last-dollar thinking and spending down over time: Ajaya outlines how OpenPhil thinks about when to spend now versus later, using diminishing returns, investment growth, and shifting opportunity sets to estimate optimal pacing of giving. Research culture, stress, and writing big reports: She reflects on the difficulty of producing publishable, decision-relevant research when the most important parts are intuitive, fuzzy, and hard to formalize, and how reframing her job improved her mental health.
Key Arguments: OpenPhil should not allocate solely by formal moral theory; it also needs to consider comparative advantage, tractability, feedback loops, and how weird or speculative a cause feels in practice. Long-termism is compelling because future potential value may be enormous, especially if space colonization and digital minds make the future vastly larger than the present. Near-termist and animal-focused work remain valuable because they are more empirically grounded, easier to measure, and less vulnerable to philosophical overreach. Fairness-style bargaining between worldviews is attractive, but hard because it depends on what prior or starting point counts as the relevant veil of ignorance. There is a strong case that biological interstellar settlement is less robust and less economically plausible than digital or uploaded-human colonization. The key uncertainty in AI timelines is not only hardware or compute, but how much compute, data, and algorithmic search are needed to find the right program. OpenPhil’s research has shifted from a few broad worldviews toward many specific tranches of giving, each evaluated on cost-effectiveness plus qualitative tags like feedback quality and self-delusion risk. The doomsday and simulation arguments are intellectually important but too slippery to drive confident practical conclusions on their own.
Data Points: OpenPhil funds/gives away: billions of dollars - Open Philanthropy is described as expecting to give away billions over its existence. EA Berkeley class grant: $5,000 - Ajaya co-ran an effective altruism class that received a GiveWell grant to award as a charity vote. OpenPhil cause prioritization update: 2017 - The 2017 cause prioritization update is cited as the source of the three major worldview buckets. Transferrable AI probability by 2036: 12–15% - Ajaya’s post-model estimate for transformative AI by 2036. Earlier OpenPhil estimate by 2036: 10% - Holden’s 2016 blog post and technical advisor discussions supported at least a 10% chance by 2036. Ajaya median estimate for transformative AI: 2050–2060 - She says her median varies but generally falls in this range, with ~2055 as a rough center. World economy growth threshold for transformative AI: 10x faster than 2020 - Ajaya operationalizes transformative AI as AI driving world economic growth to about 10 times the 2020 rate. 2020 global growth rate: 2–3% per year - Used as the baseline for comparing a 10x faster transformative growth scenario. Training compute for current top models: 1–10 million FLOPs (roughly) - She cites GPT-3 and AlphaStar as publicly inferable examples of models in this range. Brain compute estimate: 10^13 to 10^17 FLOPs/s - Joe Carlsmith’s estimate for the computational power needed to replicate brain-level tasks. Central brain-compute estimate: 10^15 FLOPs/s - The central estimate cited from the brain-computation literature. Algorithmic efficiency trend: ~13–16 months per halving (in some benchmarks) - Based on Katja Grace’s algorithmic progress paper and OpenAI’s AI efficiency work. Hardware progress assumption: ~2.5 years halving time - Ajaya’s point forecast for hardware cost declines used in the model. Hardware progress over last century: 12 orders of magnitude - She cites this as the historical scale of improvement in computing cost/performance. OpenPhil size: ~45 people - Ajaya notes OpenPhil’s growth to about 45 staff. Generalist staff at OpenPhil: ~10 people - She says there are now around ten generalists, enabling more internal collaboration. AI and compute scaling: doubling every 6 months (reported trend) - She references a recent OpenAI blog post about rapid scaling of compute used for frontier models. Long-termist last-dollar estimate: ~$200 trillion per world saved - Her biosecurity-related long-termist estimate for a highly scalable intervention. Target timeframe for patient philanthropy modeling: several decades / 50–100 years - OpenPhil models allocation over time on the scale of decades, matching donor preferences. Estimated future AI-relevant opportunity window: ~5–10 years before transformative AI - Ajaya expects a concentration of funding opportunities shortly before transformative AI arrives.
Pivotal Quotes: "I think the long-termist camp is more into philosophy than the near-termist camp." — Ajaya Kotra: She uses this to distinguish a philosophical-argument-heavy worldview from a more empirical, commonsense one. "I’m gonna get off the train before we get to the point where all we’re focusing on is existential risk because of the astronomical waste argument." — Ajaya Kotra: Her metaphor for stopping short of extreme philosophical conclusions that feel too weird or fanatical. "My job is not to get the ultimate right numbers. I think reframing that was really important for my mental health." — Ajaya Kotra: She explains how she copes with uncertainty and the pressure of high-stakes forecasting work.
Implications: For donors and researchers, the episode shows that high-impact philanthropy requires blending philosophy, empirical forecasting, and institutional pragmatism. It also suggests AI timelines and far-future risks are urgent enough to influence present-day allocation, but uncertain enough to demand humility and diversified bets.