607: Inferring Causality

Episode Summary

Executive Summary: Jennifer Hill argues that causal inference is central to science, policy, and everyday decisions, but that true causality requires careful design, transparent assumptions, and humility. She explains counterfactual thinking, why correlation is not causation, how BART and other flexible Bayesian models help with observational data, and how her ThinkCausal tool teaches and operationalizes causal analysis for non-coders.

Main Topics: What causal questions are and why they matter: Hill defines causal questions as asking how the world would differ under alternate interventions, from personal choices to public policy and scientific research. She emphasizes that these questions drive decision-making and have high stakes when applied to society. Why correlation is not causation: The conversation explores how people naturally infer causality from personal experience and correlations, but such conclusions are often invalid without a counterfactual. Hill notes that media headlines and preexisting beliefs intensify causal misinterpretation. Counterfactuals and experimental design: Hill explains counterfactuals as the missing alternate world needed to identify causal effects, and argues that randomized experiments are the cleanest way to approximate them. Observational studies require assumptions and careful design work. Machine learning and causal inference: Hill discusses how methods like matching, weighting, random forests, and BART are used for causal inference, but stresses these are still model-based tools that depend on assumptions rather than magic. BART as a preferred causal tool: BART (Bayesian Additive Regression Trees) is presented as a flexible Bayesian model that offers strong uncertainty quantification, regularization, and practical performance in causal settings, especially compared with more ad hoc approaches. ThinkCausal: interactive, no-code causal learning and analysis: Hill describes her new software platform designed to teach causal concepts in-context and let users run causal analyses via point-and-click workflows, with embedded learning modules and quizzes. Training, books, and practical skills for causal inference: Hill recommends a design-first learning path: human-subjects research methods, measurement, qualitative/community-engaged work, and then quasi-experimental and estimator-focused causal courses, alongside books by her and others.

Key Arguments: Causal questions ask what would happen under a different intervention; they are fundamentally about comparing worlds that cannot both be observed. Personal experience (n=1) is often misleading because causal inference requires a counterfactual, not just an observed outcome. Popular media and prior beliefs push people toward causal interpretations even when evidence only supports association. Randomized experiments are the most reliable way to infer causality because they balance observed and unobserved confounders on average. Most causal methods used in practice are still model-based prediction or balancing tools; their validity depends on assumptions, not on any assumption-free algorithm. BART is attractive because it is flexible, Bayesian, and provides coherent uncertainty estimates without as much ad hoc tuning or separate resampling. Machine learning can improve causal estimation, but it should not distract from the more important tasks of understanding subject matter, assumptions, and data provenance. ThinkCausal is intended to reduce the barrier to causal analysis while simultaneously teaching users the concepts and assumptions behind each decision. Learning causal inference should start with research design, ethics, measurement, and real-world context, not just with estimators or software. Communication to non-technical audiences is a core professional skill; if you cannot explain your work simply, you may not fully understand it.

Data Points: Episode number: 607 - Super Data Science Podcast episode featuring Jennifer Hill. Years cited for original book: 2006 - Data Analysis Using Regression and Multilevel Hierarchical Models was published in 2006. Citations for original textbook: 15,000+ - Hill and Gelman's textbook has been cited over 15,000 times. PhD year mentioned by host: 2007 - Host described using Hill's 2006 book during his PhD starting in 2007. Conference year first discussed BART: 2005 - Hill said she first talked about BART at the Atlantic Causal Inference Conference in 2005. Teaching award year: 2021 - Hill received the NYU Distinguished Teaching Award in 2021. Research examples of causal inference: Mass mandates, vaccines, smoking cessation, Vietnam draft lottery - Used as examples of causal and natural-experiment contexts. ThinkCausal learning modules live: About 4 - Hill said the platform currently has roughly four learning modules available. Vocabulary used in causal inference: S-demand - Explained as the specific average treatment effect of interest. Reading and teaching format: Point-and-click plus quizzes - ThinkCausal includes embedded learning modules with quizzes to check understanding. Network/website mention: superdatascience.com/607 - Show notes and materials for the episode are hosted at this URL.

Pivotal Quotes: "In causal inference, it's worse than that, right? Because you could have a huge sample size, like the whole world during COVID... but if you don't have the counterfactual world where COVID didn't happen, you're still sunk." — Jennifer Hill: Explaining why even very large observational datasets cannot identify causality without a counterfactual. "The dirty little secret is the causal tools are just prediction tools, or they're kind of post hoc design tools like matching and weighting." — Jennifer Hill: Describing the nature of practical causal methods and warning against magical thinking. "Communication does not mean communicating to technical people. Communicating means can you explain this to your grandmother? If you can't communicate what you're doing in a way that non-technical people can understand, you don't actually understand what you're doing." — Jennifer Hill: Discussing the skills needed for data scientists and researchers to be effective.

Implications: Listeners should treat causal claims skeptically, prioritize design and assumptions, and use tools like BART or ThinkCausal as aids—not substitutes—for rigorous thinking. The field is moving toward more accessible causal workflows, but human judgment and clear communication remain essential.