613: Causal Machine Learning

Episode Summary

Executive Summary: Dr. Emre Kejiman explains causal machine learning as the fusion of causal inference and ML: using domain assumptions to distinguish cause from correlation, improve robustness under distribution shift, and support decision-making. He outlines the four-step causal workflow, discusses the PyWhy/DoWhy ecosystem, applications in health, social science, agriculture, and notes that full automation of causal reasoning remains far off.

Main Topics: What causal machine learning is: Causal ML combines causal inference and machine learning: causality guides ML toward stable cause-effect patterns, while ML helps causal methods scale to high-dimensional and unstructured data. Why correlation is not enough: Conventional ML finds predictive correlations but cannot infer causal direction; causal ML adds assumptions and domain knowledge to estimate effects more reliably, especially under changing environments. Four-step causal inference workflow: The conversation details the causal workflow: modeling assumptions with graphs, identification of a valid estimation strategy, statistical estimation, and refutation/sensitivity analysis. PyWhy and DoWhy ecosystem: Emre explains the open-source PyWhy organization and DoWhy library as an end-to-end causal tooling stack focused on helping users express assumptions, perform identification, estimate effects, and validate results. Applications in social science, health, and agriculture: He describes using causal methods on social media and observational data for mental health, flu spread, behavioral dynamics, and current work in agriculture and climate-related soil carbon modeling. Limits of automation and AGI hype: He argues that current ML systems, including deep learning and copilots, still rely on statistical pattern recognition, not causal reasoning, and that automating causal assumptions is a hard, still-open problem. Research culture and hiring at Microsoft Research: Emre describes MSR as a place for researchers who want real-world impact, broad company leverage, and cutting-edge research that can translate into practical tools and publications.

Key Arguments: Causal ML is useful because it targets stable cause-effect relationships rather than fragile correlations that can break under distribution shift. Causal inference requires assumptions; there is no assumption-free way to infer causality from observational data alone. The practical causal workflow is: model assumptions, identify a valid strategy, estimate statistically, and refute/test assumptions. DoWhy was built because existing libraries focused on statistical estimation, while users struggled more with specifying assumptions and causal framing. Causal ML is especially important in decision systems, where models influence policies that then change the data distribution and can cause predictive failure. The field is still not ready for fully unstructured data and full automation; structured tabular data remains the strongest use case today. Domain knowledge is central, but reinforcement learning and experimentation may help discover causal structure in some settings. Causal methods can improve robustness, generalizability, offline learning, fairness, and reinforcement learning when integrated thoughtfully. Social media data can be used to study sensitive societal issues causally, but only when combined with theories and validation from domain experts. Microsoft Research values researchers who can turn cutting-edge ideas into world-scale impact through the broader company ecosystem.

Data Points: Episode number: 613 - Super Data Science Podcast episode featuring Dr. Emre Kejiman Years at Microsoft Research: 17+ years - Emre has worked at Microsoft Research for over 17 years PhD institution: Stanford University - Emre earned a PhD in computer science from Stanford Publications: 100+ papers - Emre’s research output mentioned in the introduction Citations: 1,000+ citations - Emre’s research citation count mentioned in the introduction Talk views: 13,000+ views - The 2021 causal inference talk with Amit Sharma has over 13,000 views World’s farmers impacted: 60%+ - Global Soil Health partnership aims to help improve soil management for over 60% of the world’s farmers Book format: 2 parts - Their upcoming causal ML book is described as having two parts: foundations and applications Podcast episode reference: Episode 607 - Prior episode with Professor Jennifer Hill on causal inference Podcast episode reference: Episode 601 - Prior episode featuring Sarah Catanzaro on venture capital and data science companies Podcast episode reference: Episode 539 - Prior episode with Serge Massis on agricultural data science and climate health Podcast episode reference: Episode 459 - Prior episode with Vince Pataccio on machine learning for climate change

Pivotal Quotes: "Causal machine learning is the intersection of causal methods with machine learning." — Dr. Emre Kejiman: Definition of causal machine learning at a high level "In order to get at cause and effect relationships with machine learning, we need to bring in assumptions, and we encode those assumptions so that we can reason over them." — Dr. Emre Kejiman: Explanation of why causality cannot be inferred from data alone "We really provide that end-to-end scaffolding for thinking about your assumptions, reasoning about them, and then after you've done your statistical estimation, validating them, refuting them, running sensitivity analyses." — Dr. Emre Kejiman: Purpose of the DoWhy library

Implications: Listeners should expect causal ML to matter most in decision-heavy domains, where shifting policies can break predictive models. The field is poised to grow, but human assumptions, validation, and domain knowledge remain essential for now.