Episode Summary
Executive Summary: Robert Oszuanes surveyed the state of causal machine learning, highlighting 2022 progress in causal discovery, causal inductive bias, causal representation learning, and actual causality. He emphasized a shift from pure graph recovery toward integrating causal reasoning with downstream ML tasks, simulators, and foundation models, and argued that LLMs can both aid causal workflows and be improved by formal causal methods.
Main Topics: State of causality research and recent momentum: The conversation frames causality as a field that surged after Bengio's system 1/system 2 talk and the pandemic, with continued momentum in ML conferences and practical applications. Deep learning for causal discovery: Discusses papers that use deep learning, continuous optimization, and transformer architectures to infer causal graphs, including settings with unknown interventions and downstream-task-driven objectives. Causal inductive bias from simulators and process models: Explores using simulators, physical process models, and generated data to inject causal knowledge into learning systems, especially in climate, biology, and neural network training. Causal representation learning: Covers learning latent causal variables from high-dimensional data, formal desiderata for good representations, and work using probabilities of causation and latent hierarchical structure. Actual causality and human causal judgment: Contrasts formal causal inference with how humans assign blame/responsibility using counterfactual simulation, normality, and probable causal narratives. LLMs, causal reasoning, and product workflows: Argues that LLMs can accelerate causal workflows by generating graphs and code, while causal tools can validate, constrain, and improve LLM outputs and reasoning. Tools, benchmarks, and education: Highlights PyWhy, WhyNot, causal-learn, CausalMBRL, and upcoming courses/books that aim to make causal ML more accessible.
Key Arguments: Causal discovery is moving from learning standalone graphs to optimizing graphs for downstream machine learning tasks like adaptation and intervention analysis. Unknown interventions can still be used as learning signals if the model predicts which nodes were targeted and how downstream variables respond. Simulators are valuable because they encode causal mechanisms directly and can be used to generate training data that spans the needed causal manifold. Causal representation learning is essentially causal discovery in high-dimensional or latent-variable settings, and it needs formal desiderata to be solvable and useful. Human causal judgment depends on counterfactual simulation, normality, and selecting the most probable causal narrative, which traditional causal inference often misses. LLMs already capture a lot of common-sense causal knowledge and can generate plausible causal DAGs, code, and workflows, but need formal validation for reliability. Causal tools can serve as an ombudsman or validator for LLM outputs, while LLMs can automate graphing, coding, and analysis for causal practitioners. A future direction is combining natural-language interfaces with formal causal operators, graph surgery, and human-in-the-loop editing of richer causal representations.
Data Points: Time since last podcast appearance: just over 3 years - Host notes it has been more than three years since Robert was last on the show. Causal library contribution: PyWhy is an open source library with Microsoft as a major contributor - Robert describes the tooling ecosystem he works on at Microsoft Research. Conference trend period: 2022 - Robert repeatedly refers to last year as the period when several causal discovery trends advanced. Intervention setting: equilibrium data - In the unknown-intervention paper, the model assumes steady-state outcomes after intervention rather than time series propagation. Causal graph equivalence: same Markov equivalence class - He explains that different DAGs can be statistically indistinguishable from observational data. LLM benchmark source: Beyond the Information Game - He cites this large benchmark suite as including causal reasoning tasks. Language model example: Bing chat can generate a plausible DAG for smoking and lung cancer - Used as an example of LLMs surfacing causal knowledge. Course offerings: 2 current course tracks plus 1 new product-building course - AltDeep offers causal machine learning, probabilistic machine learning, and a forthcoming LLM product course.
Pivotal Quotes: "I think the pandemic also shifted sights again towards causality." — Robert Oszuanes: Explaining why interest in causal modeling remained relevant after black-swan disruptions exposed limits of historical data. "The more right my graph is, the faster it should adapt to the new data set." — Robert Oszuanes: Describing a paper that uses adaptation speed as a signal for learning causal structure. "I think you could have a causal model be an ombudsman for the causal veracity of what's coming out of these models." — Robert Oszuanes: On combining causal tools with LLMs to validate and constrain generated outputs.
Implications: Causal ML is moving from theory to practical integration with deep learning, simulators, and LLMs. Expect more tools that automate causal workflows, more benchmarks for causal reasoning, and more hybrid systems where formal causal methods validate model outputs.