Episode Summary
Executive Summary: Yuri Leskovitz explains how Kumo’s relational foundation models apply pretraining and in-context learning to raw multi-table enterprise data, enabling accurate predictions without manual feature engineering. He connects this to his Stanford work in AI for science and argues that graph-like structure underlies biology, social networks, and databases. The conversation covers benchmarks, performance gains, deployment, explainability, and why agentic workflows need prediction tools designed for relational data.
Main Topics: Relational foundation models for enterprise data (Priority: 5/5): Kumo’s RFM2 is presented as a pre-trained model that reasons directly over structured relational databases, using in-context examples rather than task-specific retraining. Why multi-table data matters more than flattened tabular data (Priority: 5/5): Leskovitz argues that the real unsolved problem is learning from raw multi-table databases, not single-table tabular ML, because flattening destroys information. AI for science and the virtual cell (Priority: 4/5): At Stanford, Leskovitz works on foundation models for cells, patients, and molecules, aiming to represent biomedical systems emergently from data rather than hand-coded biology. Benchmarks, performance, and evaluation (Priority: 4/5): He describes the creation of RHEL Bench and references SAP’s SALT benchmark to evaluate relational prediction tasks fairly and at scale. Deployment, explainability, and production use cases (Priority: 4/5): The discussion covers how Kumo is deployed, why explainability is possible via attention over tables/cells, and how customers use it for fraud, recommendations, ads, and sales. Agents and API design for data science (Priority: 3/5): Leskovitz argues that agents need higher-level, relational-aware APIs to avoid subtle data leakage and modeling mistakes when building predictive systems.
Key Arguments: Relational data should be modeled directly as graphs, because the database schema already encodes meaningful entity relationships. Flattening multi-table data into a single table loses signal through aggregation and therefore limits model performance. Neural networks applied to raw relational data reduce feature engineering, bugs, leakage, and time-travel errors common in production ML pipelines. Pretrained relational foundation models can perform in-context learning: they use labeled subgraphs extracted from the database to predict new targets without training loops. Fine-tuning and pretraining are complementary; pretraining learns general relational priors, while fine-tuning can improve cost and task-specific accuracy. The model is most valuable for well-defined predictive tasks such as fraud, churn, recommendation, ads, and sales conversion. Explainability is stronger than with tree models because the system can identify attended tables, columns, and cells, then summarize them with an LLM. Agentic systems need structured predictive tools, not just LLM common sense, because real business decisions depend on accurate downstream prediction.
Data Points: RHEL Bench tasks: ~40 predictive tasks - Stanford benchmark for multi-tabular relational learning RHEL Bench databases: ~10–15 databases - Curated open multi-tabular datasets used for benchmarking Kumo RFM2 improvement over supervised SOTA: ~5% relative accuracy gain - Foundation model performance on benchmark tasks RFM2 after fine-tuning: ~12% over state of the art - Additional gain when the model is tuned for a specific task Single-cell RNA-seq dimensionality: 20,000-dimensional vector - Each cell is represented by gene/protein abundance in the biomedical example Kaggle multi-tabular competitions: 4 out of thousands - Used to illustrate the scarcity of true multi-table benchmarks Kumo model size: Billion-parameter type models - Described as relatively small and efficient compared with massive LLMs Inference speed: 0.2 to 0.5 seconds - Approximate time for a single forward-pass prediction Deployment impact at DoorDash: Hundreds of millions of dollars in revenue impact - Restaurant recommendations and notification timing use case Ad click-through lift at Reddit: Nearly double-digit increase - Advertising models built with Kumo embeddings and features Enterprise customer scale example: 70,000 customers - Illustrates why one-off per-client modeling does not scale for SAP-like platforms Coding-agent effort comparison: ~50 lines vs 500 lines vs thousands of lines of code - Kumo API vs XGBoost-style feature engineering vs PyTorch from-scratch modeling
Pivotal Quotes: "“The recent breakthrough that we had, and we just released the second version, is our what we call a relational foundation model.”" — Yuri Leskovitz: Introduces Kumo’s RFM2 and frames the core technical advance "“What we see is that the foundation model by itself improves state of the art over all supervised models ever published on this benchmark.”" — Yuri Leskovitz: Describes benchmark results for the relational foundation model "“Just give me the data. You know, like that's what I mean.”" — Yuri Leskovitz: Explains frustration with manual feature engineering and aggregation choices
Implications: Relational AI may shift enterprise ML from manual feature engineering to model-driven prediction over raw databases. If the results hold broadly, it could improve fraud, recommendations, and agentic decision-making while reducing production complexity and leakage.