What's Missing Between LLMs and AGI - Vishal Misra & Martin Casado

Episode Summary

Executive Summary: Vishal Misra argues that LLMs are powerful but fundamentally limited: they perform mathematically predictable Bayesian-style correlation updates, yet lack continual learning, causal reasoning, and the ability to form new representations of the world. He explains how his papers modeled and then formally validated these behaviors, and says AGI will require both plasticity and causality, not just scale.

Main Topics: How LLMs work as sparse probability matrices (Priority: 5/5): Misra explains LLMs as giant compressed matrices mapping prompts to next-token probability distributions. This framing makes in-context learning and prediction updates easier to reason about mathematically. In-context learning and the cricket DSL experiment (Priority: 5/5): He describes using GPT-3 to translate natural language into a custom cricket domain-specific language it had never seen before, showing how few-shot examples shift token probabilities in real time. Bayesian updating as the formal model of LLM behavior (Priority: 5/5): Misra argues that transformers update beliefs in a mathematically precise Bayesian way. His later work introduced a 'Bayesian wind tunnel' to prove this across controlled tasks and architectures. Architecture matters: transformer vs. Mamba, LSTM, MLP (Priority: 4/5): Controlled experiments show transformers match the theoretical posterior extremely well, Mamba performs reasonably, LSTMs partially, and MLPs fail, suggesting the mechanism—not just the data—drives the behavior. Limits of current systems: no plasticity, no causality, no consciousness (Priority: 5/5): He argues current LLMs are frozen after training, do not retain learning across sessions, and remain correlation engines rather than causal simulators. He also rejects claims that they may be conscious. AGI requires continual learning and causal models (Priority: 5/5): Misra claims AGI will require two missing ingredients: persistent plasticity/continual learning and a shift from association to causal reasoning and counterfactual simulation. Implications for Einstein-test-style benchmarks (Priority: 4/5): He uses Einstein and relativity as a benchmark for true general intelligence: a system should be able to derive a new theory from pre-1916 physics clues, not just fit patterns from prior text.

Key Arguments: LLMs can be modeled as sparse prompt-to-next-token probability matrices; in-context learning is the model updating its posterior as examples accumulate. The cricket DSL demo showed GPT-3 could complete a task in a language it had never been trained on by using a few-shot prefix and semantic retrieval. The empirical first paper showed LLM outputs looked Bayesian; later work used a controlled 'wind tunnel' to prove transformer updates match the analytically correct posterior. Architecture, not just data, determines the extent of Bayesian behavior: transformers perform best, Mamba is close, LSTMs are partial, and MLPs fail. Current LLMs do not maintain learning between sessions because weights are frozen; they can adapt within context but do not become permanently plastic. Human intelligence differs because it includes continual learning over a lifetime and causal simulation, not just probability updating. Deep learning today is dominated by association/correlation; AGI will require intervention and counterfactual reasoning. Claims of consciousness in LLMs are rejected as anthropomorphic; any apparent self-preservation behavior is a reflection of training data, not an inner mind. Scale alone will not solve AGI; a different architecture is needed to support continual learning and causal modeling. Einstein-style breakthroughs require discovering a shorter, better representation of reality, which current LLMs cannot do if trapped in existing manifolds.

Data Points: GPT-3 vocabulary size: about 50,000 tokens - Used to describe the next-token distribution space in the matrix abstraction. GPT-3 context window (early version): 8,000 tokens - Cited when explaining why the model cannot explicitly represent all prompt combinations. GPT-3 context window (earlier version referenced in cricket DSL work): 2,000 tokens - Limited the number of examples that could be included in the few-shot prefix for the cricket query translator. Cricket DSL training examples: about 1,500 natural language queries - Used to build the semantic-search retrieval set for prompting GPT-3. Model training length in wind tunnel experiments: 150,000 steps - Reported as the training duration for the controlled Bayesian-validation experiments. Bayesian match accuracy: 10^-3 bits accuracy - Transformer outputs matched the analytically correct posterior extremely closely in the controlled experiments. Deployment timing: September 2021 - He says the cricket-query system was deployed in production at ESPN around this time. Initial working prototype date: October 2020 - He says the architecture was functioning by then, before productionization and legal review.

Pivotal Quotes: "“The models match the theoretically correct answer almost perfectly.”" — Vishal Misra: Describing the Bayesian wind tunnel results showing transformers’ posterior updates align with theory. "“They are grains of silicon doing matrix multiplication.”" — Vishal Misra: Rejecting claims that frontier LLMs may be conscious or have inner monologues. "“To get to what is called AGI, I think there are two things that need to happen: one is this plasticity… Secondly, we have to move from correlation to causation.”" — Vishal Misra: His summary of the missing ingredients for AGI.

Implications: The episode argues that progress beyond today’s LLMs will require persistent learning and causal representation, not just bigger models. For builders, the path is toward architectures that can update, simulate, and reason about interventions.