Episode Summary
Executive Summary: The conversation argues that understanding intelligence requires reverse-engineering the brain’s learning and steering systems, not just scaling LLMs. The speakers emphasize cortex as a general omnidirectional inference engine, while evolution encodes complex loss functions, innate rewards, and cell-type-specific wiring in subcortical systems. They connect this to neuroscience infrastructure, connectomics, formal verification, and future AI alignment.
Main Topics: Brain vs. LLMs: What intelligence actually is (Priority: 5/5): The discussion centers on why LLMs still fall short of human capabilities and whether the brain’s secret lies in architecture, learning rules, initialization, or especially loss/cost functions. Cortex as omnidirectional inference / probabilistic prediction (Priority: 5/5): The speakers propose that cortex may not be mainly next-token prediction, but a general predictor that can infer any missing subset from any observed subset, closer to energy-based or probabilistic models. Steering subsystem, evolution, and innate reward wiring (Priority: 5/5): A major thesis is that evolution encodes high-level desires through a separate steering subsystem (subcortical structures, amygdala, hypothalamus, brainstem) that learns to predict and shape innate reward responses. Neuroscience tooling, cell types, and connectomics (Priority: 4/5): They argue that neuroscience is bottlenecked by technology and needs large-scale connectomics, single-cell atlases, and molecularly annotated maps to uncover the architecture of learning and reward systems. RL, value functions, and brain-like learning (Priority: 4/5): The conversation contrasts LLM training with RL and discusses whether human and animal brains use model-free RL, value functions, and model-based inference in different circuits. Formal methods, Lean, and proof automation (Priority: 4/5): The speakers describe Lean as a major opportunity for AI-driven theorem proving and verified software, with broader implications for math productivity and cybersecurity. Research infrastructure and moonshot science (Priority: 3/5): They frame projects like connectomics and formal verification as underbuilt scientific infrastructure problems that need focused funding, technology development, and coordinated effort.
Key Arguments: The brain likely relies less on simple next-token-style objectives and more on complex, evolution-shaped loss functions that bootstrap learning over development and across contexts. Cortex may function as a general-purpose omnidirectional inference engine: it can predict any subset of variables from any other subset, not just the next item in a sequence. Evolution did not need to encode every future concept; it only had to encode the mechanisms that map learned world-model features onto innate reward/steering signals. A separate steering subsystem may explain how abstract learned concepts like 'spider on your back' can trigger innate responses such as flinching or shame. Neuroscience progress is constrained by missing infrastructure; better maps of cell types and connectivity could answer questions that discussion and theory alone cannot. LLM success does not prove the brain works the same way, but AI models can still be useful hypotheses for computational neuroscience. Formal verification and Lean-style proof systems are likely to automate large parts of mathematics and software validation, even if creative conjecture remains hard. The biggest practical bottlenecks in aligning or understanding AI may be the same bottlenecks neuroscience faces: representation, reward structure, and integration across modalities.
Data Points: Human genome size: ~3 gigabytes - Used to illustrate how little information evolution can encode directly compared with the complexity of behavior and reward wiring. Mouse connectome cost estimate: a few billion dollars - Referenced as the likely cost of producing the first full mouse brain connectome under older assumptions. Mouse connectome target cost: low tens of millions of dollars - Target cost range discussed for newer connectomics technology. Human brain size relative to mouse: ~1000x bigger - Used to extrapolate the rough cost of human connectomics if naively scaled from mouse. Fixed training compute split in experiment: 16 agents; best agent gets 1/16th the compute - In the multi-agent AlphaZero-style experiment, the best agent in the population outperformed a single agent trained with all the compute. Science infrastructure estimate: few hundred fundamental capabilities - The 'gap map' was described as containing around a few hundred foundational capabilities/gaps. Projected funding scale: hundreds of millions to low billions of dollars - Estimated as the scale needed to make meaningful progress on connectomics and related neuroscience infrastructure. Mouse brain connectome cost in old model: several billion dollars - Wellcome Trust report cited for the first mouse brain connectome under earlier technology assumptions.
Pivotal Quotes: "I think evolution may have built a lot of complexity into the loss functions." — Adam Marblestone: Core thesis about why brains may learn efficiently with limited genetic information. "The cortex has typically this like six-layer structure... I've seen versions of that where what you're trying to explain is actually just how does it approximate backprop." — Adam Marblestone: Explaining one computational neuroscience lens on cortical learning. "I think the cortex is just natively made so that it can... predict any pattern in any subset of its inputs, given any other missing subset." — Adam Marblestone: Describing the omnidirectional inference hypothesis for cortex.
Implications: If these ideas are right, the path to better AI runs through neuroscience infrastructure, formal verification, and better understanding of reward/steering systems—not just scaling current models. It also suggests alignment may depend on how we engineer goals, not only capabilities.