Episode Summary
Executive Summary: Roland Memosevic argues that language, perception, and action must be unified to build human-like AI, and that current LLMs are powerful but incomplete because they lack true recurrence, durable memory, and grounded agentic interaction. He highlights Qualcomm AI Research’s work on visual reasoning, situated chat, and drawing agents as steps toward systems that learn reasoning through embodied, end-to-end supervision rather than modular pipelines.
Main Topics: Language as the foundation of human-like intelligence (Priority: 5/5): Memosevic frames language as a central ingredient for intelligence, explaining that labels, captions, and conversational interaction provide rich supervision that helps models learn concepts beyond simple classification. From noun-centric vision to verb-centric, grounded supervision (Priority: 5/5): He traces his earlier work on action- and caption-based datasets as a shift from static object recognition toward temporal, relational, and verb-like understanding that teaches object permanence, causality, and commonsense. LLMs, transformers, and the case for recurrence (Priority: 5/5): He argues that transformers’ parallel training efficiency is the key reason they dominate today, but that recurrence will likely return to improve sequential reasoning, memory, and length generalization. Reasoning as agentic, sequential behavior (Priority: 5/5): Memosevic rejects the claim that LLMs cannot reason, arguing instead that both humans and AI agents operate token-by-token over time, and that reasoning should be understood as an agentic process embedded in the world. Visual grounding and embodied reasoning research (Priority: 4/5): He describes research such as Look, Remember, Reason and related work that connects frozen language models to visual input to solve reasoning tasks requiring tracking, occlusion handling, and counterfactual inference. Situated chat and real-time AI companions (Priority: 4/5): The conversation covers a newer embodied AI system using LLMs and text-to-speech for real-time conversational coaching, with Fitness Ally as a practical example of how companionship and accountability can create user value. Open problems: memory, self, and future AI architectures (Priority: 5/5): He emphasizes unresolved questions around memory, fast weights, self-modeling, and the emergence of the word 'I,' suggesting these may be necessary for more capable and human-like AI systems.
Key Arguments: Language is not just output; it is the mechanism by which conceptual richness and common-sense supervision can be instilled into models. Captioning and action-oriented datasets are valuable because they teach models about persistence, occlusion, causality, and other world-structured concepts rather than only object labels. Transformers outperform RNNs mainly because they are far more computationally efficient to train in parallel, not because RNNs are fundamentally incapable of language understanding. LLMs exhibit reasoning-like behavior, but true reasoning should be viewed as sequential, agentic processing over time rather than a disembodied text-generation trick. Grounding should mean more than associating words with images; it should involve agents interacting with an environment, learning procedures, and linking language to action. Visual reasoning tasks are a practical way to probe whether language models can use visual context and chain-of-thought style rationales to infer hidden or counterfactual states. Recurrence and memory are likely missing ingredients for progress beyond current LLMs, especially for length generalization and stable factual retention. A system that can see, talk, act, and revise its behavior in a real environment is more likely to develop human-like commonsense and reasoning than a purely text-based model.
Data Points: Qualcomm acquisition timing: around two years ago - Memosevic says his co-founded company 20 Billion Neurons was acquired and the team moved to Qualcomm roughly two years before the interview. Previous interview recency: 2-3 years ago - He and the host note that their last conversation was around two to three years earlier. Earlier first interview: about 5 years ago - The host recalls their first interview being roughly five years before this episode. ImageNet class count: 1,000 classes - Memosevic references ImageNet training as a milestone where universal visual features emerged from supervision on a thousand categories. Research time complexity factor: t - He explains that RNN training incurs a linear time factor proportional to sequence length, unlike transformers' more parallel processing. Hypothetical RNN training duration: 100 years - He suggests an old-school LSTM/RNN trained long enough could match or exceed GPT-like performance, but would take an impractically long time. Avatar video-chat prediction lag: 2 years out - He says realistic AI avatar video chat still feels about two years away, though it has been 'two years out' for some time. Benchmarked accuracy example: 92% vs 95% - He cites typical benchmark comparisons showing strong performance gains on reasoning tasks despite simple architectures.
Pivotal Quotes: "I think recurrence will be back, right?" — Roland Memosevic: He argues that recurrence will likely re-enter future AI architectures to improve memory and sequential reasoning. "I don't agree with that." — Roland Memosevic: His response to the claim that LLMs cannot reason; he argues reasoning is compatible with sequential token generation and agentic behavior. "Grounding goes significantly further than that." — Roland Memosevic: He distinguishes true grounding from simple vision-language alignment, insisting it must include action, environment, and self-referential learning.
Implications: The conversation suggests the next AI leap may come from grounding, recurrence, and memory—not just larger language models. For industry, this points toward embodied, multimodal agents that reason through interaction rather than text alone.