Episode Summary
Executive Summary: Ken Goldberg discusses major robotics progress since 2020, emphasizing that difficult real-world manipulation problems like cable untangling, grasping, and package sorting are advancing through better hardware, self-supervised learning, and sim-to-real methods. He highlights the importance of studying failures, using geometry/physics when sensing is limited, and building practical automation systems that are robust, low-cost, and human-supervised when needed.
Main Topics: Progress in robotics since the pandemic (Priority: 5/5): Goldberg argues the past few years have been unusually productive for robotics, with major gains in research output, system capability, and practical deployment, even as some long-standing problems remain hard. Cable untangling as a systems challenge (Priority: 5/5): He describes the evolution from small surgical-robot cable experiments to a full-scale dual-armed Yumi system that untangles three-meter cables, combining perception, planning, and custom gripper hardware. Caging, new manipulation primitives, and failure analysis (Priority: 5/5): A key innovation is a gripper 'toe' that enables caging and cage-pinch-dilation. Goldberg stresses that progress comes from systematically categorizing failures and inventing new primitives to address them. Learning, self-supervision, and human-in-the-loop robotics (Priority: 4/5): The conversation covers training perception models from large collections of examples, using self-supervised data generation, and calling in humans only when uncertainty or stagnation makes automation insufficient. Sim-to-real and real-to-sim for physical robustness (Priority: 5/5): Goldberg explains why simulation alone fails for contact-rich manipulation, and how tuning simulators to match real data before generating synthetic data improves policies, especially for cable dynamics. Robotics, automation, and practical deployment (Priority: 4/5): He distinguishes robotics from software bots and emphasizes that real progress requires cost, reliability, safety, and production-ready systems, not just impressive demos or simulator results. Industry impact: Tesla Optimus, tactile sensing, and Ambi Robotics (Priority: 4/5): Goldberg comments on Tesla's humanoid push as good for the field but still far from practical household robotics, notes progress in tactile sensing, and describes Ambi Robotics' package sorting deployments with Pitney Bowes.
Key Arguments: Robotics has advanced significantly over the last few years, but the hardest open problems are still physical manipulation tasks involving contact, friction, deformable objects, and uncertainty. Studying failure modes is more valuable than only reporting average success rates, because failures reveal where new methods and primitives are needed. Hardware innovation can unlock algorithmic progress; the added 'toe' on grippers enabled caging, which made cable untangling far more effective. Deep learning helps with perception tasks like knot detection, but robotics still needs geometry, physics, and structured reasoning to operate in the real world. When sensing is insufficient, robots can use mechanics and motion strategies to infer contact or guarantee capture instead of relying on perfect measurement. Human-in-the-loop control should be sparse and confidence-driven so one person can supervise many systems without being constantly interrupted. Simulation is useful but insufficient on its own; policies learned in deterministic simulators often fail in the real world unless the simulator is calibrated to reality. The real-to-sim-to-real workflow can outperform both pure real-data training and naive simulation by using real data to tune simulators before scaling up training. Robotics differs from software automation because it must bridge the digital and physical worlds; practical adoption depends on cost, reliability, and safety. Tesla's Optimus may be strategically important for market and field attention, but practical humanoid robotics remains much harder than many assume. Ambi Robotics is focusing on real warehouse automation, where package sorting can measurably reduce labor burden and improve throughput. Tactile sensing is becoming more practical and could be a major enabler for manipulation, especially as optical tactile sensors improve in resolution and speed.
Data Points: Time since previous podcast appearance: 2.5 years - Sam notes it has been two and a half years since their last conversation in March 2020. Cable length in full-scale untangling system: 3 meters - The Yumi-based untangling project handles cables far larger than the robot's workspace. Workspace size: about the size of a dinner plate - Goldberg describes the dual-arm robot's working area as surprisingly small. Earlier cable experiments: about 8 inches - Initial untangling work used very small cable segments with a surgical Da Vinci robot. Depth camera speed: about 1 frame per second - He says the structured-light depth camera used in the system is slow and limited by scanning and specularities. Conference paper submissions: 500 papers submitted - Goldberg mentions the upcoming Conference on Robot Learning had received 500 submissions. Conference presentations: about 200 papers - He says roughly 200 papers would be presented at the conference. Conference registration cost: close to $200 - He says online registration to watch the talks is inexpensive. Ambi Robotics installations: 60 systems - He says 60 AMBI sort systems were installed across America over the summer.
Pivotal Quotes: "We're very far from being able to do that reliably." — Ken Goldberg: On why grasping and manipulation remain hard for robots even though humans do them effortlessly. "I really believe in projects that tend to continue over multiple years." — Ken Goldberg: He explains his lab's long-term approach of iterating on core problems like DexNet and cable manipulation while analyzing failures. "I don't think [Optimus is] going to quickly discover how complicated that is." — Ken Goldberg: On Tesla's humanoid robot, emphasizing that practical humanoid robotics is much harder than it may appear.
Implications: The field is shifting from flashy demos toward robust, deployable automation. Expect progress from better hardware, calibrated simulation, tactile sensing, and human-supervised systems, especially in warehouses and other structured environments.