#97 – Sertac Karaman: Robots That Fly and Robots That Drive

Episode Summary

Executive Summary: Sirtash Karaman argues that the biggest robotics challenge is not isolated autonomy, but deploying robots safely and usefully at scale in human environments. He contrasts driving, flying, Waymo, Tesla, and Optimus Ride, emphasizing simulation, machine learning, sensor fusion, human-in-the-loop operations, and the trade-off between efficiency, safety, and livability.

Main Topics: Autonomous driving vs. autonomous flying (Priority: 5/5): Karaman says consumer drone autonomy is easier today, but large-scale autonomous flying for transport/logistics is harder than driving because the airspace is less forgiving and deployment at scale is more complex. Robots in human-present environments (Priority: 5/5): He stresses that the real frontier is robots operating around untrained humans in cities, factories, and public spaces, which introduces algorithmic, social, legal, and business-model challenges beyond pure control theory. Simulation and machine learning (Priority: 5/5): Simulation is presented as essential for development and training, especially for cameras and dynamics; Karaman sees ML as necessary for perception and intelligence, while human behavior remains difficult to model. Autonomous vehicle strategy: Waymo, Tesla, Optimus Ride (Priority: 5/5): He contrasts cautious geofenced deployment, broad consumer rollout, and a middle path that uses human supervision across many vehicles to balance safety, scale, and usefulness. Human-in-the-loop operations and geofenced mobility (Priority: 4/5): Optimus Ride’s approach focuses on transportation-deprived, confined environments where a small fleet can deliver high value, use fewer shuttles, and potentially reclaim expensive parking space. Drone racing and aggressive flight (Priority: 4/5): AlphaPilot and fast drone flight are framed as a proving ground for high-throughput computing, better cameras, and new chip architectures that can support millisecond-level autonomy. Limits, timelines, and prediction uncertainty (Priority: 4/5): Karaman repeatedly cautions that 2-5 year or even 50-year forecasts are highly uncertain because technology gaps, especially in AI, are hard to quantify; iterative learning is the practical path forward.

Key Arguments: The hardest problem is not making robots move, but making them safe, efficient, and socially acceptable in environments designed for humans. Autonomous flying for consumer drones is currently easier than autonomous cars, but large-scale aerial transport is likely harder than road deployment. Simulation will become increasingly important for development, not just training, because it can model cameras, dynamics, and eventually more of the real world. Human behavior is one of the last major unsolved pieces for autonomy; predicting what other agents will do remains a core challenge. A human-in-the-loop control model can scale one operator across many vehicles, improving efficiency while keeping safety constraints in the vehicle. Optimus Ride targets geofenced, transportation-deprived areas first because they offer clearer economics, easier deployment, and visible societal benefits. Waymo, Tesla, and Optimus Ride represent different points on the spectrum of caution vs. ambition, but all are part of an iterative learning process. LIDAR is useful and often easier for demos, but camera-based autonomy is likely the long-term direction, especially with better compute and sensor fusion. Drone racing is valuable because it exposes the need for extreme perception and control at speeds beyond human capability. The best progress strategy is experimentation plus informed public communication, not overconfident timelines. Data Points: Penny production cost: 2.4 cents per penny - Lex cites Cash App ad copy about the cost of physical money. Annual penny production cost: $85 million annually - Lex mentions the yearly cost of producing pennies in the U.S. Travel example: Boston to New York: 1.5 hours - Karaman describes a possible aerial transport use case cutting a 4-hour trip to 1.5 hours. Travel time reduction example: 4 hours to 1.5 hours - Used to illustrate value of personal aerial transport. Transportation density target: Thousands or tens of thousands of autonomous vehicles - Karaman says true scale means seeing autonomous vehicles everywhere in a city. Geofence example size: Roughly 2 mile by 2 mile - He describes the type of contained environment Optimus Ride targets first. Potential parking savings: Tens of millions to billions of dollars - He estimates reclaiming parking space in geofenced deployments could save substantial land/economic value. Traffic deaths in the U.S.: 100 lives lost every day - Karaman cites road fatalities as a motivating reason for better autonomy. Human eye speed: Barely 100 Hz - Compared with drone vision systems that can operate at kilohertz rates. Drone vision system speed: 1 kHz - He notes research systems that can see much faster than humans. Vehicle control model: 10 people operate 50 vehicles - He describes Optimus Ride’s human-supervised fleet model as opposed to one driver per vehicle. Human monitoring ratio example: One person operates several vehicles - Summarizes the idea of fractional human supervision across a fleet. Data density example: 10,000 miles - Driving 10,000 miles in a confined environment yields repeated encounters and better prediction than 10,000 miles nationwide. Human-computation example: A hundred variables with 10 values each exceeds the number of atoms in the universe - Used to illustrate curse of dimensionality in decision-making.

Pivotal Quotes: "the real challenge of our time is to take these vehicles and put them into places where humans are present" — Sirtash Karaman: He explains why autonomy at scale is fundamentally harder than robots in factories or on Mars. "we want to go from that to 10 people operate 50 vehicles" — Sirtash Karaman: Describing the human-in-the-loop operating model behind Optimus Ride. "the question is you as a robot, should you be aggressive or not when faced with an aggressive robot?" — Sirtash Karaman: Discussing game-theoretic behavior and social interaction in autonomous driving.

Implications: Autonomy’s next breakthroughs may come less from fully driverless hype and more from incremental, human-supervised systems in constrained environments. The winners will likely balance safety, cost, and public trust while using simulation, vision, and better compute to scale.