What moral decisions should driverless cars make? | Iyad Rahwan

Episode Summary

Executive Summary: Iyad Rawan argues that autonomous vehicles are not just a technical innovation but a societal ethics problem. Using trolley-problem scenarios, he shows that driverless cars must make trade-offs, that people prefer utilitarian rules in theory but self-protective cars in practice, and that regulation must solve a collective-action dilemma to preserve public safety.

Main Topics: Ethics of autonomous vehicle decision-making (Priority: 5/5): Driverless cars will face unavoidable trade-offs in crash scenarios, forcing choices between harming passengers, pedestrians, or bystanders. Trolley problem as a practical framing device (Priority: 4/5): The talk uses philosophical dilemmas to illustrate that real-world autonomous driving involves probabilistic harm minimization, not simplistic binary choices. Public preference vs personal self-interest (Priority: 5/5): Survey results show people endorse utilitarian outcomes in principle but do not want to buy cars that might sacrifice them, revealing a social dilemma. Tragedy of the commons / algorithmic commons (Priority: 5/5): Individual incentives to prioritize passenger safety can collectively reduce public safety, especially when algorithms and manufacturers optimize for clients. Regulation as collective enforcement (Priority: 4/5): The speaker argues that governments must define and enforce acceptable trade-offs, but regulation itself may reduce adoption if consumers reject it. Moral Machine and global data collection (Priority: 4/5): The Moral Machine project gathers millions of decisions worldwide to map cultural attitudes toward ethical trade-offs in autonomous vehicles. Asimov and the broader societal question (Priority: 3/5): The talk closes by reframing the issue as a cooperation problem about humanity-wide harm, not just a car-by-car engineering problem.

Key Arguments: Autonomous cars will inevitably make ethical trade-offs because real crashes involve probabilistic risk distribution, not perfect options. Waiting for 100% safety is itself a choice, and delaying deployment can cost many lives if safer-than-human technology is withheld. Most people endorse utilitarian ethics for cars in surveys, but they reject buying cars that might sacrifice them, creating a classic social dilemma. The problem resembles the tragedy of the commons: each owner wants maximum personal safety, but collectively that can undermine public safety. In autonomous systems, the dilemma becomes more insidious because algorithms may learn to shift risk onto pedestrians without a human explicitly choosing it. Regulation is necessary to coordinate behavior, but if consumers refuse regulated cars, the policy could paradoxically increase harm by slowing adoption of safer vehicles. Large-scale public input tools like Moral Machine can help society understand values and inform regulation, even if they do not provide a final answer.

Data Points: Traffic deaths in the U.S. last year: 35,000 - Estimated by the Department of Transport, used to motivate the potential benefits of driverless cars. Global annual traffic deaths: 1.2 million - Worldwide deaths from traffic accidents cited at the start of the talk. Potential reduction in accidents: 90% - The speaker asks whether people would support technology that could eliminate 90 percent of crashes. Potential reduction in accidents: 99% - Used to illustrate that even near-total safety may still leave difficult trade-offs. Additional research delay: 50 more years - Hypothetical delay needed to eliminate the last 1 percent of accidents. Lives lost if current rate continues: 60 million - Projected deaths over the delay period if adoption is postponed. Public survey responses: Most people sided with Bentham - Survey participants preferred utilitarian ethics in abstract scenarios. Public purchase intent: Absolutely not - Participants said they would not buy cars that might sacrifice them, despite supporting utilitarian rules for others. Moral Machine decisions collected: Over 5 million decisions - Total decisions gathered through the website. Moral Machine participants: Over 1 million people worldwide - Number of users contributing decisions to the project. Regulatory checklist items: 15-point checklist - The Department of Transport’s first set of regulations included a checklist for car makers. Ethical consideration checklist item: Number 14 - The specific checklist item referencing ethical considerations.

Pivotal Quotes: "the problem may be a little bit more insidious" — Iyad Rawan: Describing how algorithmic decision-making can shift risk without a clear human decision-maker. "the tragedy of the algorithmic commons" — Iyad Rawan: His term for collective-action failure when autonomous systems optimize individual safety at the expense of public safety. "a robot may not harm humanity as a whole" — Iyad Rawan: Citing Asimov’s zeroth law as a broad framing for the societal stakes of autonomous systems.

Implications: Autonomous vehicle policy must balance ethics, adoption, and collective safety. The industry needs public input and enforceable standards, or safer technology may be rejected despite saving lives.