Jonathan Chen: Can algorithms make doctors better?

Episode Summary

Executive Summary: Jonathan Chen argues that AI can improve medicine by learning from the collective patterns in real clinical practice, not just images. The biggest promise is decision support from large-scale observational data, but he stresses medicine’s complexity, workflow constraints, shifting standards, local variation, bias risks, and weak regulatory clarity. He sees AI as augmenting clinicians, not replacing them.

Main Topics: AI as clinical decision support beyond imaging (Priority: 5/5): The episode frames AI as useful not only for radiology/dermatology/pathology, but also for helping clinicians choose tests, treatments, and next steps in complex cases by learning from prior similar patients and expert behavior. Medicine as a complex, variable system (Priority: 5/5): Chen emphasizes that medicine is not a clean algorithmic process; uncertainty, nonstandard practice, and human variability make it hard to automate and explain, but also create an opportunity for AI to reduce inconsistency. Collaborative filtering and learning from many clinicians (Priority: 5/5): A central concept is adapting recommender-system ideas from Amazon/Netflix to medicine: use the aggregate experience of many doctors and patients to suggest actions for a current case. Data quantity, quality, and relevance (Priority: 4/5): The discussion highlights that labeled data are expensive, unlabeled clinical data are abundant, and more data is not always better because older or heterogeneous data can add noise and outdated practice patterns. Workflow design and clinician adoption (Priority: 4/5): Chen describes prototype interfaces that offer unobtrusive suggestions rather than alerts, reflecting alert fatigue concerns and the need to fit recommendations into real clinician workflows. Bias, equity, and unintended consequences (Priority: 5/5): The conversation warns that AI can amplify existing inequities if trained on biased care patterns or if protected attributes are simply omitted, making fairness a major unresolved research and policy issue. Regulation and explainability (Priority: 4/5): They discuss uncertainty about how FDA-style regulation should apply to decision-support software, especially systems that are not fully autonomous but still influence human medical decisions.

Key Arguments: AI is most promising in medicine when it helps clinicians manage complexity and variability rather than trying to fully automate care. Image-based AI is already useful, but many high-value opportunities lie in decision support for diagnosis, test ordering, and treatment selection. Doctors effectively act as manual annotators every day; their decisions create valuable unlabeled data that could be learned from systematically. Collaborative filtering can capture the collective experience of many clinicians and reveal patterns that individual experts cannot easily articulate. More data is not automatically better; relevance, recency, and local practice patterns matter, and older data may be misleading. A well-designed AI tool must fit into the clinician workflow and avoid disruptive alert fatigue by offering optional, unobtrusive suggestions. AI can amplify both good practice and existing bias; ignoring race/ethnicity/gender does not guarantee fairness and may hide structural inequities. There is no simple technical fix for algorithmic bias; solving it likely requires collaboration with social scientists, ethicists, and policy experts. Current regulation is not well matched to partially automated medical decision-support systems, and responsibility remains unclear when clinicians rely on AI recommendations. AI should be viewed as augmenting scarce human expertise, not replacing clinicians, because people still perform the physical and interpersonal tasks of care.

Data Points: Doctors in prototype evaluation: 43 - Chen’s team had 43 real doctors test a simulated decision-support interface. Timeframe for AI/medical information systems discussion: 1980s and earlier - Chen notes that ideas about using computers to assist medicine date back at least to the 1980s. Historical shift in feasibility: 10–15 years ago - He says many current prototype ideas would have been hard to even prototype 10 to 15 years earlier because data infrastructure was insufficient. Care variation in last 6 months of life: 6x difference - He cites a known phenomenon where some doctors provide about six times more care than others near end of life. Health care received as recommended: ~50% - He mentions that patients may receive only about half of the recommended health care they are supposed to get. Data scaling complexity: 2^10 to 2^1000 scale growth - He argues that adding data is not linear in usefulness because problem complexity grows explosively with dimensionality.

Pivotal Quotes: "The reality is, I'm actually worried. I'm actually more worried than I am confident that this is going to be managed." — Jonathan Chen: On the challenge of preventing AI from worsening health inequities. "We are an army of manual annotators. That's what we do every day." — Jonathan Chen: On how everyday clinical decisions generate valuable data for learning systems. "We can now build a lot of the really cool software, but software doesn't do anything without the hardware. And the hardware, it's people basically." — Jonathan Chen: On the need to integrate AI into real clinical workflows and human care.

Implications: AI in medicine will be most useful as a carefully designed decision aid grounded in real practice data, but success depends on workflow fit, fairness, regulation, and human oversight. The field’s biggest challenge is not just accuracy, but responsible deployment.