The Ethics of AI in Healthcare: Beyond the Stochastic Parrot | Dr. Jessica Morley (Yale Digital Ethics Centre)

"AI has the potential to re-ontologize healthcare—to completely redesign what we consider to be a disease, what we consider to be a disability, and how we organise care. But we need to decide what good healthcare actually means before we AI-ify everything."— Dr Jessica MorleyIn this episode of Clinical Changemakers, Dr Jessica Morley, an AI ethics expert and researcher from Yale Digital Ethics Centre, challenges our understanding of what large language models actually are and what they mean for healthcare. Drawing from her work on data ethics and her involvement with OpenSafely during the pandemic, Dr Morley explains why viewing AI as "stochastic parrots" is crucial for healthcare implementation, explores the concept of re-ontologizing medicine, and argues why we need ethical frameworks before technological deployment rather than after.Key TakeawaysStochastic Parrots in Medicine: Large language models don't understand medical concepts—they predict the most likely next word based on probability from their training data. This means they can give you different answers to the same medical question and lack the contextual understanding crucial for patient care. Understanding this fundamental limitation is essential for safe healthcare implementation.The Re-ontologizing Power of AI: AI doesn't just replace existing tools like upgrading from a physical to digital stethoscope. It has the power to completely redesign healthcare by redefining what constitutes disease, changing how we organize care, and separating diagnosis from physical patient interaction. This transformation can be powerful and positive, but only if we're intentional about our goals.The Inverse Data Quality Law: Just as the inverse care law states that those who need healthcare most get it least, the inverse data quality law means those who need AI healthcare tools most will have the poorest quality data about themselves. This creates a two-tier system where marginalized populations get inferior AI-driven care.Social License Trumps Legal Permission: Technical feasibility and legal compliance aren't enough for successful AI implementation in healthcare. Projects like the UK's Care.data failed despite being legal because they lacked social acceptance. OpenSafely succeeded by building in privacy protections, transparency, and meaningful public engagement from the start.Where to Find Our GuestDr Jessica Morley (LinkedIn, Google Scholar)In This Episode01:27 - Stochastic parrots: Understanding what LLMs actually do04:27 - Emergent properties: Why LLMs remain sophisticated probability machines11:57 - Moral worth and parasocial relationships: When humans attach meaning to AI14:33 - Re-ontologizing healthcare: How AI redesigns medicine itself19:28 - The ethical traps: What happens when we AI-ify without thinking about outcomes25:33 - Two-tier systems: How AI could worsen healthcare inequalities28:44 - Ethics vs. law: Why we need both rules and values32:31 - Learning from NHS failures: The importance of not making assumptions47:43 - Global policy tensions: EU regulation vs. US "let it rip" approaches54:33 - False dichotomies: Moving beyond "some care vs. no care" thinking59:02 - Current global sentiment: From tech optimism to healthcare cautionReferenced* Dr Morley’s Paper - The ethics of AI in health care: A mapping review (link)* Defining a Stochastic Parrot (link)* OpenSafely platform and approach to health data research (link)* Lessons from Care.data project and its failure in the UK (link)ContactIf you have any feedback, questions or if you'd like to get in touch, reach out at [email protected] there 🙌As a small independent podcast, every rating and share makes a real difference in helping us reach more healthcare leaders. If you found value here, please rate us and pass this along to a colleague who needs to hear it.Clinical Changemakers is a podcast that explores inspiring stories of leadership and innovation in healthcare. To learn more and join the conversation, visit: www.clinicalchangemakers.com This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.clinicalchangemakers.com

There are no comments yet.

Please log in to write the first comment.