With new technologies promising to reshape patient care, pharmacy staff must help their health systems plan for the ethical dilemmas this shift could bring.
That's the overarching message presenters of the two-part Joseph A. Oddis Ethics Colloquium: Ethical Dimension of Artificial Intelligence (AI) in Pharmacy, plan to deliver Sunday, June 9, at ASHP Pharmacy Futures 2024. Speakers Scott Nelson, Casey Olsen, and Kenneth A. Richman provided a preview of their session for ASHP's News Center.
“The use of AI is becoming a lot more accessible, so we have to be careful to apply it in ethical and appropriate ways,” said Nelson, a pharmacist and associate professor in the department of biomedical informatics at Vanderbilt University Medical Center.
Plan for the ethical pitfalls
- Like most major corporations, health systems are at various stages of building plans and teams to address ethical problems that arise from the widespread collection and analysis of huge amounts of data, especially when it’s used to train machine learning models.
- “Health systems are using what has been done historically to guide their oversight in this area,” said Olsen, a pharmacist and informatics manager at Advocate Health. “But how will historic processes, such as informed consent, need to change to meet the demands of emerging technology?”
- “AI evaluation requires active monitoring structures and similar styles of oversight that we see for bringing a drug onto the formulary: clear evaluation of the modality, the benefits it provides, and the risks it brings,” Olsen said.
Beware of bias …
- Researchers have found evidence of bias in algorithms used in healthcare, including a well-known example published in Science that showed Black patients were less likely than white patients to get extra medical health, despite being sicker, when a large hospital used one particular algorithm.
- “We value justice and equity across different populations but that can conflict with the desire to have a model that’s quick, accurate, improves patient outcomes,” said Nelson. “What we’ve seen with some models though is that it can benefit some groups of people but not other groups.”
… and a lack of transparency
- Another potential AI-related problem is if the reasoning behind the technology’s recommendation is unclear. “If someone were to ask ‘How did you make your decision?’ providers have to be able to explain themselves,” said Nelson.
- “Many health systems rely on alerts or passive decision support to guide interventions and use limited real estate in electronic health records to translate the ‘why,’” said Olsen. “As the ‘why’ becomes more difficult to communicate, we need to provide ways to instill trust in the systems.”
AI will change the provider-patient relationship
- Richman, a professor of philosophy and healthcare ethics at MCPHS University in Boston, will discuss competing values: between the expectations of classic provider-patient relationships and the factors that matter to AI implementation.
- One part of the presentation focuses on the risk of “mindlessness” if providers simply follow technology’s recommendations, without reflecting on them. “If relying on AI applications reduces provider understanding, this reduces transparency, accountability, and human agency,” the presentation will note. “Is this loss outweighed by increased efficiency, reliability and/or validity?”
Speakers say their event will include in-depth discussion of case studies, but no detailed knowledge of AI is required. Rather, the program is designed to help attendees better understand the right questions to ask.
“This will help provide some additional frameworks for how you look at AI,” Nelson said. “This isn’t going to turn anyone into a data scientist but will get them familiar with the kinds of questions to ask. A lot of times, you don’t know what you don’t know.”
