https://orcid.org/0009-0001-6350-9387
Cohen’s (Citation2023) mapping exercise of possible bioethical issues emerging from the use of ChatGPT in medicine provides an informative, useful, and thought-provoking trigger for discussions of AI ethics in health. Yet, he acknowledges that it is not exhaustive. Cohen’s analysis carefully considers principles such as privacy and bias prevention, but it does not delve into areas such as explainability, responsibility, or accountability, which are essential to explore. In this commentary we build on Cohen’s foundation, adding implications that stem from a distinctive feature of ChatGPT that differentiates it from conventional medical AI tools.
In the context of our ongoing research under the NIH-funded Bridge2AI program, we are spearheading the first comprehensive scoping review of ethical and trustworthy medical AI design, employing a bioethical lens to focus on value-based aspects (Victor, Salem, and Bélisle-Pipon Citation2023). Our review spans a broad range of literature, revealing the core moral values that underscore medical AI development. This review enables us to inform a “value-sensitive design” approach to medical AI. Our work validates Cohen’s arguments regarding ChatGPT raising familiar bioethical issues such as bias and privacy, but also brings to light uncharted ethical challenges arising from ChatGPT’s status as a general AI model.
CHATGPT’S GENERAL DESIGN AS RAISING UNIQUE ISSUES
One of ChatGPT's distinguishing features, emphasized by Cohen, is its generality. Without being a “General AI” or “Artificial General Intelligence” (AGI), although there seems to be a strong will to work toward getting there, ChatGPT (and generative AI) are largely capable of action beyond “narrow AI” that are designed for one particular task—such as “chess playing” or “medical diagnosis” (Pennachin and Goertzel Citation2007). ChatGPT’s generality allows for being used in just about every domain. It is the Swiss Army Knife of AI, capable of much without excelling at tasks like specialized AIs can do. But the great thing about Swiss Army Knifes are their accessibility, versatility, and the fact that they can be used by most people. We argue that ChatGPT’s general design distinguishes it as a novel form of AI within healthcare contexts, thereby introducing several new bioethical challenges.
One new challenge arising from ChatGPT’s general character is exemplified in Cohen’s discussion of bias in medical AI. Cohen uses an example where an algorithm was found to prioritize treatment for white patients over equally sick black patients. Researchers were able to pinpoint the parameter that led to this biased outcome—the algorithm’s reliance on treatment costs as a proxy for health needs. However, due to ChatGPT’s intricate architecture and its billion-plus parameters, which are a by-product of its general design, identifying and mitigating bias becomes significantly more complex when compared to traditional medical AI.
Another example concerns privacy. Because healthcare contexts have robust privacy standards, AI designed specifically for such contexts takes such expected privacy standards into account. How widely available algorithms like ChatGPT will meet privacy requirements when used in medical contexts is a new ethical and technical question. Examples like these illustrate that established bioethical challenges emerge from the use of ChatGPT, as well as new concerns that are yet to be discussed. These are just a few examples of how ChatGPT is redefining traditional bioethical issues and challenging existing ways of tackling them.
MOST DISCUSSED VALUES
What follow is a list of three key values and concomitant ethical concerns found through our review, each contributing to Cohen’s list of familiar concerns applied to ChatGPT.
Our review identified transparency (along with related values, interpretability and explainibility) as a recurring and central moral value in discussions of medical AI ethics. Papers argue that patients deserve to understand the functioning of the technology involved in their care for reasons such as informed consent (Strikwerda et al. Citation2022), and that transparency promotes patient autonomy and trust. Furthermore, they argue that transparency plays a pivotal role in determining an algorithm’s ethical standing, as opacity can lead to concealed biases (Alami et al. Citation2020). Transparency with generative AI is challenging, as different domains require varying transparency levels. For example, those asking ChatGPT for a recipe will likely care less about transparency than those who are getting a medical diagnosis from ChatGPT. Balancing transparency against competing values such as effectiveness differs across domains of application and is a key challenge for bioethics.
Our sources also consistently highlighted trust as a major value. As the use of AI in healthcare requires patients to accept AI-based analyses, diagnoses, prognoses, and treatment recommendations, and given the current unfamiliarity of AI tools in medical contexts, trust becomes a vital aspect. The heightened possible vulnerability of patients during medical care underscores the need for trust. Trust forms the foundation for debates about transparency, explainability, safety, accuracy, privacy, respect for autonomy, and more. These prerequisites for trust apply to ChatGPT as well, which might encounter trust-related barriers due to its design for general use. For example, stakeholders will need to know that ChatGPT is medically accurate to trust it. Interestingly, trust in ChatGPT for healthcare may increase by the mere fact that it becomes a prevalent tool in other contexts.
Finally, our review found that responsibility, and the associated values of liability and accountability, are frequently mentioned as underlying the ethically sound use of AI in medicine. The attribution of moral or even legal responsibility for ethical missteps remains a contentious issue. If, for instance, ChatGPT provides medical advice that results in a fatality, who should shoulder the blame and be accountable? Should culpability apply at the individual level (e.g., patient, healthcare provider, or AI's developer) or at a systemic level (such as hospitals or healthcare systems that integrated ChatGPT into their practice, or the corporations that developed it)? Lack of clarity regarding accountability might lead to a lax approach toward responsible development and use of ChatGPT. This could result in catastrophic situations where harm is inflicted without clear fault attribution and capacity to remedy the fault.
LEAST DISCUSSED VALUES
While commonly mentioned values that have been extensively considered in the past allow for important bioethical discussions, underexplored values are also worthy of particular focus moving forward as they reveal overlooked ethical dimensions of medical AI. Some of the least discussed but nevertheless crucial values identified in our review include causability, certifiability, contestability and (in)compatibility of machine versus human judgment. These values highlight the need for AI systems like ChatGPT to not only generate results, but also to explain the reasoning behind them (causability), to ensure their algorithms and outputs can be verified (certifiability), to allow their decisions to be challenged (contestability), and to align with human logic and reasoning (compatibility).
These moral values are currently underrepresented in the literature yet can provide bioethicists with a rich lens for the analysis of ethical implications of deploying generative AI in medicine. The integration of these moral values within a bioethics framework can significantly aid in addressing concerns related to trust, responsibility, transparency, and the overall impact of generative AI in the healthcare landscape.
LOOKING AHEAD
Current speculation alone regarding ChatGPT’s known and potential bioethical implications can only provide limited insights. Empirical research (e.g., surveys, interviews) about how values underlie the use of medical AI is a first step to highlight and contrast relevant values to AI design and implementation. But to truly understand whether ChatGPT, and even more potent AI, introduces new moral and existential risks, it is necessary to study its real-world deployment.
Equally critical is the willingness of AI designers to incorporate value-based findings into future iterations of their technology. Despite the far-reaching impacts of AI technologies like ChatGPT, there is a dearth of research on the concerns of its stakeholders, including users and those affected by its use. This gap could be attributed to the profound impact of these technologies, which touch on virtually every aspect of life for a vast majority of individuals. AI calls into question not only oversight mechanisms, but also the moral foundations that guide AI development, implementation, and use in medicine, placing this technology in a rather exceptional position (Bélisle-Pipon et al. Citation2021), this is all the more true for generative AI and its enhanced capabilities and accessibility. Bioethics research ought to meticulously examine the implementation processes of generative AI, its consequential effects on stakeholders, how these effects contrast from the impacts induced by prior health technologies, but also its impacts on values that should guide the work and lens of bioethics.
Cohen asks us to reflect on the question: “What should ChatGPT mean for bioethics?” As bioethicists grapple with the rapid evolution of AI in healthcare, ChatGPT emerges as a unique entity demanding distinct bioethical consideration. Its general design and wide-ranging applicability underscore a need for a more comprehensive, values-based perspective. By recognizing and addressing the multitude of moral values linked to its application, bioethics can provide a crucial roadmap to navigating the bioethical dilemmas posed by AI like ChatGPT (Bélisle-Pipon, Ravitsky, and Bensoussan Citation2023). The profound understanding of these values should not only inform the creation of such technology, but also guide its use and regulation, fostering a future of AI in healthcare that is equitable, accountable, and truly beneficial to patients.
DISCLOSURE STATEMENT
No potential conflict of interest was reported by the author(s).
Additional information
Funding
REFERENCES
- Alami, H., P. Lehoux, Y. Auclair, M. de Guise, M.-P. Gagnon, J. Shaw, D. Roy, R. Fleet, M. A. A. Ahmed, and J.-P. Fortin. 2020. Artificial intelligence and health technology assessment: Anticipating a new level of complexity. Journal of Medical Internet Research 22 (7):e17707. doi:10.2196/17707.
- Bélisle-Pipon, J.-C., V. Couture, M.-C. Roy, I. Ganache, M. Goetghebeur, and I. G. Cohen. 2021. What makes artificial intelligence exceptional in health technology assessment? Frontiers in Artificial Intelligence 4:736697. doi:10.3389/frai.2021.736697.
- Bélisle-Pipon, J.-C., V. Ravitsky, and Y. Bensoussan. 2023. Individuals and (Synthetic) data points: Using value-sensitive design to foster ethical deliberations on epistemic transitions. The American Journal of Bioethics 23 (9):69–72. doi:10.1080/15265161.2023.2237436.
- Cohen, I. G. 2023. What should ChatGPT mean for bioethics? The American Journal of Bioethics 23 (10):8–16. doi:10.1080/15265161.2023.2233357.
- Pennachin, C., and B. Goertzel. 2007. Contemporary approaches to artificial general intelligence. In Artificial General Intelligence. Cognitive Technologies, ed. B. Goertzel & C. Pennachin, 1–30. Berlin, Heidelberg: Springer. doi:10.1007/978-3-540-68677-4_1.
- Strikwerda, L., M. van Steenbergen, A. van Gorp, C. Timmers, and J. van Grondelle. 2022. The value sensitive design of a preventive health check app. Ethics and Information Technology 24 (3). doi:10.1007/s10676-022-09662-x.
- Victor, G., S. Salem, and J.-C. Bélisle-Pipon. 2023. A scoping review of relevant moral values in health sector AI development. OSF Registry. doi:10.17605/OSF.IO/BCVK3.