Explainable artificial intelligence and machine learning: novel approaches to face infectious diseases challenges

Abstract

Artificial intelligence (AI) and machine learning (ML) are revolutionizing human activities in various fields, with medicine and infectious diseases being not exempt from their rapid and exponential growth. Furthermore, the field of explainable AI and ML has gained particular relevance and is attracting increasing interest. Infectious diseases have already started to benefit from explainable AI/ML models. For example, they have been employed or proposed to better understand complex models aimed at improving the diagnosis and management of coronavirus disease 2019, in the field of antimicrobial resistance prediction and in quantum vaccine algorithms. Although some issues concerning the dichotomy between explainability and interpretability still require careful attention, an in-depth understanding of how complex AI/ML models arrive at their predictions or recommendations is becoming increasingly essential to properly face the growing challenges of infectious diseases in the present century.

KEY MESSAGES

• AI and ML are revolutionizing human activities in various fields, and infectious diseases are not exempt from their rapid and exponential growth.

• Despite some notable challenges, explainable AI/ML could provide insights into the decision-making process, making the outcomes of models more transparent.

• Improved transparency can help to build trust among healthcare professionals, policymakers, and the general public in leveraging AI/ML-based systems to face the growing challenges of infectious diseases in the present century.

Keywords:

• Artificial intelligence
• machine learning
• explainability
• interpretability
• deep learning
• infectious diseases

Background

Artificial intelligence (AI), which can be defined as the ability of machines to exhibit cognitive-like tasks, and machine learning (ML), a component of AI encompassing the ability of machines to learn tasks without having been explicitly programmed to do so [1–4], are revolutionizing human activities in various fields, with medicine and infectious diseases being not exempt from their rapid and exponential growth [5–19].

Part of such exponential growth involves the use of complex AI/ML models in which the computations underlying the provided output (e.g. diagnosis of an infectious disease, resistance profiles of etiological agents, therapeutic recommendations, design of vaccine protective antigens) are not clear to either data scientists or physicians. These models are classically referred to as ‘black boxes’ and the lack of clarity regarding how the model exactly predicts from input data (e.g. which features or combination of features are majorly influencing the output) could preclude proper assessment of biases, as well as the correct interpretation of model results [10,20]. All of this requires careful attention in the healthcare field since misleading but still convincing outputs could theoretically and unfavorably impact diagnostic or therapeutic decisions, and, in turn, patients’ health.

Explainable AI and ML to face infectious diseases challenges

The field of explainable AI and ML has gained particular relevance and is attracting increasing interest. Explainability can be defined as the ability to explain how a black box model has produced an output, usually through replicating the prediction with another less accurate but interpretable model. Common examples are local interpretable model-agnostic explanations (LIME) and SHapley additive exPlanations (SHAP) [21,22]. Briefly, LIME generates localized explanations that are centered on individual predictions by means of interpretable models (e.g. a black box model might indicate that a patient has a specific infection, with LIME explaining that the prediction is mostly based on some specific symptoms), whereas SHAP confers importance values upon individual features within a specific prediction [21–23]. Other factors contributing to explainability may rely on the careful selection of input features and the quality of data [20,24,25]. To fully understand the challenges and opportunities of explainability, a distinction should nonetheless be made between interpretability and explainability. A model is interpretable when each step of the computations leading to the eventual prediction and the relative contribution of the different features can be easily inferred from a ‘white box’ model (e.g. logistic regression, which is interpretable through the weights conferred to features in the model equation by β coefficients, or, as defined in ML terminology, parameters), while explainability is an attempt to explain as much as possible how a ‘black box’ model produces its output [24,26]. In a certain sense, interpretability can be imagined as full explainability, whereas when a model is defined as ‘explainable’, readers should bear in mind that the aim of the explanation is to approximate as much as possible interpretability (i.e. full explainability). In this regard, a crucial issue is that the correctness of the explanation could vary greatly, and it may also be difficult to measure or ascertain. For all these reasons, scientists have taken different positions, with some providing insightful arguments in favor of interpretable over explainable models [27].

On the other hand, arguments supporting explainability rely on the advances in the field of explainable AI/ML (indeed, it cannot be excluded a priori that better explanations could be achieved through dedicated research), attention to type and quality of input data, and assessment of the efficacy and safety of an AI/ML interventions/tools by means of randomized controlled trials (RCTs). Indeed, the use or not of the explainable AI/ML models could be randomized to evaluate their efficacy and safety in clinical practice with high certainty evidence, under strict requirements similar to those ultimately leading to the approval of antimicrobials for their use in humans by regulatory agencies. Connected to this latter point, another important challenge worth noting is that the legal and ethical framework regulating the use of explainable AI/ML in medicine and infectious diseases will require continuous updates in order to guarantee constant compliance with the rapid advancements in the field and contemporary law and privacy rules [28–30].

Conclusion

Overall, infectious diseases have already started to benefit from explainable AI/ML. For example, the ability of explaining ML model results has been employed or proposed to better understand the interactions between antimicrobial drugs/peptides and microorganisms, as well as to accelerate the identification of either existing or novel chemical structures with antimicrobial activity [8,31–33]. In modern vaccinology, quantum vaccine ML algorithms have been applied to protein dynamics and proposed for the identification of candidate protective antigens [30–32]. Regarding clinical studies, explainable ML has been exploited, for example, to obtain meaningful insights from models aimed at improving the diagnosis and management of coronavirus disease 2019 or in the attempt to explain the prediction of antimicrobial resistance in clinical practice [34,35]. Against this backdrop, explainable AI/ML could provide insights into the decision-making process, making the outcomes of models more transparent and interpretable. In turn, this transparency can help to build trust among healthcare professionals, policymakers, and the general public, fostering the adoption and acceptance of AI/ML-based systems. For all these reasons, we have launched the article collection ‘Explainable artificial intelligence and machine learning: novel approaches to face infectious diseases challenges’ in Annals of Medicine, which covers the connection across explainable AI, ML, and infectious diseases. We support that a more in-depth understanding of how complex AI/ML algorithms arrive at their predictions or recommendations is nowadays essential to properly face the growing challenges of infectious diseases in the present century.

Authors contributions

DRG: conceptualization, writing of original draft, review, and editing; JF, YZ: conceptualization, review and editing.

Disclosure statement

DRG and JF are section editors at the same journal. Outside the submitted work, DRG reports investigator-initiated grants from Pfizer, Shionogi, and Gilead Italia, and speaker/advisor fees from Pfizer, Menarini, and Tillotts Pharma. The authors have no other conflicts of interest to disclose.

Data availability statement

Data sharing is not applicable to this article as no new data were created or analyzed in this study.

Additional information

Funding

No funding was received.