Machine learning in infectious diseases: potential applications and limitations

Abstract

Infectious diseases are a major threat for human and animal health worldwide. Artificial Intelligence (AI) combined algorithms including Machine Learning and Big Data analytics have emerged as a potential solution to analyse diverse datasets and face challenges posed by infectious diseases. In this commentary we explore the potential applications and limitations of ML to management of infectious disease. It explores challenges in key areas such as outbreak prediction, pathogen identification, drug discovery, and personalized medicine. We propose potential solutions to mitigate these hurdles and applications of ML to identify biomolecules for effective treatment and prevention of infectious diseases. In addition to use of ML for management of infectious diseases, potential applications are based on catastrophic evolution events for the identification of biomolecular targets to reduce risks for infectious diseases and vaccinomics for discovery and characterization of vaccine protective antigens using intelligent Big Data analytics techniques. These considerations set a foundation for developing effective strategies for managing infectious diseases in the future.

KEY MESSAGES

• Infectious diseases are a major challenge worldwide

• Artificial Intelligence (AI) combined algorithms have emerged as a potential solution to analyse diverse datasets and face challenges posed by infectious diseases

• Future directions include applications of ML to identify biomolecules for effective treatment and prevention of infectious diseases

Keywords:

• Artificial intelligence
• Big Data
• infectious diseases
• machine learning
• vaccine

Introduction

The rise of ever-changing infectious diseases, like the recent coronavirus disease 2019 (COVID-19) pandemic, emphasises the urgent need for innovative tools to combat their spread and mitigate their impact. Machine learning (ML), which is part of artificial intelligence (AI) has emerged as a potential solution due to its ability to analyse diverse datasets [1,2].

Many studies in the literature have explored the applications of ML in managing infectious diseases. These applications range from predicting outbreaks and tracing transmission routes to aiding diagnosis and developing treatment and preventive strategies. For example, Park et al. [3] developed a model that combined Deep Learning (DL) with ML models to enhance disease prediction accuracy based on laboratory tests. Their optimized ensemble model achieved a 92% prediction accuracy, showing precision and recall rates for diseases like acute hepatitis B, malaria, meningitis among other [3]. Another study reported that ML showed promising results in predicting poor outcomes of bloodstream infections, with an area under the receiver-operating characteristics curve of 0.82 [4], which supports the potential of ML models in early infection management.

However, it is important to consider that the implementation of ML in infectious disease management comes with its set of challenges [5]. These challenges include the requirement for high-quality data to ensure that the ML models can be applicable across different populations and pathogens. The ML models often face difficulties with overfitting and lacking interpretability. Biased data can result in skewed healthcare outcomes. Integrating these models into existing healthcare workflows poses additional obstacles. Given the evolution of agents and the variability in disease presentation, models can quickly become outdated, thus requiring continuous updates and validation [5,6].

The aim of this commentary is to explore the potential applications and barriers to ML, specifically in the context of infectious disease management. It explores challenges in key areas such as outbreak prediction, pathogen identification, drug discovery, and personalized medicine. Additionally, it proposes potential solutions to mitigate these hurdles and applications of ML. These considerations set a foundation for fair strategies in managing infectious diseases in the future.

From an epidemiological point of view, multiple research projects have been done focused on the prediction and prevention of epidemiological outbreaks using different and out of the box approaches. The ML algorithms have been found to be useful in evaluating outbreak risk of a zone, when diseases such as vector-borne diseases are influenced by environmental or meteorological factors. For example, in the case of malaria, researchers predicted the risk of outbreaks by using meteorological data such as rainfall, maximum and minimum temperatures, and other variables [7]. Similar approaches have been applied combined with socio-economic variables for dengue [8], and COVID-19 [9] allowing authorities to act with preventing measures.

On the prevention line but with a different approach, multiple efforts using social media data have been used to manage transmissible diseases. Tracking the evolution of diseases can be done in real-time by taking advantage of social networks immediacy. Using tools as text mining, social media analysis (SMA) or sentiment analysis is possible to find work on early detection of outbreaks, detection of infected individuals or to predict transmission patterns or risks based on interaction between users [10]. These works have achieved good accuracy by using text as input (mainly from Twitter) and evaluating the presence of some key words (such as the main synonyms). While there are concerns about privacy and usability, it could be a useful tool for precise detection. Also, there is another set of ML tools that have been trained to manage outbreaks that have started or even pandemics. For example, Moulaei et al. [11] developed a tool to predict infected individuals with a higher risk of developing a serious health condition, opening the door to better hospital management in case of saturation. A summary of these studies with more details on the ML models is disclosed in Table 1.

Table 1. Summary of reviewed studies on the use of ML in infectious diseases.

Disease	Topic	Model algorithms	References
Various	Disease classification from biochemical data	Light gradient boosting machine (LightGBM), extreme gradient boosting (XGBoost) and Deepl Neural Network (DNN)	Park et al. [3]
Bloodstream infections	Disease classification from biochemical data and patient clinic history	LightGBM	Zoabi et al. [4]
Malaria	Outbreak prediction from environmental data	Linear regression, Logistic regression, Neural networks, XGBoost, K Nearest Neighbors (KNN), Support Vector Machine (SVM) and Naïve Bayes	Kalipe et al. [7]
Dengue	Outbreak prediction from environmental data	Categorical Boosting (CatBoost), Support Vector Machine (SVM), Long Short-Term Memory (LSTM)	Sebastianelli et al. [8]
COVID-19	Outbreak prediction from environmental data	Convolution Neural Network (CNN), ADtree Classifier and BayesNet	Abdulkareem et al. [9]
Dengue and Flu	Outbreak prediction from social networks data	Random Forest (RF), K-Nearest Neighbor (KNN), Support Vector Machine (SVM), and Decision Tree (DT)	Amin et al. [10]
COVID-19	Disease severity prediction from heterogeneous data	J48 decision tree, random forest (RF), k-nearest neighborhood (k-NN), multi-layer perceptron (MLP), Naïve Bayes (NB), eXtreme gradient boosting (XGBoost), and logistic regression (LR)	Moulaei et al. [11]

Methodology

To ensure a comprehensive understanding of the application of ML in managing infectious diseases, we performed a multi-database search in PubMed (https://pubmed.ncbi.nlm.nih.gov), Scopus (https://www.scopus.com/), and Web of Science (https://clarivate.com/products/scientific-and-academic-research/research-discovery-and-workflow-solutions/webofscience-platform/), complemented by conference proceedings and grey literature to capture a wide range of discussions on ML applications in infectious diseases. The search strategy was built around a combination of keywords related to "machine learning", "infectious diseases", "predictive modelling", "data challenges", and specific terms related to the diseases discussed such as "SARS-CoV-2", "Ebola", and "influenza". Boolean operators (AND, OR) were used to refine the search results and ensure relevance. We established clear inclusion and exclusion criteria to focus on articles that directly address ML applications, challenges, and opportunities within infectious disease management. Data extraction was followed by a critical synthesis of the articles, focusing on identifying common themes, technological gaps, and the integration of diverse scientific perspectives.

Barriers to ML in infectious diseases

The application of ML in infectious diseases management presents several challenges and limitations, which can be categorized into pathogen behaviour, model adaptability, and data management challenges.

First, the unpredictable and dynamic nature of infectious disease outbreaks, including varying transmission dynamics and impacts of interventions, makes modelling with ML complex. This complexity is primarily due to the multifaceted and often unknown variables associated with the emergence and spread of pathogens. Each infectious disease exhibits unique patterns in terms of its transmissibility, virulence, and response to environmental factors and control or preventive interventions. Specifically, the complexity of biological systems presents a significant challenge [12–14]. Infectious diseases are influenced by a myriad of interconnected factors including genetic, environmental, and social variables. This complexity often exceeds the capacity of current ML models, which may not fully capture the non-linear interactions and emergent properties of biological systems. For instance, ML models struggle with the integration of multi-scale data ranging from molecular to epidemiological scales, which is crucial for understanding and predicting disease dynamics.

An example is the emergence of novel pathogens such as SARS-CoV-2 that introduces a high degree of uncertainty. Key characteristics like the incubation period, asymptomatic transmission, and mutation rates initially remain largely unknown, complicating predictive modelling efforts. Furthermore, the effectiveness of public health interventions, which can significantly influence the course of an outbreak, adds another layer of variability. These interventions are often subject to societal behaviours and policy decisions, making their impact challenging to quantify accurately in predictive models [15]. Another example is the Ebola outbreak in West Africa between 2014 and 2016, which posed significant challenges in predictive modelling due to its complex transmission dynamics and varying factors [16]. The sub-exponential growth patterns of Ebola, influenced by factors like social contact networks and cultural practices, differed significantly from diseases with aerosol transmission like influenza, complicating early modelling efforts. Response strategies, such as increasing Ebola Treatment Unit capacities and isolating patients, introduced additional variables that were difficult to incorporate accurately into models due to data limitations [17]. Furthermore, the spatial and temporal analysis of the outbreaks requires understanding both local dynamics and cross-border transmission, challenged by the paucity of precise and timely data [18].

Second, infectious agents like viruses and bacteria, can mutate rapidly, making it difficult for ML models to stay current without constant retraining. For example, the influenza virus, which is notorious for its high mutation rate undergoes antigenic drift each year, a process where small genetic changes accumulate over time. These changes can alter the virus’s surface proteins, hemagglutinin, and neuraminidase, enough that people’s immune systems may not recognize the virus, even if they were previously vaccinated or infected [19]. As a result, ML models used to predict influenza trends or vaccine effectiveness need regular updates to incorporate the latest genetic information about circulating strains. Accordingly, systematically reviewing the literature is required to explore the use of ML techniques to generate predictions of influenza virus phenotypes based on genomic and/or proteomic datasets [16]. The results showed that the ever-changing genetic landscape of the influenza virus poses a substantial challenge to applying ML in influenza management. The rapid evolutionary adaptability of pathogens like viruses adds another layer of difficulty [20]. These organisms can mutate rapidly, sometimes even within the course of an outbreak, thereby altering their behaviour and confounding existing predictive models. This necessitates continuous updates to the models, a process that is both data-intensive and computationally demanding. The constant evolution of pathogens means that historical data might quickly become outdated, reducing the predictive accuracy of ML models unless they are frequently recalibrated with new data.

Third, ML encounters challenges in managing infectious diseases due to data issues. These include data availability for emerging pathogens, inconsistencies in records leading to biases, and variations across regions. These challenges pose a risk of perpetuating healthcare disparities. When it comes to real-time data integration for outbreaks, privacy concerns arise necessitating the implementation of anonymization techniques and governance protocols. The deployment of ML in clinical decision support systems (CDSS) requires a range of diverse data for training and validation purposes, including both structured and unstructured information [5]. Ensuring the inclusivity of data and compatibility with different technology platforms is crucial for disease management. It is essential to address biases, particularly those affecting groups, while also ensuring that clinicians can properly interpret the outputs generated by ML-CDSS systems. Robust plans must be implemented to handle failures and cyber threats, along with validation processes to ensure safety and reliability in public health [5].

In addition to data quality and interpretability challenges in ML for diseases, black-box models may present significant limitations [21]. These models often lack transparency in their outputs like diagnostics or treatment plans, hindering understanding by data scientists and healthcare professionals. This obscurity can lead to misinterpretation or bias oversight, crucial for informed medical decisions.

Lastly, the socio-economic context plays a critical role in the spread and management of infectious diseases, and its variability across different regions and populations introduces additional complexity to modelling efforts [22,23]. Factors such as population density, healthcare infrastructure, public health policies, and community norms can significantly influence disease transmission and the effectiveness of intervention strategies. The ML models often do not adequately account for these diverse and dynamic social determinants of health, leading to potential biases and inaccuracy in predictions.

Opportunities to overcome ML limitations in infectious diseases management

Several review articles have explored the efficacy and limitations of ML techniques in managing infectious diseases. Santangelo et al. [12] demonstrated the potential of ML to predict infectious disease outbreaks by combining various techniques to achieve accurate and plausible results. The study emphasized the need for high-quality data to enhance prediction capabilities. Another study provided an overview of how supervised ML techniques are increasingly applied in laboratory medicine for diagnosing various infectious diseases such as COVID-19, sepsis, and tuberculosis, and stressed the utility of these techniques in improving diagnostic accuracy and efficiency [24]. A systematic review and meta-analysis analyzed the efficacy of ML models in predicting sepsis onset in intensive care units [25]. Their work detailed how models like Random Forest and XGBoost can use clinical predictors such as age, creatinine levels, and vital signs to forecast sepsis onset, thereby potentially improving patient outcomes through early intervention.

To deal with the challenges posed by nature of pathogens and its outbreaks, it is crucial to develop adaptive models that can learn and update as data becomes available. An example of these models is a study by Bhadriraju and colleagues [26], which introduced an adaptive model identification framework that uses sparse regression and feature selection to understand and predict the dynamics of complex processes with smaller data sets. They proposed a three-step procedure involving Sparse Identification of Nonlinear Dynamics (SINDy), ordinary least-squares regression and stepwise regression, demonstrating its effectiveness by comparing the modelling of a continuous stirred tank reactor with traditional SINDy methods [26].

Additionally, using domain knowledge to incorporate information about similar pathogens is a promising strategy. A recent study developed a mixed-effect ML model to predict gene editing CRISPR interference (CRISPRi) guide efficiency in Escherichia coli using integrated data from multiple gene target screens [27]. The model, which combines a linear random-effect model with a fixed-effect random forest, effectively separates the influence of the targeted gene from guide efficiency. The method applied in their work is insightful in situations where direct measurements are challenging to obtain.

Furthermore, ML models should be flexible to account for varying transmission patterns and intervention scenarios. A team of researchers from the United States developed a method that combines a Bayesian time series model and a random forest algorithm within a compartmental framework [28]. This approach provided data-driven predictions for COVID-19. They evaluated the model by extending the training period throughout the pandemic and assessing its accuracy in forecasting over 21-day periods. Interestingly, the model revealed variations in projection trajectories and uncertainties across states, as observed in New York, Colorado, and West Virginia [28].

To address the challenges posed by mutating agents and the lack of transparency in "black box" models used for disease prediction, various advanced methodologies are employed. Firstly, continuous learning approaches play a role as they allow models to automatically integrate data, ensuring that predictions remain relevant even with ever-changing pathogens [29,30]. This is particularly important when dealing with viruses that mutate quickly since traditional static models become outdated rapidly. Secondly, explainable AI (XAI) techniques are used to enhance transparency in model decision-making processes. The XAI enables an understanding of how models arrive at their decisions, which helps build trust and identify biases in algorithms [30]. It plays a role in unravelling AI processes, making them accessible to a broader range of users, including healthcare professionals and policymakers. The main hurdle and significant challenges lie in harmonizing advancements with the practical aspects of healthcare, protecting data privacy, securing consent, and ensuring access to these advanced tools.

Soon, we envision progress in the field moving towards integrated and adaptable systems. There will likely be a transition towards developing models that are not only accurate but also understandable and transparent. This shift will foster trust and enhance usability among healthcare professionals. Additionally, we expect advancements in real-time data analysis and continuous learning models, enabling responses to emerging infectious diseases.

An area of research that currently holds intrigue is the use of ML in personalised medicine within the field of infectious diseases. This approach tailors treatment and prevention strategies based on patient profiles, potentially leading to better outcomes in infectious disease management. The incorporation of data with ML algorithms presents an avenue for developing such personalized strategies. However, it is imperative to address the logistical challenges associated with handling sensitive data.

Potential applications

Catastrophic evolution: biomolecular targets to reduce risks for infectious diseases

Catastrophic selection is a phenomenon associated with organisms resistant to pathogens and other stringent environments due to some deviant genome signatures and physiological differences [31,32]. Transposable sequences of viral origin or endogenous retroviruses (ERVs) are dormant in the modern human genome and when activated may regulate transcription contributing to innate immunity, cellular senescence, and tissue aging [33,34]. The hypothesis and recent evidence support that humans evolved with catastrophic selection associated with pathogen infection, which resulted in genomic signatures leading to biomolecular targets (proteins and post-translational modifications) to reduce risks for infectious diseases [35]. To address this hypothesis, it is possible to apply comparative genomics to study human genome in comparison with Old-World monkeys and other species to identify inactivated genes that may be related to evolutionary adaptations to pathogen infection. For example, the major histocompatibility complex (MHC) is a chromosome 6 locus containing polymorphic genes that code for cell surface proteins essential for adaptive immunity. Comparative genomic analysis of MHC and other regions between sympatric host species might shed light on the adaptive immune system dynamics. Using ML algorithms and disease modelling it may be possible to find driving factors for host-pathogen interactions and disease susceptibility. Then, using a systems biology approach for analysis of omics datasets (transcriptomics, proteomics, metabolomics, interactomics, regulomics) to characterize molecular pathways and biomolecules associated with the identified inactivated genes, and applying ML algorithms to predict which identified biomolecular targets are able to interfere with pathogen infection from a biomedical perspective may result in candidate biomolecules for possible interventions to reduce risks for infectious diseases. Finally, considering that biomolecules do not act in isolation but interacting with each other forming a network collectively known as interactome, sub-networks of genes and proteins that interfere with pathogen infection are prime candidates for therapeutic target selection. For this, we may use both local and global network topological properties of genes/proteins such as degree and betweenness centrality as input features and explore graph representation learning with measures of success including interference with immune response pathways, metabolic pathways, and known action of pharmaceuticals for the prevention and control of infectious diseases. However, possible negative trade-off effects of biomolecules associated with catastrophic selection in some individuals should be considered and included in ML algorithms [31].

Vaccinomics: high throughput discovery and characterization of vaccine protective antigens using intelligent Big Data analytics techniques

Vaccinomics is defined as the use of genome-scale omics technologies together with bioinformatics to have a holist approach for the development of next-generation vaccines [36–38]. Regarding the use of AI and ML algorithms, one of the most promising applications is the prediction of vaccine efficacy and effectiveness based on the identification of the best protective antigens [38]. However, the main obstacles include the lack of reliability on ML to use these predictive models on complex real cases and data composed of difficult to obtain features. The lack of reliability is based on two inherent aspects of this technology that increases with the more high-dimensional the data are. Although ML models have been validated and there are various metrics we can use, there is not an individual measure of certainty for each output. Some of the most used metrics are Mean Absolute Error (MAE), True/False negative/positive rates and derived metrics as the Area Under the Receiver Operating Characteristic curve (AUC-ROC CURVE), the recall or the precision. We must use these different indicators and interpret them in order to determine the quality of the model, but we cannot determine the interval of confidence of each individual prediction [39]. Another problem is understanding the processes or decisions made by the algorithm to get the predictions [40]. Second, the immuno­logical datasets are based on laboratory experimentation and as previously mentioned the effect of evolutionary mechanisms make ML datasets difficult to keep updated due to modifications in pathogen and vector derived antigens and associated host adaptations thus compromising the flexibility of the model. Approaching these limitations requires the combination of different algorithms and Big Data analytics such as the combination of a supervised ML model as Random Forest (RF) with other non-supervised models such as Hierarchical Clustering (HCA) and Fuzzy Deformable Prototypes (FDP) to overcome the confidence and data complexity problems [38]. The HC has been used as a step prior to RF training to reduce features in very complex and high dimensional datasets [41,42]. Additionally, and despite its limitations [30], black-box models have been used to interpret the predictions of complex ML algorithms such as deducing the patterns learned by deep neural networks to understand how the algorithm works once trained and to detect biases [43]. The HC could also be applied to make clusters of the different individual trees used to make the predictions and choose one of these trees to graphically represent the decision-making process in a simplified way and considering not only the predicted variable but also the variables used as inputs [44]. The identification of vaccine antigens is a good example of FDP as there is a complex question in which it is difficult to choose a perfect prototype for a vaccine candidate considering that it is defined not as an element but as a group of good, mediocre, and bad elements of a category, being less intuitive but more accurate to reality [45]. Accordingly, FDP could be used to facilitate prediction of antigen feasibility as a vaccine candidate and to extend the response variables predicted by the supervised method to obtain a more complex profile of the features of predicted interest. In summary, the combination of ML algorithms may lead to a robust Big Data method that allows us to predict the performance of antigens as vaccine candidates and considering data uncertainty and vagueness.

Additional applications. The potential applications discussed above are examples of ML approaches to infectious diseases. However, other potential applications including RNA interference-mediated gene control [46], structure-based drug design [47], and therapeutic interventions against drug resistance [48] are under development with possible impact on the prevention and control of infectious and non-infectious diseases.

Conclusions

To advance in ML and AI in infectious diseases it is important to approach challenges with innovative solutions (Figure 1). The application of ML for the management of infectious diseases holds promise but also faces significant challenges, such as the rapid mutation of pathogens, biases in data, and concerns about privacy. To overcome these challenges, it is crucial to adopt learning approaches, use interpretable models, and focus on inclusive data practices. Future directions include applications of ML to identify biomolecules for effective treatment and prevention of infectious diseases.

Figure 1. Challenges to ML in infectious diseases and potential approaches and applications.

Authors contributions

Authors A.Z. Al Meslamani and J. de la Fuente: conception and design. Authors A.Z. Al Meslamani. I. Sobrino and J. de la Fuente: analysis and interpretation of the data. Authors A.Z. Al Meslamani. I. Sobrino and J. de la Fuente: drafting of the paper. Author J. de la Fuente: revising it critically for intellectual content; and the final approval of the version to be published. All authors agree to be accountable for all aspects of the work.

Acknowledgments

The authors thank Margarita Villar and Marinela Contreras (Genomics, Proteomics and Biotechnology at SaBio, IREC), Agustin Estrada-Peña (University of Zaragoza) and José A. Olivas (Technologies and Information Systems Institute, University of Castilla-La Mancha) for their contribution to our initiatives on ML applications.

Disclosure statement

The author J. de la Fuente is section editor in the same journal. We declare no other competing interests.

Data availability statement

Data sharing is not applicable to this article as no new data were created or analyzed in this study.

Additional information

Funding

The authors did not receive funding for this study.