Overcoming barriers to machine learning applications in toxicity prediction

KEYWORDS:

• Advanced toxicity prediction
• AI
• machine learning
• deep learning
• drug toxicity

1. Introduction

Traditional approaches for assessing drug toxicity have primarily relied on animal testing and in vitro studies. These methods, while foundational, bring complex methodological challenges, accompanied by ethical considerations, significant expenditures, and lengthy timelines. However, Machine Learning (ML), a subset of Artificial Intelligence (AI), has surfaced as a revolutionary tool, catalyzing advancements not only in drug toxicity prediction but also in every aspect of drug discovery. It enables computers to glean insights and make informed decisions from voluminous datasets without explicit programming [1,2].

Machine Learning (ML), in terms of drug toxicity, refers to computational methods which use patterns extracted from vast datasets to predict harmful effects of substances based on predictions gleaned from them [3]. Toxicity Prediction involves the process of identifying potentially toxic effects of compounds, and understanding Endocrine Disrupting Chemicals (EDCs) is key to grasping the significance of Machine Learning within this field. Drug toxicity prediction cannot be overstated: it serves as an essential checkpoint in drug development pipeline, helping ensure only safe and efficacious medicines reach the market. Accurate toxicity predictions protect public health by keeping harmful drugs out of circulation while sparing pharmaceutical companies from costly late-stage failures. Regulatory authorities rely on accurate toxicity data when making informed decisions about drug approvals that ensure public safety is never compromised.

ML has distinctly refined toxicity prediction strategies, demonstrated notably by the innovative work of Ting Li et al. [4], where deep learning was utilized to formulate a model uniquely designed for carcinogenicity prediction. This model surpassed four comparable advanced models, demonstrating a 37% average improvement rate on test sets from a national database. ML models have illustrated their proficiency, displaying remarkable predictive accuracy in various domains such as oral toxicity [5], cardiotoxicity [6], hepatotoxicity [7], and respiratory toxicity [8]. Additionally, these models have demonstrated unparalleled specificity and sensitivity in predicting eye irritation and corrosion [9]. Moreover, ML has emerged as crucial in precisely assessing chemicals for their risk as Endocrine Disrupting Chemicals (EDCs) [10].

However, the incorporation of ML within toxicity prediction paradigms is punctuated with inherent challenges, predominantly the paucity of high-caliber data and the intrinsic complexity and potential biases of ML models, impacting the reliability and interpretability of resultant predictions [1]. There is a notable scarcity of studies focusing on surmounting these intrinsic barriers, amplifying the difficulty of applying machine learning (ML) techniques in drug toxicity predictions.

This editorial aims to explore strategies for overcoming the barriers and challenges inherent in applying ML to toxicity prediction. It intends to meticulously explore the aforementioned barriers and proffer practical mitigation strategies, aspiring to facilitate more informed, ethical, and rapid advancements in the field of drug development and toxicity prediction.

2. Barriers to ML application in toxicity prediction

Despite the potential of ML, numerous barriers that prevent it from reaching its full potential persist. These challenges can be systematically categorized into key components of machine learning: data, model construction, generalizability, interpretability, and tool-related barriers (Figure 1).

Figure 1. Barriers and mitigation strategies to ML in toxicity prediction.

2.1. Data-related barriers

The limited availability and varying quality of toxicity data pose challenges in developing optimal prediction models. This variation can be attributed to the varied experimental approaches employed to address the diverse toxicity data points, such as cardiogenic toxicity, hepatic toxicity, toxicity due to pharmacokinetic properties, and idiosyncratic reactions [11].

Data imbalance is another common issue in toxicity datasets like Tox21 and ToxCast. These datasets are characterized by an overrepresentation of nontoxic compounds. Such imbalance has resulted in models being biased toward predicting compounds as nontoxic, posing a significant risk to human health by overlooking potentially harmful chemicals. Moreover, this could affect regulatory decisions, potentially resulting in prolonged public exposure to toxic substances and associated long-term health impacts. Moreover, Over-reliance on molecular descriptors, such as physicochemical properties and structural features, for representing toxicity data can be foundational for predictions. Structural features identify toxicophores, and physicochemical properties elucidate ADME (Absorption, Distribution, Metabolism, and Excretion) characteristics. However, they may fall short in capturing the biological interactions and dynamics of in vivo environments [11]. This can lead to prediction errors, particularly in cases of biotransformation that produce toxic metabolites or when toxicity stems from interactions with the biological system.

Additionally, High feature count can cause challenges such as overfitting, increased computational demand, and data sparsity. This phenomenon, where high data dimensionality negatively impacts model performance, is termed the ‘curse of dimensionality.’

2.2. Models-related barriers

In toxicity prediction, traditional models like decision trees and support vector machines differ from advanced deep learning (DL) models in data handling and representations. Traditional models rely on manually selected weighted features, making them unable to capture intricate toxicological patterns and interactions. In contrast, DL models process data hierarchically, allowing them to identify subtle toxicological patterns. DL can also generate task-specific features, representing toxicophores that are crucial for toxicity predictions. While traditional models offer simplicity, they fall short in accuracy for complex toxicological scenarios. DL models, offering enhanced accuracy, face challenges due to high computational demands and complexity.

Recent advancements in toxicity prediction include graph-based learning approaches. These methods are exemplified in the DeepTox pipeline [12], efficiently enhancing toxicity data representations, and demonstrating notable improvements in predictive efficiency. However, achieving model robustness remains a challenge. This is evident in ‘toxicity cliffs’ scenarios. Such cases are well illustrated by a study on substituted phenols, where minor molecular modifications like fluoro- or bromo-substitution led to significantly different toxicity levels [13,14]. Some models might not be adept enough to pinpoint these nuanced structural changes, resulting in marked variances in their toxicity predictions.

2.3. Generalizability-related barriers

Generalizability in machine learning models for toxicity prediction is challenged by the diversity of toxicity endpoints and the complexity of in vivo biological systems. These endpoints provide an evaluation of the compounds’ toxicity profiles and underscore the varied interactions within biological systems. Typically, models are developed for specific endpoints, each requiring unique data parameters, Consequently, a model effective for one type of toxicity might not perform reliably for another.

Moreover, Models built based on descriptors do not capture the full intricacy of biological systems, thereby compromising prediction generalizability. This is further highlighted by how toxicity may be altered by the dynamic nature of the biological systems. For instance, new enzymatic pathways can emerge within these systems, leading to alterations in the metabolism and toxicity profiles of substances. Additionally, the ability of compounds to interact with multiple biological targets may result in a range of diverse toxicological responses.

2.4. Interpretability-related barriers

Accuracy in ML models is crucial, but so is the transparency of their predictions. Often operating as ‘black boxes’, especially in DL models with complex architectures. These models obscure the reasoning behind their outcomes due to their complex architectures, which involve multiple layers of interconnected nodes that autonomously select features for predictions. The inner workings of these models remain opaque even to their developers. In predictive toxicology, understanding the rationale behind toxicity predictions is crucial, where decisions have far-reaching implications for drug development and public health.

Without clear interpretability, researchers might struggle to answer crucial questions like, ‘To what degree is this compound toxic?’ or ‘Is the detected toxicity a valid reason to exclude a compound, or does it still qualify as a potential hit?.’ This information is vital for toxicologists and chemists, particularly when the compound in question is a promising lead in drug development. A clear understanding of the model’s rationale could guide modifications to the compound to mitigate toxicity while retaining its therapeutic properties. Moreover, in regulatory settings, the inability to interpret model predictions for toxicity can result in overly cautious decisions, potentially overlooking effective compounds.

2.5. Tool-related barriers

The choice between open-access and commercial tools is pivotal in toxicity prediction. Open-access tools offer wide accessibility but often lack user-friendly services and advanced predictive features, which compromises their usability, reliability, and predictability. Conversely, commercial tools are powerful and user-centric, However, they come with higher costs and potential issues in transparency. This choice significantly influences drug development efficiency and safety [15]. The key challenge is finding a balance between the accessibility of open-access tools and the precision of commercial tools, essential for optimizing the drug discovery process and reducing the risk of late-stage drug withdrawals [16].

3. Current strategies to mitigate challenges of ML in toxicity prediction

Current strategies that address the shortcomings in toxicity predictions focus on refining toxicity data representations, employing recent modeling techniques, and enhancing interpretability.

Cavasotto and Scardino [17] suggested incorporating Adverse Outcome Pathway (AOP) data to map causal links from molecular events to adverse outcomes, providing a clear framework to understand the mechanisms underlying toxicological effects. A practical application of this is seen in the development of the AOP-Wiki, an interactive tool that offers a repository of AOPs to be utilized in toxicity predictions. Similarly, Wu and Wang [18] propose including genetic information as training features to provide a richer biological context, which enables more toxicity data representation. A notable example is the use of the Toxicogenomics Project-Genomics Assisted Toxicity Evaluation system (TG-GATEs), a comprehensive database that provides genomic biomarkers related to drug-induced liver injury.

To address ML toxicity prediction data imbalance, techniques such as Random Undersampling (RUS) and Synthetic Minority Over-sampling Technique (SMOTE) are used. RUS removes majority class samples, risking data loss, while SMOTE improves class balance but may not accurately represent real-world data. A refined approach combines SMOTE with the Edited Nearest Neighbor (ENN) algorithm (SMOTE+ENN) to remove inconsistent samples. Additionally, conformal prediction frameworks [17] offer an advanced solution, effectively identifying minority class instances despite a higher false-positive rate.

Transfer learning and multitask learning have recently advanced toxicity prediction accuracy. A deep learning framework utilizing molecular features and Simplified Molecular Input Line Entry System (SMILES) embeddings has been developed to model in vitro, in vivo, and clinical toxicity data concurrently [19]. This approach leverages transfer learning, enabling a model trained on one type of toxicity data to adapt for other types, thus improving their prediction accuracies. Moreover, it incorporates contrastive molecular explanations, clarifying the impact of specific molecular features on toxicity predictions and improving the models’ interpretability. For improved interpretability, response maps [20] are applied to Deep Neural Networks (DNN) models. This method adjusts chemical structures and assesses how these changes affect toxicity predictions, visually correlating structural modifications with predicted toxicity and deepening the understanding of molecular toxic effects.

To sum up, ongoing research aims to overcome current challenges in toxicity prediction to help scientists eliminate potentially toxic compounds from the early stages of the drug discovery process with more scientifically robust and measurable outcomes.

4. Expert opinion

Machine learning introduces a paradigm shift in toxicity prediction, diverging from the expensive and time-consuming conventional methods. However, ML in toxicity prediction is riddled with several challenges. Our editorial aims to shed light on the myriad barriers encountered during the development of machine learning models for toxicity prediction. Addressing these barriers and suggesting practical mitigation strategies can streamline the model-building process, enhance model performance, guide toxicologists more effectively, and ultimately boost drug discovery campaigns by maximizing hit rates and minimizing associated costs.

This editorial shows that while advancements are being made, there is a significant disparity between the challenges faced and the strategies currently in place to address them. The current literature primarily focuses on the pillar of toxicity data, emphasizing the integration of biological and genetic data, and the adoption of balancing techniques and frameworks to enhance accuracy and reduce bias. However, overlooking several other crucial pillars that need more attention, such as the need for models that strike a balance between simplicity and superior performance, the expansion of chemical coverage and generalizability in toxicity models, and the imperative for models that not only predict but also elucidate and guide.

Toxicity data quality can be enhanced by bridging the gap between in vitro and in vivo data, especially by incorporating Physiologically-Based Pharmacokinetic (PBPK) data. This integration translate in vitro toxicity results into real-world in vivo scenarios and elucidates the mechanistic pathways drugs follow within the body. Similarly, Establishing feedback loops with experimentalists for timely data updates can also refine ML model training. Moreover, techniques like segmenting toxicity data into distinct categories like toxicokinetics data and toxicodynamics data enriches training data, improving model accuracy and interpretability for better drug discovery decision-making. Moreover, given the sparsity of features in toxicity prediction, open-source packages like Featurewiz (https://github.com/AutoViML/featurewiz) and xverse (https://pypi.org/project/xverse/) can be useful in extracting and prioritizing the most essential features, thereby optimizing model performance.

Looking forward, hybrid approaches that combine conventional and deep learning techniques, alongside automated ML frameworks [21], hold promise in revolutionizing toxicity prediction. Hybrid models adeptly combine the interpretability of traditional methods with the advanced pattern recognition of deep learning. Automated ML streamlines the selection of optimal models, parameters, and data handling techniques, which enables for improved predictions accuracies and accessibility.

Furthermore, the potential of Large Language Models (LLM) in toxicity prediction is noteworthy. Their advanced pattern recognition and natural language processing capabilities could markedly improve both the accuracy and interpretability of toxicity predictions. Machine learning serves as a key driver in enhancing toxicity prediction, especially through its role in digitalization. This digital transformation, as underscored by the integration of technologies like blockchain, is revolutionizing toxicity prediction by enhancing data security, traceability, and transparency. Eventually, these are pivotal in advancing risk assessment and public health [22].

Interpretability, however, continues to be a challenge in toxicity prediction. Explainable AI Tools like GNNExplainer [23], and LIME [24], offer promising solutions to this barrier. GNNExplainer enhances understanding of graph neural network decisions by isolating key graph structures and node features that influence the model’s output. In toxicity predictions, this tool helps in elucidating toxicophores and deciphering the molecular patterns or interactions responsible for a substance’s toxic classification. Similarly, LIME (Local Interpretable Model-agnostic Explanations) offers interpretability by generating simple, local surrogate models that approximate the predictions of complex machine learning models. It achieves this by perturbing input data and monitoring the changes in predictions. It may be used to offer clear explanations and insights into toxicity predictions. These tools enhance transparency in AI-driven toxicity assessments, significantly contributing to more informed drug discovery initiatives

5. Conclusion

There are several challenges facing ML in drug toxicity prediction, which are related to data, model construction, generalizability, interpretability, and accessibility. While the current literature offers limited insights to address these barriers, this editorial emphasizes the need to prioritize data quality enhancement, integrate biological data, and employ advanced modeling techniques. Ultimately, such advancements can revolutionize the landscape of drug discovery, guiding toxicologists more effectively, and ensuring more efficient outcomes and a more cost-effective approach. This not only benefits the pharmaceutical industry but also ensures safer and more affordable medications for patients worldwide

Declaration of interest

The authors have no relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript. This includes employment, consultancies, honoraria, stock ownership or options, expert testimony, grants or patents received or pending, or royalties.

Reviewer disclosures

Peer reviewers on this manuscript have no relevant financial or other relationships to disclose.

Acknowledgments

We would like to thank Professor. Mohammad A. Ghattas for his contributions to reviewing the manuscript and providing comments that have improved both the scope and quality of the work.

Additional information

Funding

This paper was not funded.