How far have decision tree models come for data mining in drug discovery?

KEYWORDS:

• QSAR
• decision tree induction
• random forest
• decision forest
• deep learning

1. Introduction

Machine learning (ML) methods assist in drug discovery mostly by way of data mining in virtual screening (VS). If the target is sufficiently characterized, say by knowledge of its three-dimensional (3D) structure or gene sequence, we can take a structure-based VS (SBVS) approach and run molecular docking and dynamics, or 3D-similarity matching experiments. More often, however, we only know a set of molecular structures and their biological activities, and so we may perform a ligand-based VS (LBVS) [1]. The results of an LBVS, which are computationally much less expensive to obtain than those of an SBVS, can be the basis of chemical database queries and, optimally, at an early stage of drug discovery enhance our understanding of how a molecule’s action may come about (hypothesis generation). The concept of decision trees is well suited for this.

2. Divide and conquer

The decision tree induction (DTI) paradigm is one of the staples of ML. By splitting a dataset into increasingly homogenous subsets through rule-based decisions (a ‘divide and conquer’ approach), it acts as a classifier that not only mimics human decision-making but that can also be easily visualized in the form of a tree, making it easy to convey and interpret results. In its most basic iteration, DTI provides binary classification, but modifications exist that allow for numerical and regression predictions. Because of these properties and their wide-spread implementations in easy-to-use software packages, we have seen many applications of DTIs for endpoints relevant to drug discovery, such as absorption, distribution, metabolism, and elimination (ADME), and toxicity [2]. Its roots can be traced back to the 1970s, with pioneering work from Topliss and his manually deduced ‘operational schemes for analog synthesis in drug design’ which are – in essence – decision trees [3,4]. Ever since then, DTI has been in use for questions relevant to drug discovery and design. In the 2000s, Wagner and Geerestein developed a DTI model to distinguish between potential drugs and nondrugs by identifying important structural features [5]. One more recent example is a study from 2015 in which Su et al. employed an approach based on C5.0, which is a variation of the classic DTI algorithm C4.5 [6] and uses an information entropy approach to find attributes that best classify the data at each given level. They established rule sets for structural features related to inhibition of five major cytochrome P₄₅₀ (CYP) enzymes (CYP1A2, CYP2C19, CYP2C9, CYP2D6, and CYP3A4) from training datasets of over 10,000 substances [7]. Another study by Newby et al. in 2015 used DTI to define the roles of permeability and solubility on the prediction of oral absorption in the lower intestines [8]. For drug discovery, prediction of e.g. metabolism via CYP enzymes or oral absorption helps to identify promising new compounds, applying the rule sets of the decision trees. For more historical perspective, the reader is referred to prior reviews [9–11].

There are important limitations to DTI that can hamper its use for the high volumes of data often encountered in data mining within the pharmaceutical sciences (e.g. from CHEMBL, National Center for Biotechnology Information, or PDB [12–14]. For instance, because the structure of the final model depends to a great deal on the training dataset, seemingly innocuous changes to these data can result in rather different models. This is due to the decreasing influence rules have as the process proceeds downstream [15]. Smaller datasets exhibit this behavior more readily than larger ones, but larger ones are prone to overfitting even if the number of predictive attributes is carefully selected, trees are pruned, and cross-validation measures are in place. While it is impossible to place precise limits on the number of instances (e.g. molecules) for which DTI paradigms yield best results, personal experience shows that datasets in the hundreds to low thousands produce robust models that perform well in cross-validated settings.

3. Strength in numbers

Although DTI models are fast and can successfully predict even noisy and complex clinical endpoints, it is important to be wary of their susceptibility to errors due to variance. When very large datasets are being analyzed, as is increasingly becoming the case with the massive amounts of data generated from high-throughput assays, or if the data are highly skewed – e.g. when only a few molecules show the desired property (positive controls) in a dataset with a larger proportion of negative controls – ensemble variations of classic DTI may give the more robust model and are seeing increasing use. In an ensemble predictor, many ‘weaker’ predictors can be combined to give a single ‘stronger’ predictor that represents the final hypothesis, and these are finding more and more acceptance in drug discovery applications.

Random Forests (RF) and Decision Forests (DF) are two examples that differ only modestly in their implementation. Both are consensus modeling techniques. The probably better known of the two, RF, creates a large number of shallow trees by resampling with replacement from the training data, while DF, distributed by the National Center for Toxicological Research (NCTR) and the Food Drug Administration, does the opposite: it relies on a small sample of deep, complex trees [16]. A recent case study by the NCTR reported on a model for drug-induced liver injury (DILI) using DF with fivefold cross-validation in 1241 compounds that predicted the trinary endpoint (‘most-DILI’, ‘less-DILI’, and ‘no-DILI’), with an average accuracy of 72.9% [17]. This is nearly identical to results from a study published in 2017 using support vector machines (SVM), RF, and 10-fold CV for the trinary classification of 721 drugs that reported accuracies of 73.8% and 72.6% [18]. Recently, RF and DTI algorithms were used to model OATP1B1 inhibition in a dataset of 1203 molecules, where they were on par with SVM in terms of predictive accuracy [19]. Furthermore, RF was used in 2017 to predict human intestinal absorption in a dataset with 970 substances with a high accuracy of over 90% [20].

Aside from outright prediction, RF can work as a powerful feature selector, e.g. to automatically condense a large set of molecular descriptors into a smaller ranked list of the most predictive attributes. A recent study showed how an RF-based approach to feature selection gave stronger classification results than SVM and artificial neural networks (ANN) with manually selected features on small molecule drugs [21].

Adaptive Boosting (AdaBoost), though not strictly a DTI ensemble predictor, is often used with DTI as the ‘weak’ learner. In 2018, Afolabi et al. used AdaBoost with DTI classifiers for the prediction of on-target activities in three mid-size datasets (5083–8568 instances) from the MDL Drug Data Report database, and saw consistently better results compared to SVM models [22].

4. Going deeper

Cheaper computational power has recently led to a renaissance of ANN in the form of deep learning (DL), two terms that are now often used in collusion with each other. This is already being applied to the high-volume datasets encountered in drug discovery [23]. In 2015, authors at Merck contrasted DL and RF for 15 large in-house datasets from a Kaggle [24] competition they sponsored in 2012 (endpoints spanning ADME (e.g. CYP inhibition, plasma protein binding) and on-target activity (e.g. angiotensin-II receptor interaction)), and found DL to be superior in up to 13 of these endpoints [25]. A more exhaustive analysis of other datasets for ADME and on-target activities ranked DL over SVM and a set of ‘classical’ ML algorithms, of which DTI-based paradigms were represented by AdaBoost DTI and RF, by comparing spider plots of different metrics of predictive power [26].

Although the DL concept of stacked algorithms representing hierarchical layers of representation of data is commonly implemented with ANN, there are already efforts being made to integrate other learners such as DTI or RF directly into DL setups [27]. This has not seemed to be applied to drug discovery yet, although successful ensemble implementations exist in toxicology. One example is the DeepTox pipeline, the grand challenge winner in the 2014 Tox21 Data Challenge hosted by the National Center for Advancing Translational Science [28]. Data consisted of over 12,000 drugs and environmental chemicals with up to 12 different toxicity endpoints. The DeepTox pipeline, while relying primarily on DL ANN, used RF, SVM, and elastic nets to supplement the DL models [29].

5. Expert opinion

Although the paradigm has been around for decades, DTI in its many different guises has managed to remain relevant in drug discovery. While single tree DTI algorithms such as C5.0, Classification and Regression Trees, or Chi-squared Automatic Interaction Detector are giving way to more complex ensemble setups, they are versatile, fast, and easy to handle learners that can drive hypothesis generation, generate intuitively understandable visualizations, and are often sufficient in small to midsize datasets. All of this makes the DTI family of learners an attractive tool for (quantitative) structure-activity relationships ((Q)SAR) that can even be used by researchers outside of the ML community. The investment is low due to the multitude of often free software packages with intricate graphical user interfaces (e.g. the Weka collection of data mining software [30] or the Konstanz Information Miner [31]). Of course, a basic knowledge of data mining and ML principles are necessary for users to build robust models and avoid drawing the wrong conclusions because models were misspecified.

Still, ensemble and consensus learners such as RF or AdaBoost are becoming more widespread, both in predictive model building and as an adjunct tool for feature selection. Even though tree-based algorithms are sometimes outperformed by other learners, one of their main advantages is the prioritization of the predictors, which allows deducing general rule sets that might also be applied without using the whole model or in dimensionality reduction and feature selection.

In the face of the rise of DL and big data for drug discovery, DTI has yet to carve out its niche within this subdomain of ML. It is unlikely that single-tree learners will play a role here by themselves, but ensemble predictors are well up to the task of handling big data problems. DL workflows mostly employ ANN (deep nets), but efforts outside of drug discovery have shown that DTI can act as one of the weak learners in such a setup. DL at its heart is multilevel learning, where raw data is subsequently integrated into more complex features to predict the actual outcome of interest. In drug discovery, this could span physicochemical properties such as lipophilicity, proceed to cell membrane passage and target engagement, and, finally, central nervous activity. Individual subproblems can be structured very differently, and different weak learners may be more capable of describing certain features than others.

For VSs, we will in all likelihood see more and more technically involved workflows and algorithms, although the success of DL will depend on our ability to provide it with high-quality data (a considerable feat giving that by its very nature huge data stores cannot be surveyed and curated in as much detail by human experts as could the small to midsize datasets that are the substrate for more traditional ML studies). Such models, while fully recognizing all their predictive power, will have one major drawback: they are black boxes, even to ML experts. Single decision trees will probably remain the most powerful tool for machine-assisted hypothesis generation.

Declaration of interest

The authors have no relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript. This includes employment, consultancies, honoraria, stock ownership or options, expert testimony, grants or patents received or pending, or royalties.

Reviewer Disclosures

Peer reviewers on this manuscript have no relevant financial or other relationships to disclose.

Additional information

Funding

This manuscript was not funded.