How to correctly develop q-RASAR models for predictive cheminformatics

KEYWORDS:

• q-RASAR
• read-across
• machine learning
• QSAR
• data fusion
• leave-same-out
• predictions
• modeling

1. Introduction

One of the earliest and simplest forms of in silico predictive cheminformatics is the Quantitative Structure-Activity Relationship (QSAR). This algorithm aims to develop a mathematical model that establishes a relationship between the structural features and the biological activity [1]. However, developing QSAR models can sometimes be challenging, especially when the number of experimentally known data points is limited [2,3]. In such cases, to avoid the statistical complexity, Read-Across has emerged as an alternative approach. However, one of the key drawbacks of the Read-Across approach is the lack of quantitative interpretability of the features. To combine the advantages of both QSAR and Read-Across, Banerjee and Roy [4] developed statistical models using Read-Across-derived similarity and error-based measures serving as descriptors. This unconventional modeling approach was termed quantitative Read-Across Structure-Activity Relationship (q-RASAR) that encodes Read-Across-derived information into a statistical modeling framework. Over a relatively short period since its introduction, this technique has found use for modeling various pharmaceutical activity/toxicity/property endpoints [4–12], environmental property endpoints [13–29], and materials property-based endpoints [30–35], and in most of the cases, the external predictivity was enhanced. Some case studies, additionally, reported enhancement in the internal validation metrics. Table 1 lists various endpoints modeled using the q-RASAR approach. Within the q-RASAR paradigm, there is also a provision for modeling multiple endpoints simultaneously using the application of data fusion q-RASAR, which is somewhat similar to the multitask modeling approach using the perturbation-theory machine learning methodology [36–38].

Table 1. Various endpoints modeled using the q-RASAR approach.

Endpoint type	Specific endpoint	Authors	Reference
Pharmaceutical toxicity/activity/property	Androgen receptor binding affinity	Banerjee and Roy (2022)	[4]
	Cardiotoxicity	Banerjee and Roy (2023)	[5]
	Skin sensitization	Banerjee and Roy (2023)	[6]
	Skin sensitization	Banerjee and Roy (2023)	[7]
	Mutagenicity	Pandey and Roy (2023)	[8]
	Multitarget inhibition of anti-Alzheimer compounds	Kumar et al. (2024)	[9]
	Blood brain barrier permeability	Kumar et al. (2024)	[10]
	Inhibition of MGMT (an anticancer target)	Sun et al. (2023)	[11]
	Inhibition of histone deacetylase 8 (an anti- leukemia target)	Khatun et al. (2024)	[12]
Environmental toxicity	Various toxicity endpoints (honey bee toxicity, inhalation toxicity, avian toxicity, etc.)	Banerjee and Roy (2023)	[13]
	Cytotoxicity of metal oxide nanoparticles	Banerjee et al. (2023)	[14]
	Toxicity of mixtures against bacterial species	Chatterjee and Roy (2023)	[15]
	Honey-bee toxicity of mixtures	Chatterjee et al. (2023)	[16]
	Fish toxicity of pesticides	Ghosh et al. (2023)	[17]
	Subchronic oral safety	Ghosh and Roy (2024)	[18]
	Acute fish toxicity	Chen et al. (2023)	[19]
	Pesticide risk during pregnancy	Sobanska (2023)	[20]
	Toxicity of chemicals against Gammarus species	Yang et al. (2023)	[21]
	Toxicity of chemicals against tilapia fish	Yang and Kar (2024)	[22]
	Toxicity of chemicals against Rohu	Gallagher and Kar (2024)	[23]
	Toxicity of chemicals toward T. pyriformis	Ghosh et al. (2024)	[24]
	Agrochemical phytotoxicity in oat and corn	Pandey et al. (2024)	[25]
	Tadpole ecotoxicity assessment	Khan et al. (2024)	[26]
	Ecotoxicity assessment of veterinary pharmaceuticals on terrestrial species	Banjare et al. (2024)	[27]
	Hazardous effects of pesticides on chickens	Das et al. (2024)	[28]
	Acute toxicity for Chlorella vulgaris	Jiang et al. (2024)	[29]
Materials property	Specific surface area of perovskites	Banerjee et al. (2023)	[30]
	Retention time of pesticide residues	Ghosh et al. (2023)	[31]
	Power conversion efficiency of organic dyes	Pore et al. (2023)	[32]
	Detonation heat of nitrogen containing compounds	Pandey et al. (2023)	[33]
	Power conversion efficiency of organic dyes	Pore et al. (2024)	[34]
	Performance and stability parameters of energetic materials	Pandey and Roy (2024)	[35]

2. Selection of the important molecular features – a basic need for the computation of RASAR descriptors

A proper selection of the essential molecular features (descriptors) is an important aspect that determines the performance of the q-RASAR models. This is because the selected descriptor matrix is used to define the similarity among the compounds. This similarity is best defined when the modeler judiciously selects certain essential features from the complete feature pool using standard feature selection algorithms. If the similarity is defined based on the entire descriptor pool without feature selection, noise, inter-correlation, and redundancy will hinder the estimation of the actual similarity among compounds.

3. How to compute the RASAR descriptors

For the development of q-RASAR models, the basic pre-requisite is to compute the RASAR descriptors. This can be done using a simple Java-based RASAR descriptor calculator tool – RASAR-Desc-Calc-v3.0.3, which is freely available from https://sites.google.com/jadavpuruniversity.in/dtc-lab-software/home. This process is different from the standard procedure for the computation of conventional QSAR descriptors in:

1. Unlike the conventional approach, the RASAR descriptors are computed after dataset division and initial feature selection of the molecular descriptors. Therefore, the RASAR descriptors are computed separately for the training and test sets.

2. The standard molecular descriptors for a particular compound describe the chemical properties of the compound of interest. However, the RASAR descriptors describe the information of the close source neighbors for a particular query compound, and not the query compound itself.

The basic algorithm is that this tool computes the RASAR descriptors for the query set, using the chemical information from the source set. It is to be noted that the source set is always the training set of compounds that serve as the close source neighbors for a particular query compound. This information clarifies that when one intends to compute the RASAR descriptors for the test set, the training set file is used as the source set input file, while the test set file is used as the input for the query set. Similarly, when the training set RASAR descriptors are computed, the training set serves as both the source and query set inputs. This information has been reflected in Table 2 for the better understanding of the readers. It is to be noted that in this particular case, a particular training set compound (serving as a query compound here) finds itself as the most similar compound in the list of close source compounds. In this case, the RASAR descriptor calculation is based on the subsequent nearest neighbors without taking the identical compound, thus removing the bias involved in the computation of the RASAR descriptors for the training set. This algorithm is termed Leave-Same-Out (LSO) [14] where the same compound is not included in the RASAR descriptor computation. Please note that the LSO is automatically applied by the RASAR descriptor calculator tool, and the user does not require performing any additional operation. Additionally, this descriptor calculating tool requires a set of read-across-derived hyperparameters (σ, γ, and the number of close source compounds) associated with the best similarity algorithm. The user may either proceed with the default setting of the hyperparameters (σ = 1, γ = 1, number of close source compounds = 10) or use an optimized hyperparameter setting derived from Read-Across, which is mentioned in the next section.

Table 2. Input file specifications for the computation of the RASAR descriptors.

Scenario	Source set input	Query set input
Computation of the training set RASAR descriptors	Training set file*	Training set file*
Computation of the test set RASAR descriptors	Training set file	Test set file

*Both file names must be same during training set RASAR descriptor computation.

4. Optimization of the read-across-derived hyperparameters

This can be done using the tool Auto_RA_Optimizer-v1.0 (available from https://sites.google.com/jadavpuruniversity.in/dtc-lab-software/home). This tool performs a grid search involving different combinations of the hyperparameters associated with the different similarity-based measures stated previously and reports the prediction performance. The user needs to divide the training set into a sub-training and a sub-test set (a.k.a. a calibration and a validation set), and these files should be the input for the tool. The set of hyperparameters generating the best (or among the best) prediction for the sub-test set should be considered as the optimized hyperparameter setting. This setting should be used for the computation of the RASAR descriptors. It should be noted that the user should not use the original training and test sets as input for the Auto_RA_Optimizer tool, since this concept does not adhere to the general concepts of hyperparameter optimization in Machine Learning. Additionally, the user may want to manually perform the grid search using the Read-Across-v4.2.2 (available from https://sites.google.com/jadavpuruniversity.in/dtc-lab-software/home) by taking the sub-training and sub-test sets as inputs and executing the tool for every possible combination of hyperparameters. This manual operation should also be done for classification-based RASAR modeling.

5. Data fusion and model development

Once the RASAR descriptors are computed for both the training and test sets, the user may want to fuse the RASAR descriptors with the original set of selected molecular features to generate a complete descriptor pool (data fusion). However, from the previous studies, it is observed that this step is essential in the case of q-RASAR [7], for which the reason has already been explained in our previous publication [13], but this step can be skipped in the case of classification-based RASAR (c-RASAR) [6]. The complete descriptor pool should be further subjected to feature selection for the development of an array of statistical and machine-learning models. This feature selection algorithm should ideally be the same that was used to select the molecular features in the QSAR model for a comparable analysis.

A schematic representation of the modeling workflow for q-RASAR modeling and its difference with QSAR has been represented in Figure 1.

Figure 1. A schematic representation of the modeling workflow for q-RASAR (LSO = leave-same-out).

6. Dos and don’ts during the q-RASAR model development

This section will point out different scenarios with possible errors during the q-RASAR model development with examples found from a few recent publications of different research groups.

1. q-RASAR is a hybrid modeling approach employing molecular descriptors (as used in QSAR) and similarity descriptors (as used in Read-Across). However, it is not an ensemble prediction approach. Srisongkram [39] worked on predicting the skin cytotoxicity exerted by various compounds using chemometric modeling approaches. They have initially developed two QSAR models and two sets of Read-Across predictions using different descriptors and fingerprints. A final stacking regressor, in the form of a Random Forest, was used to generate predictions. This is an example of ensemble modeling where predictions from different models (here these are QSAR and Read-Across) are used as descriptors for a final stacking regression or a voting classification. It should be noted that this methodology is not a quantitative Read-Across Structure-Activity Relationship (q-RASAR), since this method is nothing but ensembling the predictions of Read-Across and QSAR. On the other hand, for the development of q-RASAR models, the basic pre-requisite is to compute the Read-Across-derived similarity and error-based descriptors that can be used in a modeling framework.

2. The q-RASAR approach uses the same training and test sets as used in QSAR analysis.Gallagher and Kar [23] developed a q-RASAR model to assess the toxicity of organic chemicals on Labeo rohita. They state that after the data fusion step, the authors have again divided the dataset into training and test sets using a random division approach. Although the authors mentioned that the composition of the training and test set was the same during the RASAR descriptor calculation, this step was unnecessary and error-prone. The composition of the training and test set should remain entirely the same throughout the modeling approach since the RASAR descriptors are computed based on the source compounds. (training set compounds)

3. The main input files for the RASAR descriptor computation are per Table 2 where the RASAR descriptors for the test set should be generated taking the training set as the source set, and the test set as the query set, while the RASAR descriptors for the training set should be generated taking the training set as inputs for both the source and query sets. However, in the works of Pandey et al. [25], this fact has been incorrectly represented, as the whole dataset is stated to be used as the query set.

The use of correct training and test set files and the application of the correct feature selection (as used in the case of QSAR) have correctly been shown in the works of Sun et al. [11] and Chen et al. [19].

7. Expert opinion

The concept of q-RASAR is a simple, yet highly conceptual algorithm that computes descriptors based on the properties of the close congeners of a particular query compound and not using the properties of that query compound itself. Therefore, it is highly important that the users need to understand that the training set is always the source set and the query molecules may belong to either the training or the test set, depending upon the input. It is to be remembered that the RASAR descriptor tool always calculates the RASAR descriptors for the query set, taking the training set compounds as the source set from where close source congeners are identified.

In terms of interpretability, it should be noted that the interpretation of the RASAR descriptors is relative. This means, unlike the conventional molecular descriptors where the descriptors describe the properties of a particular compound, the RASAR descriptors describe the properties of the close congeners of a particular query compound.

The RASAR descriptor calculator tool asks for a set of hyperparameters and the choice of similarity algorithm that is used to define the similarity between the source and query compounds. Typically, we recommend proceeding with the default setting of the hyperparameters (σ = 1, γ = 1, number of close source compounds = 10) when the dataset size is comparatively large and one may expect that the training set is sufficiently large enough to find at least 10 similar compounds for a particular query compound. We also recommend that the Gaussian Kernel similarity-based algorithm should be adopted when computing the RASAR descriptors using the default setting. The hyperparameter σ is associated with the Gaussian Kernel similarity algorithm while the hyperparameter γ is associated with the Laplacian Kernel similarity algorithm. The number of close congeners is common to all three similarity algorithms (i.e. Euclidean distance-based similarity, Gaussian Kernel similarity, and Laplacian Kernel similarity). If the modeler decides to optimize the hyperparameters, the training set may be divided into sub-training and sub-test sets, which may be fed into a grid search using the tool Auto_RA_Optimizer-v1.0. Additionally, the modeler may just select the best-performing similarity algorithm from the grid search results and then employ this similarity measure along with the corresponding default setting of the hyperparameters to compute the RASAR descriptors.

Another possibility where things can go wrong is when the user wants to analyze the applicability domain (AD) of the RASAR descriptor space or the fused data space and remove outliers just before the development of q-RASAR/c-RASAR models. While removal of the outliers/outside AD compounds from the test set is perfectly okay, the same is not straightforward for the training set. This is because the training set compounds are used as a source set for the computation of the RASAR descriptors, and the compound that is intended to be removed may be a member of the close source congeners for several query compounds. Therefore, when the user intends to remove such outliers from the training set, it is important to re-calculate the relevant RASAR descriptors using the same setting of the hyperparameters. This revised descriptor pool can now safely be used to develop q-RASAR/c-RASAR models.

Another issue is that the RASAR descriptor calculator tool may not run or may take more time to compute the RASAR descriptors when the size of the training and test sets are very large. Although the user can split the query set into different sections and input one section at a time to compute the RASAR descriptors for the query set, this is not applicable to the training set/source set since this will result in the improper identification of the close congeners as the tool will identify the required number of close source compounds from the limited sample size of the training set only. One should keep in mind that the training set/source set compounds should remain unaltered during the computation of the RASAR descriptors. Therefore, to tackle this issue, the user should allot a higher usable memory to the RASAR descriptor calculator tool. This can easily be done with the help of command prompt. In cases where the system specifications do not allow the allotment of a higher memory, alternative system may be tried.

In cases of classification RASAR (c-RASAR) modeling, after computation of the RASAR descriptors, the modeler should remove SD_Activity, SE and CVact from the modeling analysis. These descriptors are the weighted standard deviation of the activity values of the close source congeners, the weighted standard error of the activity values of the close source congeners and the coefficient of variation of the activity values of the close source congeners. Since these descriptors deal with the observed response values of the close source congeners, they become irrelevant when the response values are graded and not quantitative values.

Additionally, indicator 1 and indicator 2 should not be considered for q-RASAR/c-RASAR modeling analysis since these are just notations for assessing the modelability of the dataset and are related to s_m¹ and s_m².

To summarize, it is highly essential for the users to completely understand the algorithm and concepts of q-RASAR/c-RASAR and use them carefully and judiciously to eliminate chances of errors. The readers may refer to the compilation on q-RASAR [40] for an elborate understanding of the concepts and may also visit the RASAR webpage (https://sites.google.com/site/kunalroyindia/home/rasar) for the relevant concepts, publications, and videos.

Declaration of interest

The authors are the developers of the q-RASAR approach and the free tools (RASAR Descriptor Calculator and Auto RA Optimizer within a ‘q-RASAR’ project funded by Life Science Research Board, DRDO, India (LSRB/01/15001/M/LSRB-394/SH&DD/2022)). The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed.

Reviewer disclosures

One reviewer declares employment with Charles River Laboratories. Peer reviewers on this manuscript have no other relevant financial relationships or otherwise to disclose.

Additional information

Funding

The authors are funded by the Defence Research and Development Organisation (DRDO) through the Life Sciences Research Board via a senior research fellowship [no. LSRB/01/15001/M/LSRB-394/SH&DD/2022]