Decision tree models to classify nanomaterials according to the DF4nanoGrouping scheme

Abstract

To keep pace with its rapid development an efficient approach for the risk assessment of nanomaterials is needed. Grouping concepts as developed for chemicals are now being explored for its applicability to nanomaterials. One of the recently proposed grouping systems is DF4nanoGrouping scheme. In this study, we have developed three structure-activity relationship classification tree models to be used for supporting this system by identifying structural features of nanomaterials mainly responsible for the surface activity. We used data from 19 nanomaterials that were synthesized and characterized extensively in previous studies. Subsets of these materials have been used in other studies (short-term inhalation, protein carbonylation, and intrinsic oxidative potential), resulting in a unique data set for modeling. Out of a large set of 285 possible descriptors, we have demonstrated that only three descriptors (size, specific surface area, and the quantum-mechanical calculated property ‘lowest unoccupied molecular orbital’) need to be used to predict the endpoints investigated. The maximum number of descriptors that were finally selected by the classification trees (CT) was very low– one for intrinsic oxidative potential, two for protein carbonylation, and three for NOAEC. This suggests that the models were well-constructed and not over-fitted. The outcome of various statistical measures and the applicability domains of our models further indicate their robustness. Therefore, we conclude that CT can be a useful tool within the DF4nanoGrouping scheme that has been proposed before.

Keywords:

• Computational toxicology
• (Q)SAR
• nanomaterials
• oxidative stress
• grouping
• categorization

Introduction

The development of efficient approaches for the assessment of potential human health effects of nanomaterials is an ongoing effort. The potential for nanomaterials to improve material properties and consumer products is widely acknowledged (Godwin et al. 2015). Modifications of the (surface-related) physicochemical properties of nanomaterials may change the interactions of the materials with biological components (Hussain et al. 2015; Sauer et al. 2015). Risk assessment approaches as established for chemicals are under pressure as nanomaterials are being developed in an increasing number of types, and for different applications (Stone et al. 2014). It seems impracticable and economically undesirable to fully test all nanomaterials variabilities. This would also contradict the societal demands and legal requirements to reduce animal testing (EU 2006; ECHA 2008; Hartung 2009; Burden et al. 2017). At the same time it needs to be acknowledged that a wealth of experimental data on the toxic potential of nanomaterials has been generated (Krug 2014). The selection of data to be used in the risk assessment of nanomaterials is challenging as quite some discordant study results have been published, even for relatively clear-cut toxicological endpoints, such as genotoxicity (Oesch and Landsiedel 2012).

To keep pace with the rapid development of nanomaterials, grouping concepts as already developed for chemicals (ECHA 2008) are currently established for nanomaterials as well (Arts et al. 2014, 2015; Oomen et al. 2014; Stone et al. 2014; Bos et al. 2015; Godwin et al. 2015; Braakhuis, Oomen, and Cassee 2016; Landsiedel 2016). Most of the current grouping approaches involve to some extent the nanomaterial intrinsic properties and their biophysical interactions (i.e. system-dependent properties) in the perspective of the nanomaterial’s life cycle (Arts et al. 2014). Recently, a functionality driven framework for grouping of nanomaterials, the so-called DF4nanoGrouping framework has been proposed (Arts et al. 2015, 2016). In this approach nanomaterials are categorized in four groups (with possible subgroups). The first main group (MG1) consists of soluble nanomaterials. The second main group (MG2) contains nanomaterials with high bio persistence and high aspect ratio. In the third main group (MG3), there are nanomaterials without obvious biological effects (‘passive’). Finally, in the fourth main group (MG4) there are ‘active’ nanomaterials – those demonstrating surface-related specific toxic properties (Arts et al. 2015). To some extent, this approach combines intrinsic properties with system-dependent properties and toxicological effects. For example, the solubility of a nanomaterial can be regarded as an intrinsic property, but its dissolution rate is also highly dependent on the environmental conditions (Tantra et al. 2016). Detailed sub-grouping, especially in the group of active nanomaterials should be amended to this concept, and should be related to mechanisms of action, ideally integrated into the adverse outcome pathway concept (Ankley et al. 2010; Becker et al. 2015; Riebeling et al. 2016). A Bayesian network approach toward this concept has recently been described (Marvin et al. 2017).

A few computational (Quantitative) Structure–Activity Relationship ((Q)SAR) approaches have been developed for nanomaterials, mainly to explain and predict the toxicity of metal oxide nanomaterials (Fourches, Pu, and Tropsha 2011; Puzyn et al. 2011b; Gajewicz et al. 2015; Toropova et al. 2015; Pan et al. 2016). A basic principle of (Q)SAR modeling is to find structural parameters (descriptors) that are responsible for the observed biophysical interactions and toxicological effects. Some of these (potential) nanomaterial descriptors can be determined experimentally with the currently available analytical nanomaterial characterization techniques. Others can be calculated using theoretical chemistry approaches (Puzyn et al. 2011b). Modeling can be performed either in quantitative (QSAR) or qualitative (SAR) manner, dependent on the way of expressing the endpoint (i.e. numeric value or class).

In the context of recent discussions on grouping criteria, there is a need to explore relationships between the properties (intrinsic and extrinsic) and toxicity endpoints measured for nanomaterials. Thus, within this work we aimed to develop a series of SAR classification models that support the DF4nanoGrouping scheme by linking activity of 19 nanomaterials in three assays (namely, protein carbonylation, ferric reducing ability of serum, and NOAEC from a short-term inhalation study in rats) to their structural properties.

Methods

In this study we used data from 19 nanomaterials that were synthesized and characterized extensively in the previous BMBF funded project nanoGEM (nanoGEM 2012; Driessen et al. 2015) and other projects, such as EU FP7 project MARINA. Subsets of these nanomaterials have been used in toxicological studies, both in vitro and in vivo (short-term inhalation studies) of which the data have been published before (Landsiedel et al. 2010, 2014a, 2014b; Klein et al. 2012; Buesen et al. 2014; Hofmann et al. 2016; Wiemann et al. 2016) (using the same nanomaterials, additional measurements were performed as described below). In total we have obtained a comprehensive data set to develop and evaluate a novel modeling approach.

Materials characterization

Data on 19 nanomaterials is used in this study. As described previously (nanoGEM 2012; Wohlleben et al. 2016) nine nanomaterials were specifically synthesized with well-specified surface modifications on three core materials (i.e. silica and zirconia; see Table 1). These nine nanomaterials were synthesized by sol–gel or precipitation and were available in aqueous suspensions. Other nanomaterials were BaSO₄ NM-220, TiO₂ (NM-105, TLSF), CeO₂ (Al, NM-211, 212), and DPP (Bulk, Nano, and Red), all of which were obtained as powders. A complete characterization data set is available from these studies on the nanoGEM webpage (nanoGEM 2012) or in the JRC nanomaterial characterization reports (only for nanomaterials as provided by the JRC) and the studies as cited (Comero et al. 2011; Singh et al. 2011; Cotogno et al. 2013, 2014; Rasmussen et al. 2014).

Table 1. Experimentally determined physicochemical properties.

Name	Primary particle Size (nm)^a	Shape	Mean particle size (nm)^b	DLS particle size (nm)^c	DLS particle AUC (nm)^c	SSA^d (m^2/g)	Zeta potential (mV)	IEP (pH)^e
BaSO4.NM220	32	3	80	350	350	41	−39	3.3
CeO2	200	3	N/A	N/A	N/A	33	6	N/A
CeO2.Al	81	3	N/A	N/A	N/A	46	18	N/A
CeO2.NM211	12	3	N/A	N/A	N/A	33	16	N/A
CeO2.NM212	40	3	N/A	N/A	N/A	27	42	N/A
SiO2.NH3	15	3	27.5	42	20	200	0	7.2
SiO2.PEG	15	3	26.5	50	21	200	−26	4
SiO2.PO3	15	3	27.5	40	20	200	42.9	1
SiO2.UNMOD	15	3	27.5	40	19	200	−39	1
TiO2.NM105	50	1	35	478	980	51	−17	6.7
TiO2.TLSF	50	1	N/A	N/A	N/A	100	−3	N/A
ZrO2	42.5	3	N/A	N/A	N/A	24.9	−12	N/A
ZrO2.ACR	9	3	9	9	27	117	−39	1
ZrO2.NH3	10	3	12.5	315	38	105	−3.9	1
ZrO2.PEG	9	3	11	27	13	117	−7.8	7
ZrO2.TOD	9	3	9	9	11	117	−6.5	7.1
DPP.BULK	200	1	N/A	N/A	N/A	42	−11.4	N/A
DPP.NANO	400	1	N/A	N/A	N/A	64	−12.3	N/A
DPP.RED	43	3	N/A	N/A	N/A	30	−16	N/A

a Determined by TEM/SEM.

b Primary particle size (distribution) mean value calculated from the given range in Table S1.

c DLS size measured in water.

d Specific surface area (BET intrusion).

e Isoelectric point. Data extracted from (Comero et al. 2011; Singh et al. 2011; nanoGEM 2012; Cotogno et al. 2013; Cotogno et al. 2014; Rasmussen et al. 2014).

Protein carbonylation

A previous study indicated that the analysis of protein carbonylation is correlated to the nanomaterials’ overall toxicity as evaluated by the WST-1 assay (Driessen et al. 2015). Therefore, we used the carbonyl patterns induced after exposing NRK-52E cells in vitro to a subset of the above described suite of nanomaterials (Driessen et al. 2015). Briefly, carbonyl patterns were retrieved after exposing NRK-52E cells to the different nanomaterials, followed by cells lysis. Protein carbonyls were then labeled using OxiBlot Kit (Merck Millipore, Darmstadt, Germany) according to the manufacturer’s instruction, separated by 1D gel electrophoresis and analyzed by immunoblotting. The overall level of protein carbonylation for each nanomaterial was then qualitatively classified based on the immunoblotting results as being either negative, weak, medium, or strong compared to control (Driessen et al. 2015).

Nanomaterial intrinsic oxidative potential (IOP)

To determine the IOP of the nanomaterials we used the ferric reduction ability of serum (FRAS) assay. The purpose of the FRAS assay was to quantify the surface reactivity of nanomaterials under physiological conditions, the so-called biological oxidative damage. The assay quantifies the decrease in the total antioxidant capacity in human blood serum which was incubated with nanomaterials as compared to unexposed human blood serum. An optimized FRAS assay protocol was described by Hsieh et al. (2013) and further developed into an advanced multi-dose protocol by Gandon et al. (2017). Briefly, human blood serum is incubated with nanomaterials, which are then removed by centrifugation. The remaining antioxidant capacity of human blood serum is determined by the reduction of a colorless Fe³⁺ complex to form a blue Fe²⁺ complex. This is then quantified by measuring the light absorbance at 593 nm. The depletion of the anti-oxidative capacity in human blood serum by nanomaterial’s surface reactivity reduces the blue color. The read-out of the FRAS assay is calibrated by a concentration series of Trolox (6-hydroxy-2,5,7,8-tetramethylchroman-2-carboxylic acid, a Vitamin E analogon) and the result expressed as μUFRAS/m²h.

Short-term rodent inhalation study: derivation of NOAECs

The nanomaterials from the nanoGEM project were studied in short-term inhalation studies (Buesen et al. 2014; Landsiedel et al. 2014a; Hofmann et al. 2016). Briefly, Wistar rats were head–nose exposed to respirable dust (3 or 10 mg/m³) for 6 h/d at five subsequent days. Animals were sacrificed at study days 5, and 26 for histopathological evaluations. In addition, body weights, clinical parameters, and broncho-alveolar lavage was evaluated (Buesen et al. 2014; Landsiedel et al. 2014a; Hofmann et al. 2016). From these studies NOAECs were derived and used here.

Structural descriptor calculations

In addition to 89 structural descriptors derived from experimental characterization, 562 structural descriptors were calculated as follows. These calculated descriptors include (i) 28 quantum-mechanical descriptors computed at the semi-empirical level of the theory using PM7 Hamiltonian (Stewart 2013) with MOPAC 2012 software package (Stewart, J.J.P., Stewart Computational Chemistry, Colorado Springs, CO, http://OpenMOPAC.net; 2012); (ii) 503 constitutional, topological, and topostructural descriptors calculated with DRAGON version 4.0 software (Talette SRL, Milan, Italy, http://.talete.mi.it/products/software.htm; 2006); and (iii) 31 elemental descriptors computed with an Elemental Descriptors Calculator version. 1.0 (DTC Lab, Kolkata, India, http://teqip.jdvu.ac.in/QSAR_Tools/; 2013). The calculations were performed based on the simplest molecular models that include only a single molecular fragment that is repeated in the shell or core structure (e.g. polyethylene glycol monomer or titanium dioxide molecule). There were, in total, 517 calculated descriptors representing properties of the nanomaterial shell and 45 descriptors of the core.

Modeling

Data on the selected endpoints were all modeled using the classification trees (CT) technique for the supervised selection of the most significant descriptors that affect the modeled endpoint and for obtaining the predictive model (Questier et al. 2005). The process is based on growing a tree that divides nanomaterials into mutually exclusive groups (nodes). The algorithm starts with a ‘root node’ that contains all nanomaterials. Then, they are divided into ‘child nodes’ by recursive binary splitting. Each split is defined by a rule (threshold) considering values of the descriptor selected at a given stage. The descriptors are selected at each step based on the highest reduction of ‘impurity’ that means the binary split performed with the developed rule should lead to obtaining two maximally homogeneous groups of nanomaterials. The impurity is measured with Gini index G defined as: (1) where n_j is the number of nanomaterials from class j present in the node (Questier et al. 2005).

The modeling was performed in R software package with using recursive partitioning with RPART routines.

Validation of the models

In order to appropriately validate the predictive ability of the classic SAR models, the data are split into two sets: a training set (used to develop the classification tree) and a validation set (used for verifying the classification tree’s predictive ability). However, since CT developed on the basis of very small data sets might not be robust enough, we have carried out multiple splitting to investigate the potential influence of the splitting procedure on the modeling results (Puzyn et al. 2011a). First, the nanomaterials were sorted in an ascending order of their activity value, i.e. starting with the least active. Every third nanomaterial was included in the validation set (V), whereas the remaining nanomaterials formed the training set (T). Then, the procedure was repeated for various splits and the results of modeling based on different training sets were compared each other. After confirming sufficient robustness of the modeling procedure, we have developed the final predictive model trained on all 19 nanomaterials.

To verify the classification tree’s predictive ability, statistical measures commonly used for qualitative modeling were applied in this study (Figure 1). The first, sensitivity, measures the proportion of active nanomaterials that were correctly identified as such (Equation (2))(2) . The second, specificity, defines the proportion of inactive nanomaterials that were correctly identified as such (Equation (3))(3) (Baldi et al. 2000; Lever, Krzywinski, and Altman 2016; Pradeep et al. 2016).

Figure 1. Confusion matrix describing the performance of a classification model (or ‘classifier’) on a set of test data for which the true values are known.

It should be emphasized, however, that limited and unbalanced training dataset might face the risk of a bias toward the majority class and yield, therefore, a very optimistic accuracy estimate. This is so-called the ‘accuracy paradox’. Thus, to alleviate this problem, balanced accuracy, which is defined as the arithmetic mean of class-specific accuracies, was employed instead of standard accuracy (Equation (4))(4) . It is worth noting that the balanced accuracy will be reduced to the standard accuracy (i.e. the number of correct predictions divided by the total number of predictions), if the developed classification tree will perform equally well on either class. Finally, balanced error that describes the total number of misclassified nanomaterials was employed (Equation (5))(5) . (2) (3) (4) (5)

As the predictive ability of the model may depend on the splitting, so-called ‘consensus fuzzy predicting’ was applied, i.e. calculating the probability of being active and inactive based on the responses from models derived with different splits.

Among the various validation aspects, the applicability domain (AD) plays a crucial role for estimating the uncertainty in predictions. Thus, in order to verify whether a given nanomaterial lies within the intended domain of model’s applicability or not, as well as to confirm whether the prediction for particular nanomaterial is trustworthy or not, the AD was determined by using basic theory of the standardization approach. The standardization is a simple method for defining the X-outliers (in the case of the training set) and identifying the compounds that reside outside the domain (in the case of the validation set and prediction set) (Roy, Kar, and Ambure 2015).

Results

Experimentally derived physicochemical properties

In this study, we used data from 24 nanomaterials synthesized and investigated in a previous study. We extracted the data and focused on the physicochemical properties to be used in our models. The main experimental descriptors used for modeling are summarized in Table 1 (and Supplementary Table S1). For most of the nanomaterials surface chemistry variations were available; the chemical structures in the shell of these nanomaterials is indicated in Table 2.

Table 2. Nanomaterial surface coatings.

Calculated quantum-mechanical properties

The complete list of descriptors (both experimental and calculated) can be found in the Supplementary material (Table S2). Descriptor pre-processing was done in two stages: (a) removing descriptors with missing values, (b) removing descriptors with zero variance (descriptors with non-unique values, e.g. all equal to zero). After winnowing out only the useful descriptors, they were used for developing the predictive models. In effect 285 descriptors were retained for modeling (Table S1a).

Nanomaterial IOP

The IOP of the nanomaterials was determined using the FRAS assay, the result expressed as μUFRAS/m²h (Table 3).

Table 3. Summary of the nanomaterial intrinsic oxidative potential, protein carbonylation, and NOAECs derived from short-term inhalation studies (STIS).

Nanomaterial	Protein carbonylation^a	Intrinsic oxidativepotential [μUFRAS/m^2h]^b	STIS-derivedNOEAC [mg/m^3]^c
SiO2.UNMOD	Strong	0.015	2.50
TiO2.NM105	Medium	0.024	1.00
SiO2.NH3	Weak	0.011	50.00
SiO2.PO3	Weak	0.007	50.00
BaSO4.NM220	Negative	0.050	50.00
SiO2.PEG	Negative	N/A	50.00
ZrO2.ACR	Negative	N/A	50.00
ZrO2.NH3	Negative	N/A	N/A
ZrO2.PEG	Negative	N/A	N/A
ZrO2.TOD	Negative	N/A	50.00
CeO2	N/A	N/A	0.25
CeO2.Al	N/A	N/A	0.25
CeO2.NM211	N/A	0.007	0.25
CeO2.NM212	N/A	0.032	0.25
DPP.BULK	N/A	0.020	10.00
DPP.NANO	N/A	0.026	30.00
DPP.RED	N/A	N/A	30.00
TiO2.TLSF	N/A	N/A	0.50
ZrO2	N/A	N/A	10.00

a (Driessen et al. 2015).

b (Gandon et al. 2017).

c From (Buesen et al. 2014; Landsiedel et al. 2014a; Hofmann et al. 2016).

Protein carbonylation

The protein carbonylation data were taken from (Driessen et al. 2015). The labels used as input for CT were based on the original qualitative values. Nanomaterials lacking a protein carbonylation effect were labeled as ‘Inactive’. Nanomaterials with weak, medium, and strong protein carbonylation effect were labeled as ‘Active’.

NOAECs derived from the literature

For the modeling in our study we extracted the NOAECs from the short-term inhalation studies as reported earlier (Buesen et al. 2014; Landsiedel et al. 2014a; Hofmann et al. 2016). The NOEACs used are listed in Table 3.

Modeling

Protein carbonylation

Protein carbonylation data for 10 nanomaterials were included in this study. Six of them were inactive, whereas, four exhibited various levels of activity (Table 4). Thus, the studied nanomaterials were divided into two classes: active (A) and inactive (I).

Table 4. Activity in protein carbonylation assay.

			Activity class
Nanomaterial	Measured activity	Split	Observed	Predicted	Correctness	AD info
SiO2.UNMOD	Strong	Training	A	A	True	Not an X-outlier
TiO2.NM105	Medium	Training	A	A	True	Not an X-outlier
SiO2.NH3	Weak	Training	A	A	True	Not an X-outlier
SiO2.PO3	Weak	Training	A	A	True	Not an X-outlier
BaSO4.NM220	None	Training	I	I	True	Not an X-outlier
SiO2.PEG	None	Training	I	A	False	Not an X-outlier
ZrO2.ACR	None	Training	I	I	True	Not an X-outlier
ZrO2.NH3	None	Training	I	I	True	Not an X-outlier
ZrO2.PEG	None	Training	I	I	True	Not an X-outlier
ZrO2.TOD	None	Training	I	I	True	Not an X-outlier
CeO2	N/A	Prediction	–	I	–	Outside AD
CeO2.Al	N/A	Prediction	–	A	–	Outside AD
CeO2.NM211	N/A	Prediction	–	I	–	Within AD
CeO2.NM212	N/A	Prediction	–	I	–	Within AD
DPP.BULK	N/A	Prediction	–	I	–	Outside AD
DPP.NANO	N/A	Prediction	–	A	–	Outside AD
DPP.RED	N/A	Prediction	–	I	–	Within AD
TiO2.TLSF	N/A	Prediction	–	A	–	Within AD
ZrO2	N/A	Prediction	–	I	–	Within AD

N/A: not measured experimentally.

In the next step, we developed a classification tree model for predicting protein carbonylation activity of the remaining (not analyzed experimentally) nine nanomaterials. The developed model (Figure 2) utilizes two descriptors for discriminating nanomaterials into either active or non-active group. These are: ‘Size’ – the primary particle diameter and ‘SSA’ – surface area (BET intrusion). Only one nanomaterial was classified incorrectly (Table 4), which gives the following formal values of balanced accuracy: 80%, balanced error: 10%, sensitivity: 80%, and specificity: 100%. Those parameters are satisfying, since in practice, models with accuracy at the level of 70–90% are widely accepted (Linden and Yarnold 2016).

Figure 2. Classification tree for the protein carbonylation assay. Active nanomaterials are indicated with red; inactive with green color. Please note that the classification tree uses the auto scaled values of the descriptors. Thus, size≥ −0.41 corresponds to 12.5 nm and SSA≥ −1.4 corresponds to 47.9 m²/g.

The assessment of the AD of the developed models indicated no outliers and neither of the validation compounds reside outside the AD (Table 4). At the same time it can be observed that four of nine prediction compounds (i.e. experimentally untested nanomaterials) were found outside the AD. This can be explained by the fact that all those compounds (i.e. CeO₂, CeO₂.Al, DPP.BULK, and DPP.NANO) lie beyond the range of the values of the size descriptor used to develop a classification tree.

Please note that the model discussed above was obtained by using a complete experimental data set, available for 10 nanomaterials. We validated our model by multiple splitting the data in two sets: ‘training set’ – with seven nanomaterials, and ‘validation set’ consisted of three remaining nanomaterials. Detailed results of the multiple validation together with the ‘splitting patterns’ are presented in Table 5.

Table 5. Validation of the model robustness (protein carbonylation assay).

	Splitting patterns
Nanomaterial	#1	#2	#3	#4	#5
SiO2.UNMOD	T	T	V	T	T
TiO2.NM105	T	T	T	T	V
SiO2.NH3	V	T	T	T	T
SiO2.PO3	T	V	T	V	T
BaSO4.NM220	T	T	T	T	V
SiO2.PEG	V	T	V	V	T
ZrO2.ACR	T	V	T	T	T
ZrO2.NH3	T	T	T	V	V
ZrO2.PEG	V	T	V	T	T
ZrO2.TOD	T	V	T	T	T
Statistics for training set of nanomaterials
Sensitivity	100%	75%	100%	100%	75%
Specificity	100%	100%	100%	100%	100%
Balanced accuracy	100%	87.5%	100%	100%	87.5%
Balanced error	0%	12.5%	0%	0%	12.5%
Statistics for validation set of nanomaterials
Sensitivity	50%	100%	50%	50%	50%
Specificity	100%	100%	100%	100%	100%
Balanced accuracy	75%	100%	75%	75%	75%
Balanced error	25%	0%	25%	25%	25%

T: training set; V: validation set.

Interestingly, independent from the splitting pattern, we observed that one inactive nanomaterial, namely, SiO₂.PEG was falsely predicted to be active (false positive), while remaining nanomaterials were classified correctly. Since each validation set contained one active and two inactive nanomaterials, the models were also sensitive enough to discriminate active from inactive nanomaterials. However, since the predictive ability of the model may depend on the splitting, we applied so-called ‘consensus fuzzy predicting’ concept. That is, we calculated the probability of being active and inactive based on the responses from models derived with different splits and put the results together with the activity predicted based on the model developed for the complete set of experimental data (Table 6).

Table 6. Original and consensus fuzzy predictions of protein carbonylation for untested nanomaterials.

	Splitting pattern					Consensus fuzzyprediction based on multiple splitting
Nanomaterial	#1	#2	#3	#4	#5	A	I	Prediction based on thecomplete set of data
CeO2	I	I	I	I	I	0%	100%	I
CeO2.Al	A	A	A	A	A	100%	0%	A
CeO2.NM211	I	I	I	I	I	0%	100%	I
CeO2.NM212	I	I	I	I	I	0%	100%	I
DPP.BULK	I	I	I	I	I	0%	100%	I
DPP.NANO	A	A	A	A	A	100%	0%	A
DPP.RED	I	I	I	I	I	0%	100%	I
TiO2.TLSF	A	A	A	A	A	100%	0%	A
ZrO2	I	I	I	I	I	0%	100%	I

Independently on the splitting, we observed that three of nine untested compounds (i.e. CeO₂.Al, DPP.NANO, and TiO₂.TLSF) were predicted as active, while remaining prediction nanomaterials were predicted as inactive (Table 6). This is also in agreement with the results predicted by the model developed without dividing nanomaterials into the training and validation sets.

Nanomaterial IOP

A similar procedure has been applied for modeling the nanomaterial IOP. Experimental measurements have been performed for nine nanomaterials. Activity in IOP was transformed into binary scale by setting up a threshold of 0.01921 μU FRAS/m²h (1% of Mn₂O₃ reactivity, personal communication Dr. Wendel Wohlleben). Reactivity of nanomaterials was divided into three classes based on the reactivity of Mn₂O₃ as positive control (this material is highly reactive and non-soluble, thus effects are truly surface-induced): reactive >10% of positive control; intermediate 1–10%; non-oxidative <1% of positive control (Arts et al. 2015). Five nanomaterials with activities of ≥0.01921 μU FRAS/m²h (Table 7) were qualified as active (A), whereas, four with activities of <0.01921 μU FRAS/m²h were qualified as inactive (I). The developed model (Figure 3) utilizing only one descriptor (‘Size’ – primary particle diameter) was characterized by 100% of balanced accuracy. Here, too, we repeated the modeling in order to validate robustness of the model for three various splits (Table 8). Independent from the split, the accuracy for validation was always 100%.

Figure 3. Classification tree for nanomaterial intrinsic oxidative potential. Active nanomaterials are indicated with red; inactive with green color. Please note that the classification tree uses the auto scaled values of the descriptors. Thus, size≥ −0.41 corresponds to 23.83 nm.

Table 7. Nanomaterial intrinsic oxidative potential.

	Measured activity		Activity class
Nanomaterial	[μUFRAS/m^2 h]	Split	Observed	Predicted	Correctness	AD info.
BaSO4.NM220	0.050	Training	A	A	True	Not an X-outlier
CeO2.NM212	0.032	Training	A	A	True	Not an X-outlier
DPP.NANO	0.026	Training	A	A	True	Not an X-outlier
TiO2.NM105	0.024	Training	A	A	True	Not an X-outlier
DPP.BULK	0.020	Training	A	A	True	Not an X-outlier
SiO2.UNMOD	0.015	Training	I	I	True	Not an X-outlier
SiO2.NH3	0.011	Training	I	I	True	Not an X-outlier
CeO2.NM211	0.007	Training	I	I	True	Not an X-outlier
SiO2.PO3	0.007	Training	I	I	True	Not an X-outlier
CeO2	N/A	Prediction	–	A	–	Within AD
CeO2.Al	N/A	Prediction	–	A	–	Within AD
DPP.RED	N/A	Prediction	–	A	–	Within AD
SiO2.PEG	N/A	Prediction	–	I	–	Within AD
TiO2.TLSF	N/A	Prediction	–	A	–	Within AD
ZrO2	N/A	Prediction	–	A	–	Within AD
ZrO2.ACR	N/A	Prediction	–	I	–	Within AD
ZrO2.NH3	N/A	Prediction	–	I	–	Within AD
ZrO2.PEG	N/A	Prediction	–	I	–	Within AD
ZrO2.TOD	N/A	Prediction	–	I	–	Within AD

N/A: not measured experimentally.

Table 8. Validation of model robustness (nanomaterial intrinsic oxidative potential).

	Splitting pattern
Nanomaterial	#1	#2	#3
BaSO4.NM220	T	T	V
CeO2.NM212	V	T	T
DPP.NANO	T	V	T
TiO2.NM105	T	T	V
DPP.BULK	V	T	T
SiO2.UNMOD	T	V	T
SiO2.NH3	T	T	V
CeO2.NM211	V	T	T
SiO2.PO3	T	V	T
Statistics for training set of nanomaterials
Sensitivity	100%	100%	100%
Specificity	100%	100%	100%
Balanced accuracy	100%	100%	100%
Balanced error	0%	0%	0%
Statistics for validation set of nanomaterials
Sensitivity	100%	100%	100%
Specificity	100%	100%	100%
Balanced accuracy	100%	100%	100%
Balanced error	0%	0%	0%

T: training set; V: validation set.

As performed on the classification tree for the protein carbonylation assay, the model’s AD was defined using the standardization approach. The results summarized in Table 7, confirm that one nanomaterial from the validation set (i.e. DPP.BULK) resides outside the AD. This could have been expected, realizing that DPP.BULK was shown to have the highest values of the size descriptor used to develop a classification tree. As DPP.BULK is dissimilar from all other compounds used for the model development in terms of structural domain, its model prediction should be treated, therefore, with considerable caution as potentially less reliable.

By performing multiple splitting and applying consensus fuzzy prediction, we were able to confirm previous predictions (Table 9). The activity of the studied nanomaterials in the IOP assay was predicted using only the primary diameter of the particle. We noticed that nanomaterials of size equal or larger than 23.83 nm (that corresponds to the auto scaled value of −0.38) were classified as active. Although, this is in contrast to the usually observed trend according to which the activity increases with decreasing size (Oberdorster, Oberdorster, and Oberdorster 2005; Ivask et al. 2014).

Table 9. Original and consensus fuzzy predictions of nanomaterial intrinsic oxidative potential for untested nanomaterials.

	Splitting pattern			Consensus fuzzy prediction
Nanomaterial	#1	#2	#3	A	I	Prediction based on thecomplete set of data
CeO2	A	A	A	100%	0%	A
CeO2.Al	A	A	A	100%	0%	A
DPP.RED	A	A	A	100%	0%	A
SiO2.PEG	I	I	I	0%	100%	I
TiO2.TLSF	A	A	A	100%	0%	A
ZrO2	A	A	A	100%	0%	A
ZrO2.ACR	I	I	I	0%	100%	I
ZrO2.NH3	I	I	I	0%	100%	I
ZrO2.PEG	I	I	I	0%	100%	I
ZrO2.TOD	I	I	I	0%	100%	I

NOAEC derived from a short-term inhalation study in rats

The modeling procedure has been applied in case of the NOAECs derived from a short-term inhalation study in rats. The experimental data (Table 10) were obtained for 17 nanomaterials and then transformed into binary scale: 10 inactive (NOAEC ≥10 mg/m³) and 7 active (NOAEC <10 mg/m³) (Arts et al. 2015) (Figure 4).

Figure 4. Classification tree for a short-term inhalation study in rats. Active nanomaterials are indicated with red; inactive with green color. Please note that the classification tree uses the auto scaled values of the descriptors. Thus, LUMO_C< −0.61 corresponds to −2.40 eV; SSA ≥0.99 corresponds to 158.48 m²/g, and Shelpres <0.05 to the value of 0.5.

Table 10. Activity in a short-term inhalation study in rats.

	Measured NOEAC		Activity class
Nanomaterial	[mg/m^3]	Split	Observed	Predicted	Correctness	AD info.
BaSO4.NM220	50.00	Training	I	I	True	Not an X-outlier
SiO2.NH3	50.00	Training	I	I	True	Not an X-outlier
SiO2.PEG	50.00	Training	I	I	True	Not an X-outlier
SiO2.PO3	50.00	Training	I	I	True	Not an X-outlier
ZrO2.ACR	50.00	Training	I	I	True	Not an X-outlier
ZrO2.TOD	50.00	Training	I	I	True	Not an X-outlier
DPP.NANO	30.00	Training	I	I	True	Not an X-outlier
DPP.RED	30.00	Training	I	I	True	Not an X-outlier
ZrO2	10.00	Training	I	I	True	Not an X-outlier
DPP.BULK	10.00	Training	I	I	True	Not an X-outlier
SiO2.UNMOD	2.50	Training	A	A	True	Not an X-outlier
TiO2.NM105	1.00	Training	A	A	True	Not an X-outlier
TiO2.TLSF	0.50	Training	A	A	True	Not an X-outlier
CeO2	0.25	Training	A	A	True	Not an X-outlier
CeO2.Al	0.25	Training	A	A	True	Not an X-outlier
CeO2.NM211	0.25	Training	A	A	True	Not an X-outlier
CeO2.NM212	0.25	Training	A	A	True	Not an X-outlier
ZrO2.NH3	N/A	Prediction	–	I	–	Within AD
ZrO2.PEG	N/A	Prediction	–	I	–	Within AD

N/A: not measured experimentally.

The model utilizes four descriptors: LUMO_C (calculated energy of the lowest unoccupied molecular orbital of nanomaterial’s core), size (primary particle diameter), SSA (surface area/BET intrusion), and SHELPRES (coating presence, 0 – not or 1 – yes binary values). Mechanistic interpretation of the model is provided in ‘Discussion’ section.

Balanced accuracy calculated based on the training set containing all 17 nanomaterials was 100%, balanced error: 0%, sensitivity: 100% and specificity: 100%. When applying multiple splitting validation (Table 11), those statistics are not so perfect, but still do not differ substantially from other models published in the literature. As shown in Table 10, the results obtained using the standardization approach, confirmed that there is no outlier in the case of the training set and neither of the compounds reside outside the AD in the case of the validation and prediction sets.

Table 11. Validation of model robustness (a short-term inhalation study in rats).

	Splitting pattern
Nanomaterial	#1	#2	#3	#4	#5	#6
BaSO4.NM220	T	V	T	T	V	T
SiO2.NH3	T	T	V	T	T	V
SiO2.PEG	V	T	T	T	T	T
SiO2.PO3	T	V	T	V	T	T
ZrO2.ACR	T	T	V	T	V	T
ZrO2.TOD	V	T	T	T	T	V
DPP.NANO	T	V	T	V	T	T
DPP.RED	T	T	V	T	T	T
ZrO2	V	T	T	T	V	T
DPP.BULK	T	V	T	V	T	V
SiO2.UNMOD	T	T	V	T	T	T
TiO2.NM105	V	T	T	T	T	T
TiO2.TLSF	T	V	T	V	V	T
CeO2	T	T	V	T	T	V
CeO2.Al	V	T	T	T	T	T
CeO2.NM211	T	T	T	V	T	T
CeO2.NM212	T	T	V	T	V	V
Statistics for training set of nanomaterials
Sensitivity	100%	100%	100%	100%	100%	100%
Specificity	100%	100%	100%	100%	100%	100%
Balanced accuracy	100%	100%	100%	100%	100%	100%
Balanced error	0%	0%	0%	0%	0%	0%
Statistics for validation set of nanomaterials
Sensitivity	100%	100%	100%	100%	100%	100%
Specificity	100%	100%	75%	100%	100%	100%
Balanced accuracy	100%	100%	87.5%	100%	100%	100%
Balanced error	0%	0%	12.5%	0%	0%	0%

T: training set; V: validation set.

Predictions were independent on the way the model was developed and the splitting pattern; both untested nanomaterials were predicted to be inactive (Table 12).

Table 12. Original and consensus fuzzy predictions of activity in a short-term inhalation study in rats for untested nanomaterials.

	Splitting pattern						Consensus fuzzy prediction
Nanomaterial	#1	#2	#3	#4	#5	#6	A	I	Prediction based on thecomplete set of data
ZrO2.NH3	I	I	I	I	I	I	0%	100%	I
ZrO2.PEG	I	I	I	I	I	I	0%	100%	I

Discussion

Based on the classification tree technique, we have developed three structure-activity relationships (SAR) models for nanomaterials that support the DF4nanoGrouping scheme by linking the activity of 19 nanomaterials in three independent assays (that is: protein carbonylation, nanomaterial’s IOP, and rodent short-term inhalation study outcome) to their structural properties.

Most recently published Nano-(Q)SAR models (Fourches, Pu, and Tropsha 2011; Puzyn et al. 2011b; Gajewicz et al. 2015; Toropova et al. 2015; Mu et al. 2016; Pan et al. 2016; Basant and Gupta 2017; Fjodorova et al. 2017; Gajewicz 2017) have been developed based on a very small data set compared with the data sets underlying classic (chemical) (Q)SAR models. The problem of limited availability of experimental data that are suitable for nano-(Q)SAR modeling has been widely discussed in the literature (Puzyn et al. 2011b; Oksel et al. 2016; Banares et al. 2017). Although models developed on limited data are certainly less robust and do not cover the whole space of all theoretical variations of the structure, we do believe they provide useful information. In particular, these models can be used to reveal the most important structural factors (i.e. key descriptors) bridging to nanomaterial properties and the observed effects. Oksel et al. (2017) proposed three criteria on datasets for modeling: data should be obtained from a preferably single and standardized protocol; its accuracy and suitability for (Q)SAR analysis should be examined; and datasets should be large enough to allow rational division of the data into training and test sets (Oksel et al. 2017). Here we have used a unique dataset that fulfills these criteria, in that it comprises a very well characterized panel of nanomaterials that has been evaluated in different assays, both, in vitro and – importantly – in vivo (short-term inhalation). Available larger data sets usually comprise data from nanomaterials that have been synthesized in house and are evaluated with relatively straightforward in vitro screens (see Oksel et al. 2016). Here we have used a dataset of materials that in part have industrial relevance, that uniquely incorporates in vivo data, and could be evaluated using the DF4nanoGrouping scheme.

The principle of the DF4nanoGrouping scheme (Arts et al. 2014) is to assign a nanomaterial to one of four main groups: soluble nanomaterials [main group (MG1)]; bio persistent high aspect-ratio nanomaterials (MG2); bio persistent, non-fibrous passive nanomaterials (MG3); and bio persistent, non-fibrous, active nanomaterials (MG4). The grouping scheme is structured into three tiers of data collection. In the first tier mainly the intrinsic nanomaterial properties are characterized (Tier 1). This is followed by the assessment of system-dependent (extrinsic) nanomaterial properties (Tier 2) in combination with in vitro studies. Analytical chemistry characterization of the nanomaterials in their natural environment and data of a limited number of in vitro studies are used to assess whether a nanomaterial can be assigned to MG2 and to distinguish between passive and active nanomaterials (MG3 and MG4). Finally, in vivo studies addressed in Tier 3 are aimed to confirm the distinction between passive and active nanomaterials (Arts et al. 2014). While others have proposed different grouping schemes they, all boil down to the same principle: identifying key descriptors of nanomaterials that are most straightforward to experimentally determine and relate these to the respective biological effects (Oomen et al. 2014; Dekkers et al. 2016).

Obviously, classification of nanomaterials into MG1 and MG2 is possible using relatively easily obtained data (Tier 1, i.e. solubility and aspect ratio). The most problematic step is to decide whether the nanomaterial neither previously assigned to MG1 nor MG2 is passive (MG3) or active (MG4). For this purpose SAR modeling might be very valuable, either to directly predict the nanomaterial activity or to prioritize subsets of nanomaterials for further experimental evaluation. Here we analyzed three cases. First, the case of protein carbonylation, for this two parameters were identified as suitable descriptors, the primary particle diameter (‘Size’), and specific surface area (SSA) were identified as the key structural parameter for grouping for this endpoint. The key structural parameter for grouping was size that separated the smallest coted ZrO₂ nanomaterials from the others. Subsequently, the SSA was identified for subsequent classification, where nanomaterials with a large SSA were classified as active, as described previously (Driessen et al. 2015). The intrinsic oxidation potential could be predicted using the primary particle diameter (‘Size’), as the only descriptive parameter. Surprisingly it separates nine smaller nanomaterials (i.e. CeO₂.NM-211, SiO₂.NH₃, SiO₂.PEG, SiO₂PO₃, SiO₂.UNMOD, ZrO₂.ACR, ZrO₂.NH₃, ZrO₂.PEG, and ZrO₂.TOD) from the others. Even, although the number of studied nanomaterials was not large, and not all possible variations in the structure have been represented in the data set, the classification results lead to interesting conclusions. The simple rule ‘the smaller the size the larger the toxicity’ does not always hold true. Our results indicate that this needs to be refined taking the surface reactivity into account. For instance, the small ZrO₂ nanomaterials all contained a coating. If a nanomaterial’s core is small, but has a coating on its surface, the (absolute) surface that is available for reactions is smaller compared to the surface of larger nanomaterials. Unfortunately, no uncoated ZrO₂ nanomaterials were available.

The third model was developed for the NOAECs derived from a rodent short-term inhalation study. In this case, the most important parameter deciding on the activity was the energy of the lowest unoccupied molecular orbital calculated with quantum-mechanical methods for the nanoparticle’s core (LUMO_C). This parameter is related to the surface redox activity of nanomaterials. In the Hartree–Fock calculations the energy level of LUMO corresponds to the material’s electron affinity. Thus, in case of those nanomaterials having high negative values of, large amounts of energy are released in response to the attachment of additional electrons at the surface. For semi-conducting nanomaterials, the energy of LUMO can be used as the first approximation of the conduction band level. As previously discussed by (Gajewicz et al. 2015), the energy of the conduction band is the key parameter for assessing the ability of a nanomaterial surface to interfere with the naturally occurring intracellular redox reactions. The high intensity of such reactions may disturb the natural redox balance and result in high concentration of reactive oxygen species. In our study, nanomaterials having a LUMO_C< −2.40 eV (i.e. nanomaterials with TiO₂ or CeO₂ cores) were classified as active at the first stage. In the next stage, the nanomaterials with a relatively small (<158.49 m²/g) SSA were classified as inactive. The only nanomaterials with sufficiently large SSAs (≥158.48 m²/g) were those having a SiO₂ core. However, only uncoated SiO₂ (SiO₂.UNMOD) was active; the presence of a coating (SHELPRES =1) led to reduction of the activity.

CT are widely used for classification and prediction, as they automatically select the descriptors most relevant to a specific endpoint (Wang et al. 2006; Ma et al. 2008). They have successfully been used for both large and small datasets (Oksel et al. 2016). CT have proven especially useful with ‘noisy’ data, e.g. nanoparticle toxicity (Thomas et al. 2014; Oksel et al. 2015; Oksel et al. 2016), where they can be used for pre-screening and reducing redundant variables. Besides key descriptors a classification tree also gives a set of property value thresholds, which also may be useful for designing nanomaterials. The most commonly used methods for verification of the quality of CT predictions are the four statistical measures used in this study: accuracy, error rate, sensitivity, and specificity. One should note that the values of these statistical measures obtained in this study are quite high and comparable with the parameters developed in similar studies (Oksel et al. 2016). In addition we assessed the AD of our models as this is crucial for estimating the uncertainty in predictions. Only the classification of one nanomaterial in one of our models was outside its AD. Moreover, the maximum number of descriptors that were finally selected by the CT was very low – one for IOP, two for protein carbonylation, and three for NOAEC. This suggests that the models were well constructed and not over-fitted.

From an integrated testing strategy perspective, there is a strong incentive to maximally increase the (cost) efficiency of experimental testing and to reduce animal experimentation. The three models developed here use three descriptors, the primary particle diameter (‘Size’), SSA, and the energy of the lowest unoccupied molecular orbital calculated with quantum-mechanical methods for the nanoparticle’s core (LUMO_C). We explored if one of the models could predict the outcome of the other, or if the in vitro data-based models predicts the in vivo results, this however, did not result in a reliable correlation (data not shown). Grouping has been identified as an important tool to assess weather hazard data on one form of a nanomaterial can extrapolated to other forms of this nanomaterial. ECHA has identified that a robust scientific explanation is needed to underpin the definition of a group (ECHA 2017). CT like developed here, can contribute to such a scientific underpinning of grouping. Here we have shown that CT can be a useful tool within the DF4nanoGrouping scheme that has been proposed before.

Conclusion

In this study, we have developed three SAR classification tree models showing that even with relatively small data sets reliable predictions can be achieved. The maximum number of descriptors that were finally selected by the CT was very low – one for IOP, two for protein carbonylation, and three for NOAEC. This suggests that the models were well constructed and not over-fitted. The outcome of various statistical measures and the ADs of our models further indicate their robustness. We, therefore, conclude that CT can be a useful tool within the DF4nanoGrouping scheme that has been proposed before.

Disclosure statement

Authors declare that they have no financial or non-financial competing interests. All authors collaborate with several companies via National and European projects. RL and MIL are employers of BASF SE, a chemical company producing and marketing chemical formulations.

Additional information

Funding

The research described here was funded by the CEFIC Long-rang Research Initiative (LRI N4 project on grouping of nanomaterials) and European Chemical Industry Council. Authors also acknowledge previous funding that was used to generate large parts of this experimental data set, in particular BMBF nanoGEM.