Artificial Flora Algorithm-Based Feature Selection with Gradient Boosted Tree Model for Diabetes Classification

Figures & data

Table 1 Comparative Analysis of Available Feature Selection Methods

Types of Techniques	Feature Selection Methods	Functionality of Feature Selection Methods and Demerits	Demerits
Semi – Supervised^Citation16	Graph-Based	Clustering of original features into several dependent features of dominant set and includes only a small set of features.	It cannot use approximation properly. This method is used for feature extraction and division of multimodal images.
	Co-Training based	Utilizes two integral features of unlabeled information for one another and to foresee the test together.	It explores what excess features have impact on the semi-supervised learning techniques and also it finds irrelevant features of data.
	Self-Training based	Initiates an initial characterization by utilizing the labelled part of the information and uses this model to group the unlabeled part.	It provides simultaneous grouping of data instead of proving the quantification. It could not be proceeding with linear transformation of data.
	SVM based	Utilized for solving the dual optimization issue.	It provides only approximate solution instead of complete solution.
Supervised^Citation17	Co-relation Based	A multivariate channel, it selects the subsets of features that themselves are uncorrelated.	It shows high relationship with the class.
	Consistency-based	A multivariate procedure, it selects the subsets of features, however chooses features as per the degree of consistency with the class.	It utilizes an irregularity criterion to decide an adequate data decrease rate.
	INTERACT	It is executed by the symmetrical vulnerability of all features considered, and sorting features accordingly.	It assessing the factors by their consistency commitment, in order to choose just those whose consistency commitment exceeds a foreordained limit.
	Information Gain	A basic univariate channel that registers the common data for each features and class.	It delivers an arranged positioning of the limited feature set only.
	Relief	A mainstream multivariate channel in view of nearest neighbors. It works by haphazardly selecting test Features and looking for nearest neighbors from a similar class, but it missing all others classes.	The values of the selected test are contrasted and the hits and misses, and afterwards the pertinence score for each component is refreshed.
	Lasso regularization	Regularizes model boundaries by contracting the regression coefficients and decreasing some of the elements to zero.	This method puts a requirement on the amount of the supreme features of the model boundaries instead of target label.
Un-Supervised^Citation18	Filer	Chooses the most applicable features through the actual information ie, features are assessed dependent on characteristic properties of the information, without utilizing any grouping methods that could control the inquiry of important features.	The demerits of this techniques are limitation in speed and adaptability.
	Wrapper	Assesses feature subsets by utilizing the consequences of a particular grouping techniques. This method was implemented by discovering feature subsets that add to improve the nature of the outcome of the grouping method.	In any case, the fundamental inconvenience of this feature selection methods having computational expense, and they are restricted to be utilized related with a specific grouping technique.
	Hybrid	Attempts to misuse the characteristics of the filter and wrapper.	It attempting to have a decent trade-off between proficiency (computational exertion) and viability (quality in the related target task when utilizing the chosen features).

Abbreviation: SVM, support vector machine.

Table 2 Description of the Datasets Used in the Work

Dataset Type	Description	Values
Type I Diabetes Mellitus	Number of Instances	306
	Number of Attributes	22
	Number of Class	2
	Number of Positive Samples	152
	Number of Negative Samples	154
Type II Diabetes Mellitus	Number of Instances	101,766
	Number of Attributes	49
	Number of Class	2
	Number of Positive Samples	78,363
	Number of Negative Samples	23,403
Gestational Diabetes Mellitus	Number of Instances	768
	Number of Attributes	8
	Number of Class	2
	Number of Positive Samples	268
	Number of Negative Samples	500

Figure 1 List of attributes in the type 1 diabetes mellitus dataset.

Figure 2 List of attributes in the type 2 diabetes mellitus dataset.

Figure 3 List of attributes in the gestational diabetes mellitus dataset.

Figure 4 Block diagram of the artificial flora algorithm (AFA)-based feature selection with gradient boosted tree (GBT)-based data classification (AFA-GBT) model. GWO, grey wolf optimization; PSO, particle swarm optimization GA, genetic algorithm.

Table 3 Details of Missing Values and Categorical Variables in the Dataset

Data Set Type	Missing Values		Number of Categorical Variables
Type I Diabetes Mellitus	Missing Feature	Count	30
	Race	2273
	Weight	98,569
	Payer_Code	40,256
	Medical_Specialty	49,949
	Diag_1	21
	Diag_2	358
	Diag_3	1423
Type I Diabetes Mellitus	None	0	17
Gestational Diabetes Mellitus	None	0	1

Figure 5 Basic migration and reproduction processes in the artificial flora algorithm.

Note: Reproduced Cheng L, Wu XH, Wang Y. Artificial flora (AF) optimization algorithm. Appl Sci. 2018;8(3):329. doi:10.3390/app8030329. https://creativecommons.org/licenses/by/4.0/.⁴⁹

Figure 6 Flowchart presenting the steps of the artificial flora algorithm.

Note: Reproduced Cheng L, Wu XH, Wang Y. Artificial flora (AF) optimization algorithm. Appl Sci. 2018;8(3):329. doi:10.3390/app8030329. https://creativecommons.org/licenses/by/4.0/.⁴⁹

Table 4 Features Selected by the Proposed AFA from the Applied Datasets

Dataset	Selected Features	No. of Features Selected	No. of Features Selected	Total No. of Features	Percentage of Reduction from the Total Features
	Feature ID	Feature Name
Type – 1	1	Age	12	22	45%
	3	Area of Residence
	4	HBA1C
	5	Height
	6	Weight
	7	BMI
	8	Duration of Disease
	15	Impaired Glucose Metabolism
	16	Insulin Taken
	17	How Taken
	18	Family History Affected in Type-1 Diabetes
	20	Hypoglycemis
Type – II	1	Encounter_ID	24	49	51%
	2	Patient_NBR
	4	Gender
	6	Weight
	7	Admission_Type_ID
	8	Discharge_Disposition_ID
	9	Admission_Source_ID
	12	Medical_Specialty
	14	Num_Procedures
	15	Num_Medications
	16	Number_Outpatient
	19	Diag_1
	22	Number_Diagnoses
	24	A1CResult
	25	Metformin
	27	Nateglinide
	29	Glimepiride
	31	Glipizide
	33	Tolbutamide
	36	Acarbose
	37	Miglitol
	41	Citoglipton
	42	Insulin
	44	Glipizide-Metformin
GDM	1	Preg	6	8	25%
	2	Plas
	3	Pres
	5	Insu
	6	BMI
	8	Age

Abbreviations: DM, diabetes mellitus; GDM, gestational diabetes mellitus; PREG, pregnant; PLAS, plasma; PRES, pressure; INSU, insulin; BMI, body mass index.

Table 5 Comparative Analysis of the Different Feature Selection Models

Feature Selection Methods	Best Cost
AFA-FS	0.034299
GWO-FS	0.052637
PSO-FS	0.042489
GA-FS	0.087642

Abbreviations: AFA-FS, artificial flora algorithm-based feature selection; GWO-FS, grey wolf optimization-based feature selection; PSO-FS, particle swarm optimization-based feature selection; GA-FS, genetic algorithm-based feature selection.

Figure 7 The best cost analysis of different feature selection models. (A) Artificial flora algorithm-based feature selection (AFA-FS). (B) Grey wolf optimization-based feature selection (GWO-FS). (C) Particle swarm optimization-based feature selection (PSO-FS). (D) Genetic algorithm-based feature selection (GA-FS).

Table 6 Analysis of the Confusion Matrix Data

Classes	Types of Diabetes				Total No. of Instances
	Type I	Type II	GDM	Normal
Type I (0)	145	4	3	0	152
Type II (1)	44	77,305	14	0	77,363
GDM (2)	4	10	254	0	268
Normal (3)	4	45	2	23,006	23,057
Total No. of Instances	197	77,364	273	23,006	100,840

Abbreviation: GDM, gestational diabetes mellitus.

Figure 8 Confusion matrix generated for the proposed artificial flora algorithm (AFA)-based feature selection with gradient boosted tree (GBT)-based data classification (AFA-GBT) model.

Figure 9 Precision and recall of the artificial flora algorithm (AFA)-based feature selection with gradient boosted tree (GBT)-based data classification (AFA-GBT) model.

Figure 10 F-score and kappa of the artificial flora algorithm (AFA)-based feature selection with gradient boosted tree (GBT)-based data classification (AFA-GBT) model.

Figure 11 Accuracy of the artificial flora algorithm (AFA)-based feature selection with gradient boosted tree (GBT)-based data classification (AFA-GBT) model.

Table 7 Classification Performance of the Artificial Flora Algorithm (AFA)-Based Feature Selection with Gradient Boosted Tree (GBT)-Based Data Classification (AFA-GBT) Model Based on Different Measures (%)

Classes	Precision	Recall	Accuracy	F-Score	Kappa
Type I	73.60	95.39	99.94	83.09	93.08
Type II	99.92	99.92	99.88	99.92	99.67
GDM	93.04	94.77	99.96	93.90	93.87
Normal	100	99.77	99.94	99.88	99.85
Average	91.64	97.46	99.93	94.19	96.61

Abbreviation: GDM, gestational diabetes mellitus.

Table 8 Comparative Analysis of the Accuracy (%) of Different Classification Models

Classifiers	Accuracy
AFA-GBT	99.93
K-means with LR	95.42
Fuzzy Neural Classifier (FNC)	94.50
Hybrid Prediction Model (HPM)	92.38
Artificial Metaplasticity On Multilayer Perceptron (AMMLP)	89.93
J48 (pruned)	89.30
J48 (unpruned)	86.60
Hybrid Model	84.50
MLP	81.90
LR	78.20
J48	76.70
Stochastic Gradient Descent (SGD)	76.60
ELM	75.72
Naïve Bayes	74.90
BayesNet	74.70
CART	72.80
KNN	67.60

Abbreviations: AFA-GBT, artificial flora algorithm-based feature selection with gradient boosted tree-based data classification; LR, Linear Regression; MLP, multilayer perceptron; ELM, Extreme Learning Machine; CART, Classification and Regression Tree; KNN, k-nearest neighbor.

Sheikhpour R , Sarram MA , Gharaghani S , Chahooki MAZ . A survey on semi-supervised feature selection methods. Pattern Recognit . 2017;64:141–158. doi:10.1016/j.patcog.2016.11.003

 Web of Science ®Google Scholar

Remeseiro B , Bolon-Canedo V . A review of feature selection methods in medical applications. Comput Biol Med . 2019;112:103375. doi:10.1016/j.compbiomed.2019.103375 31382212

 PubMed Web of Science ®Google Scholar

Solorio-Fernández S , Carrasco-Ochoa JA , Martínez-Trinidad JF . A review of unsupervised feature selection methods. Artif Intell Rev . 2020;53(2):907–948.20. doi:10.1007/s10462-019-09682-y

 Web of Science ®Google Scholar

Cheng L , Wu XH , Wang Y . Artificial flora (AF) optimization algorithm. Appl Sci . 2018;8(3):329. doi:10.3390/app8030329

 Google Scholar