Adaptive fusion of K-means region growing with optimized deep features for enhanced LSTM-based multi-disease classification of plant leaves

Figures & data

Table 1. Features and challenges of multi-disease classification of plant leaves.

Author [citation]	Approaches	Upper Hands	Downsides
(Dai et al. 2020)	DATFGAN	This model enhances the classification accuracy It is utilized in real-world benefits by minimizing the number of network constraints.	This model is harder to train for very larger image datasets.
(Sharma et al. 2020)	CNN	This technique attains high accuracy even for large data sets and more types of diseases. It provides remedies for the classified disease.	It suffers from complexity to train the neural network since this model requires large datasets for training. This model has slowed down when processing max-pool-like operations.
(Atila et al. 2021)	EfficientNet deep learning	This method gives high accuracy even for lower-resolution images.	This model is not suitable for rapid plant disease identification using smartphones.
(Zhang et al. 2018)	GoogLeNet and Cifar10	This model has improved recognition and training efficiency.	This model has overfitting problems while using a large number of datasets.
(Liu et al. 2020)	Leaf GAN	This technique has effectively generated sufficient leaf images with prominent lesions.	The accuracy rate downs slightly when it deals with high-resolution images.
(Chen et al. 2020)	GMDH-Logistic method	This model recognizes plant diseases well as well as monitors the growth of plants for future use.	This model causes deviations in the result in the case of slight and moderate disease due to a negligible gap.
(Chohan et al. 2018)	BRBFNN	It provides great computational efficiency for identification and classification.	This model is not improved to detect bacterial or viral diseases in plant leaves.
(Pham et al. 2020)	ANN	This model can be developed in resource-constrained devices.	It requires more trainable parameters for large-size images.

Figure 1. Presentation of an offered multi-disease classification method for plant leaves.

Figure 2. Sample images are taken from dataset 1 with categories of different leaf diseases.

Figure 3. Sample images are taken from dataset 2 for citrus leaves.

Table 2. The statistics of the whole dataset for the suggested method.

Different types of plant leaves	Number of Classes	Classes names	Number of Images
Dataset 1
Apple	4	Apple cedar apple rust, apple healthy, apple Blackrot, and apple scab.	1943
Cherry	2	Cherry-powdery mildew and healthy.	877
Corn	5	Corn healthy, corn northern leaf blight, corn common rust, Corn Cerco sporal leaf spot, gray leaf spot, and.	1829
Grape	4	Grape leaf blight, Blackout, grape esca, and grape healthy.	1805
Peach	2	Peach healthy and peach bacterial spot.	891
Pepper	2	Bacterial spots and healthy leaves.	975
Potato	3	Early blight, late blight, and healthy leaves.	1426
Strawberry	2	Healthy and leaf scorch.	900
Tomato	10	Tomato Early blight, tomato mosaic virus, Tomato late blight, tomato spider mites two-spotted spider mite, tomato target spot, Bacterial spot, Tomato healthy, Tomato septoria leaf spot, Tomato leaf Mold, and tomato yellow leaf curl virus.	4585
Dataset 2
Citrus	5	citrus canker, citrus melanosis, citrus greening, Citrus black spot, and citrus healthy	609

Figure 4. Flow chart of the proposed FSJ-FOA algorithm.

Figure 5. Abnormality segmentation using the AFKMRG fusion model.

Figure 6. Solution encoding for optimal constants in segmentation.

Figure 7. Optimization of constraints in the proposed classification model.

Figure 8. Proposed CNN-based feature extraction and enhanced LSTM-aided classification method.

Figure 9. Experimental results of proposed FSJ-FOA-CNN-LSTM-based multi-disease classification of plant leave with original, pre-processed, segmented, and abnormality segmented images for diverse types of plant.

Figure 14. Sensitive analysis on the parameters considered as (a) Population size and (b) Area-limit of the proposed model.

Table 3. Calculation of the offered scheme on optimization algorithms for different types of plant leaves.

Algorithms	GWO-CNN-LSTM (Sreedharan et al. 2018)	DHOA-CNN-LSTM (Brammya et al. 2019)	FOA-CNN-LSTM (Naz et al. 2019)	JA-CNN-LSTM (Ramesh and Vydeki 2020)	FSJ-FOA-CNN-LSTM
Apple
Average	0.9332	0.9334	0.928	0.96	0.9732
Standard deviation	0.007386	0.014773	0.008602	0.006481	0.0036
Cherry
Average	0.944	0.9424	0.9388	0.9454	0.9788
Standard deviation	0.015925	0.013245	0.01087	0.018906	0.0024
Corn
Average	0.9424	0.9502	0.9432	0.938	0.9754
Standard deviation	0.013306	0.01574	0.013526	0.018974	0.003323
Grape
Average	0.9326	0.9396	0.9628	0.941	0.976
Standard deviation	0.010151	0.016181	0.008495	0.016346	0.003847
Peach
Average	0.9306	0.9618	0.948	0.9432	0.9752
Standard deviation	0.012468	0.006554	0.01858	0.019385	0.004915
Pepper
Average	0.9488	0.9422	0.9522	0.944	0.9786
Standard deviation	0.019208	0.011873	0.010515	0.009633	0.003826
Potato
Average	0.9398	0.9422	0.949	0.9426	0.977
Standard deviation	0.012205	0.009368	0.016912	0.021077	0.003033
Strawberry
Average	0.9378	0.961	0.9344	0.934	0.9742
Standard deviation	0.014538	0.009757	0.016305	0.007457	0.002227
Tomato
Average	0.9424	0.9378	0.9428	0.9394	0.9788
Standard deviation	0.012274	0.020576	0.016881	0.015615	0.002926
Citrus
Average	0.9336	0.9526	0.9424	0.9384	0.9714
Standard deviation	0.011943	0.01758	0.016181	0.017738	0.002728

Table 4. Examination of the proposed multi-disease classification of plant leaves model on classifiers for different types of plant leaves.

Classifiers	NN (Martens et al. 2009)	SVM (Wu et al. 2016)	CNN (Rawat and Wang 2017)	LSTM (Akilan et al. 2020)	CNN-LSTM (Nagda and Poovammal 2018)	FSJ-FOA-CNN-LSTM
Apple
Average	0.913	0.8948	0.9518	0.8998	0.9444	0.9732
Standard deviation	0.020219	0.01828	0.018777	0.019009	0.013351	0.0036
Cherry
Average	0.915	0.921	0.9372	0.9136	0.9424	0.9788
Standard deviation	0.024585	0.018547	0.014274	0.023252	0.021341	0.0024
Corn
Average	0.9052	0.9084	0.9304	0.8998	0.9558	0.9754
Standard deviation	0.022337	0.026635	0.008015	0.020498	0.008232	0.003323
Grape
9Average	0.9062	0.9272	0.9458	0.9162	0.9384	0.976
Standard deviation	0.022194	0.016412	0.013422	0.034184	0.018757	0.003847
Peach
Average	0.9044	0.9172	0.944	0.914	0.9486	0.9752
Standard deviation	0.027215	0.019783	0.014199	0.031235	0.011792	0.004915
Pepper
Average	0.9188	0.9144	0.9384	0.9152	0.9566	0.9786
Standard deviation	0.020827	0.018789	0.013937	0.028519	0.014108	0.003826
potato
Average	0.9002	0.9002	0.9366	0.901	0.9444	0.977
Standard deviation	0.023138	0.026977	0.012532	0.022865	0.014678	0.003033
Strawberry
Average	0.9042	0.9104	0.9518	0.9108	0.9602	0.9742
Standard deviation	0.02801	0.03025	0.013948	0.018659	0.00598	0.002227
Tomato
Average	0.8924	0.8988	0.9344	0.8914	0.9352	0.9788
Standard deviation	0.006184	0.024506	0.012241	0.021001	0.012921	0.002926
Citrus
Average	0.8988	0.9138	0.9312	0.9048	0.9422	0.9714
Standard deviation	0.032536	0.023836	0.014824	0.024498	0.019467	0.002728

Table 5. Calculation of running time for the offered scheme for different types of plant leaves.

Different types of plant leaves	Running time (Sec)
Apple	108.804
Cherry	43.735
Corn	100.493
Grape	100.591
Peach	36.948
Pepper	47.727
Potato	74.001
Strawberry	44.504
Tomato	527.465
Citrus	54.813

Table 6. Total image images of both datasets for the designed model.

Different types of plant leaves	Number of images
Peach	891
Cherry	877
Strawberry	900
Corn	1829
Grape	1805
Potato	1426
Apple	1943
Pepper	975
Tomato	4585
Citrus	609

Figure 15. Graph result for confusion matrix on the suggested plant leaves classification with multiple diseases model.

Table 7. Evaluation of confusion matrix with conventional optimization algorithms for the offered method.

Algorithms	GWO-CNN-LSTM (Sreedharan et al. 2018)	DHOA-CNN-LSTM (Brammya et al. 2019)	FOA-CNN-LSTM (Naz et al. 2019)	JA-CNN-LSTM (Ramesh and Vydeki 2020)	FSJ-FOA-CNN-LSTM
Apple
TP	908	940	905	936	957
TN	19	16	17	17	12
FP	15	16	15	16	14
FN	58	28	63	31	17
Cherry
TP	923	934	920	943	961
TN	16	17	18	16	14
FP	16	19	18	15	14
FN	45	30	44	26	11
Corn
TP	924	924	937	911	964
TN	17	18	18	17	12
FP	15	16	15	16	12
FN	44	42	30	56	12
Grape
TP	902	941	953	922	969
TN	19	15	16	18	12
FP	16	19	18	18	13
FN	63	25	13	42	6
Peach
TP	939	947	943	941	967
TN	16	17	19	19	12
FP	16	18	16	18	12
FN	29	18	22	22	9
Pepper
TP	953	923	937	940	967
TN	19	16	15	15	14
FP	18	19	18	18	14
FN	10	42	30	27	5
Potato
TP	936	910	941	953	968
TN	18	17	19	17	13
FP	19	18	19	17	12
FN	27	55	21	13	7
Strawberry
TP	939	952	903	921	964
TN	17	16	15	17	14
FP	19	17	17	15	13
FN	25	15	65	47	9
Tomato
TP	921	929	943	906	961
TN	19	17	18	18	13
FP	17	17	17	15	12
FN	43	37	22	61	14
Citrus
TP	913	947	907	901	957
TN	16	18	15	17	14
FP	15	16	15	16	13
FN	56	19	63	66	16

Table 8. Estimation of confusion matrix with conventional classifiers for the designed method.

Algorithms	NN (Martens et al. 2009)	SVM (Wu et al. 2016)	CNN (Rawat and Wang 2017)	LSTM (Akilan et al. 2020)	CNN-LSTM (Nagda and Poovammal 2018)	FSJ-FOA-CNN-LSTM
Apple
TP	932	910	905	855	919	957
TN	18	17	19	19	17	12
FP	18	16	15	18	19	14
FN	32	57	61	108	45	17
Cherry
TP	862	917	900	869	903	961
TN	17	18	15	16	19	14
FP	17	15	17	16	15	14
FN	104	50	68	99	63	11
Corn
TP	851	893	921	911	934	964
TN	17	19	17	18	15	12
FP	17	19	16	15	15	12
FN	115	69	46	56	36	12
Grape
TP	850	897	919	858	910	969
TN	17	18	17	15	15	12
FP	16	19	18	19	15	13
FN	117	66	46	108	60	6
Peach
TP	935	895	936	925	941	967
TN	18	16	19	17	18	12
FP	19	18	17	17	16	12
FN	28	71	28	41	25	9
Pepper
TP	890	897	907	936	954	967
TN	17	18	16	18	19	14
FP	17	15	16	16	17	14
FN	76	70	61	30	10	5
Potato
TP	871	918	905	888	913	968
TN	16	19	15	17	19	13
FP	19	16	17	18	16	12
FN	94	47	63	77	52	7
Strawberry
TP	871	876	917	905	946	964
TN	15	19	18	19	18	14
FP	18	17	18	17	16	13
FN	96	88	47	59	20	9
Tomato
TP	876	895	919	858	915	961
TN	15	15	17	15	16	13
FP	19	18	16	15	15	12
FN	90	72	48	112	54	14
Citrus
TP	862	871	908	900	910	957
TN	17	15	16	19	17	14
FP	15	16	17	17	18	13
FN	106	98	59	64	55	16

Table 9. Examination of the offered FSJ-FOA-CNN-LSTM-based multi-disease classification method of plant leaves model.

Metrics	GWO-CNN-LSTM (Sreedharan et al. 2018)	DHOA-CNN-LSTM (Brammya et al. 2019)	FOA-CNN-LSTM (Naz et al. 2019)	JA-CNN-LSTM (Ramesh and Vydeki 2020)	FSJ-FOA-CNN-LSTM
Accuracy	95.42	95.75	96.42	97.02	98.35
Sensitivity	95.40	95.77	96.34	97.04	98.36
Specificity	95.44	95.73	96.51	96.99	98.35
Precision	95.58	95.86	96.61	97.09	98.40
FPR	4.56	4.27	3.49	3.01	1.65
FNR	4.60	4.23	3.66	2.96	1.64
NPV	95.44	95.73	96.51	96.99	98.35
FDR	4.42	4.14	3.39	2.91	1.60
F1-Score	95.49	95.82	96.47	97.06	98.38
MCC	90.84	91.50	92.84	94.03	96.71

Table 10. Validation of the offered FSJ-FOA-CNN-LSTM-based multi-disease classification method of plant leaves model with classifiers.

Metrics	NN (Martens et al. 2009)	SVM (Wu et al. 2016)	CNN (Rawat and Wang 2017)	LSTM (Akilan et al. 2020)	CNN-LSTM (Nagda and Poovammal 2018)	FSJ-FOA-CNN-LSTM
Accuracy	95.49	96.18	96.71	97.35	97.71	98.35
Sensitivity	95.49	96.15	96.71	97.32	97.70	98.36
Specificity	95.49	96.22	96.70	97.38	97.72	98.35
Precision	95.63	96.33	96.80	97.46	97.79	98.40
FPR	4.51	3.78	3.30	2.62	2.28	1.65
FNR	4.51	3.85	3.29	2.68	2.30	1.64
NPV	95.49	96.22	96.70	97.38	97.72	98.35
FDR	4.37	3.67	3.20	2.54	2.21	1.60
F1-Score	95.56	96.24	96.76	97.39	97.74	98.38
MCC	90.98	92.36	93.41	94.70	95.42	96.71

Figure 16. Training accuracy graph of the suggested plant leaves classification with multiple diseases model (a) Model accuracy (b) Model loss.

Dai Q, Cheng X, Qiao Y, Zhang Y. 2020. Crop leaf disease image super-resolution and identification with dual attention and topology fusion generative adversarial network. Digit Object Identifier. 8:55724–55735.

 Google Scholar

Sharma, et al. 2020. Classification of plant leaf diseases using machine learning and image pre-processing techniques. In 10th International Conference on Cloud Computing, Data Science & Engineering (Confluence); p. 480–484.

 Google Scholar

Atila U, Ucar M, Akyol K, Ucar E. 2021. Plant leaf disease classification using EfficientNet deep learning model. Ecol Inf. 61:101182.

 Web of Science ®Google Scholar

Zhang, et al. 2018. Identification of maize leaf diseases using improved deep convolutional neural networks. Inst Electr Electron Eng Access. 6:30370–30377.

 Google Scholar

Liu B, Tan C, Li S, He J, Wang H. 2020. A data augmentation method based on generative adversarial networks for grape leaf disease identification. Instf Electr Electron Eng Access. 8:102188–102198.

 Google Scholar

Chen J, Yin H, Zhang D. 2020. A self-adaptive classification method for plant disease detection using GMDH-Logistic model. Sustain Comput: Inform Syst. 28:100415.

 Web of Science ®Google Scholar

Chouhan SS, Kaul A, Singh UP, Jain S. 2018. Bacterial Foraging Optimization Based Radial Basis Function Neural Network (BRBFNN) for identification and classification of plant leaf diseases: An automatic approach towards plant pathology, Institute of Electrical and Electronics Engineers Access, 6, 8852–8863.

 Google Scholar

Pham TN, Tran LV, Dao SVT. 2020. Early disease classification of mango leaves using feed-forward neural network and hybrid metaheuristic feature selection. Inst Electr Electron Eng Access. 8:189960–189973.

 Google Scholar

Sreedharan, et al. 2018. Grey Wolf optimization-based feature selection and classification for facial emotion recognition. IET Biom. 7(5):490–499.

 Web of Science ®Google Scholar

Brammya G, Praveena S, Preetha NSN, Ramya R, Rajakumar BR, Binu D. 2019. Deer hunting optimization algorithm: a new nature-inspired meta-heuristic paradigm. The Computer Journal.

 Google Scholar

Naz M, Zafar K, Khan A. 2019. Ensemble based classification of sentiments using forest optimization algorithm. Multidiscip Digit Publ Inst. 4(2):76.

 Google Scholar

Ramesh S, Vydeki D. 2020. Recognition and classification of paddy leaf diseases using Optimized Deep Neural network with Jaya algorithm. Inform Process Agric. 7(2):249–260.

 Google Scholar

Martens D, Baesens BB, Gestel TV. 2009. Decompositional rule extraction from support vector machines by active learning. IEEE Trans Knowl Data Eng. 21(2):178–191.

 Web of Science ®Google Scholar

Wu D, Pigou L, Kindermans P-J, Le ND-H, Shao L, Dambre J, Odobez J-M. 2016. Deep dynamic neural networks for multimodal gesture segmentation and recognition. IEEE Trans Pattern Anal Mach Intell. 38(8):1583–1597.

 PubMed Web of Science ®Google Scholar

Rawat W, Wang Z. 2017. Deep convolutional neural networks for image classification: a comprehensive review. Neural Comput. 29(9):2352–2449.

 PubMed Web of Science ®Google Scholar

Akilan T, Wu QJ, Safaei A, Huo J, Yang Y. 2020. A 3D CNN-LSTM-based image-to-image foreground segmentation. IEEE Trans Intell Transport Syst. 21(3):959–971.

 Web of Science ®Google Scholar

Nagda M, Poovammal E. 2018. Image classification using a hybrid LSTM-CNN deep neural network. Soc Netw Anal Min. 8(6):1342–1348.

 Google Scholar

Data availability statement

The analysis of multi-disease detection of plant leaves is performed by using different leaf images gathered from two diverse datasets such as Kaggle and Mendeley.