A Hybrid Feature Extraction and Classification using Xception-RF for Multiclass Disease Classification in Plant Leaves

Figures & data

Table 1. A comparison of studies pertaining to plant classification and algorithms.

S.No	Reference	Models Used	Database	Performance and Limitation
1	Gayathri, Gopi, and Palanisamy (2021)	Multipath Convolutional Neural Network (M-CNN) with e J48 classifier	IDRiD Kaggle MESSIDOR	This work achieved a high level of accuracy of 99.62%. Some limitations include taking up more time and overfitting. There are no detailed dataset name links or descriptions available.
2	(Lamba, Gigras, and Dhull 2021)	Auto-Color Correlogram as image filter, DL as classifiers	Rice dataset and the Kaggle PlantVillage dataset.	The accuracy for binary class is 99.4%, while the accuracy for multiclass is 99.2%. Training DL classifiers takes time, depending on the number of epochs.
3	(Toda and Okura 2019)	Convolutional neural networks (CNNs)	PlantVillage dataset	99% have been achieved. Because the PlantVillage dataset was obtained in a controlled environment, the model must be validated using alternative datasets or real-time data.
4	(Abdu, Mohd Mokji, and Ullah Sheikh 2020)	Color coherence vector (CCV) and local binary pattern (LPB) for FE with Machine learning classifiers	PlantVillage	This work has given an accuracy of 99%. As PlantVillage dataset has been collected in a controlled environment, the model has to be tested with other datasets or real-time data.
5	(Gokulnath and Usha Devi 2021)	LF-CNN	PlantVillage Database	This work provides an accuracy of 98.93%. PlantVillage dataset was taken in a controlled environment and it is a commonly used dataset. The main limitation is that the model isn’t been tested under different conditions such as illumination, occlusion, pose variation, lightning factors etc.
6	(Zhao and Liu 2020)	CNN	MINST dataset	The experimental results show that the classifiers fusion can achieve the classifcation accuracy of≥98%. CNN classifiers slow down the training time with respect to number of epochs. Hence ML hybrid methods could be considered.
7	(Hassan et al. 2021)	Deep convolutional-neural-network (CNN)	PlantVillage dataset	The performance accuracy was found to be 98% and above. The convolutional neural network classifier doesn’t give higher performance.
8	(Mohameth, Bingcai, and Amath Sada 2020)	CNN-SVM	PlantVillage	An accuracy of 97.82% have been attained. Collecting the own dataset and comparing it to the PlantVillage dataset will provide best efficiency of the model.
9	(Mahum et al. 2021)	CNN through Local Binary Patterns (LBP) and Histogram of oriented gradient (HOG) with SVM, RF, and KNN	Mendeley dataset IV	Achieved an accuracy of 97%. LBP and HOG is ineffective at capturing discriminating characteristics which slow down the recognition speed especially on large-scale face database.
10	(Vijayalakshmi and Joseph Peter 2021)	Convolution neural network (CNN)	Fruits 360 dataset	Have achieved an accuracy of 96.98%. Could infuse ML classifiers as it is fast and increases the performance.
11	(Li et al. 2019)	DCNR: deep cube CNN with random forest	Images from KSC (Kennedy Space Center), PU (Pavia University) and IP (Indian Pines)	96.78%, 96.08% and 94.85% accuracy was obtained on KSC, IP and SA datasets. As DCNR is trained with random sampling instead of active learning may lead to the loss of certain key pixels. Moreover, the time cost in DCNR training is much higher than CCS model.
12	(Jadhav, Udupi, and Patil 2019)	AlexNet, GoogleNet, VGG16, ResNet101, and DensNet201	PlantVillage database	95%, 96.4%, 96.4%, 92.1%, 93.6%. Attempt to develop our own CNN model in order to increase the performance rate of the model by varying the minibatch size, bias learning rate, and weight.
13	(Kumar, Chaudhary, and Khaitan Chandra 2021)	Neural Network: Faster Region-based CNN (Faster R-CNN), Region-based Fully CNN(R-CNN) and Single shot Multibook Detector (SSD)	PlantVillage Dataset	Accuracy of 94.6% have been attained but the network may take a long time to attain convergence because all samples from a single image may be correlated (i.e. their features are similar).
14	(Yang et al. 2020)	CNN-RF	Sentinel-2 images	94.27% have been obtained. Performance could be improved. Because clouds can disrupt Sentinel-2 data, future work could focus on establishing a multi-source remote sensing imagery fusion strategy for crop categorization.
15	(Li, Nie, and Chao 2020)	SCNN-KSVM (Shallow CNN with Kernel SVM) and SCNN-RF (Shallow CNN with RF)	Maize dataset, Apple dataset and Grape dataset from PlantVillage datset	This work obtaines an accuracy of 94%. As PlantVillage is commonly used dataset, and taken under controlled environments, the accuracy could have been improved.
16	(Singh et al. 2020)	Convolutional Neural Network (CNN)- Support Vector Machine (SVM)	DREBIN dataset	92.59% accuracy has been achieved. The scope of this experiment was limited to evaluating the performance of the proposed model using malware images. It could be tested with another dataset to know the model’s actual performance.
17	(Scarpa et al. 2018)	Convolutional neural network (CNN)	Sentinel-1, Sentinel-2, SAR image	90% obtained from the average of five months. Accuracy could be increased by changing the classifier to ML.
18	(Barbedo 2019)	CNN	Manually collected images(OWN).	Above 75% in most of the cases. Requires more time and performsnce could be improved.
19	(Kwak et al. 2021)	CNN-random forest (CNN-RF)	UAV images using eBee unmanned aerial system	Overall accuracy increased by maximum 6.7%p and 4.6%p in two study areas. As no proper accuracy value have been mentioned, the exact performance is unknown.
20	(Simonyan and Zisserman 2014)	Convolutional networks (ConvNets)	ILSVRC-2012 dataset	Have provided performance results as top-1 validation error of 23.7, top-5 validation error of 6.8 and top-5 test error of 6.8. Could be tried with other datasets and could be explicit in displaying the accurate accuracy/performance of the model.
21	(Xie et al. 2022)	Convolutional neural network (CNN), SVM, RF	Case Western Reserve University (CWRU) bearing fault datasets	Increase of about 1–2% accuracy compared to traditional methods. Up to 99% in some cases. Deep learning models with high interpretability could be considered.

Figure 1. Sample images from the dataset.

Figure 2. Overall workflow diagram.

Figure 3. Workflow of proposed method.

Figure 4. Working of Xception model.

Figure 5. RF Architecture.

Figure 6. Confusion matrix with CNN classifier.

Figure 7. Confusion matrix with RF classifier.

Figure 8. Accuracy comparison of the models.

Figure 9. F1 Score, Precision and Recall.

Table 2. Comparison with existing hybrid works of DL+ML.

Models	Accuracy
Multipath-CNN(M-CNN) + J48 classifier	99.62%
CNN+SVM	97.82%
DCNR: deep cube CNN + RF	96.78%
CNN-RF	94.27%
SCNN-KSVM (Shallow CNN with Kernel SVM)	94%
CNN+SVM	92.59%
Proposed work	100%

Table 3. Accuracy percentage comparison of classifiers.

Pre-trained CNN	CNN CLASSIFIER	RF
VGG 16	50.2857%	98.85%
RESNET	46.857%	97.14%
MOBILENET	58.28%	98.85%
INCEPTIONRESNETV2	57.142%	99.42%
INCEPTION	51.999%	98.28%
XCEPTION	58.714%	100%

Table 4. F1 score, precision, and recall of pretrained models with RF.

Hybrid combinations	F1 SCORE	PRECISION	RECALL
VGG 16+RF	0.989	0.987	0.989
RESNET+RF	0.971	0.973	0.975
MOBILENET+RF	0.989	0.990	0.990
INCEPTIONRESNETV2+RF	0.994	0.995	0.995
INCEPTION+RF	0.983	0.983	0.983
XCEPTION+RF	1.000	1.000	1.000

Gayathri, S., V. P. Gopi, and P. Palanisamy. 2021. Diabetic retinopathy classification based on multipath CNN and machine learning classifiers. Physical and Engineering Sciences in Medicine 44 (3):639–53. doi:10.1007/s13246-021-01012-3.

 PubMedGoogle Scholar

Lamba, M., Y. Gigras, and A. Dhull. 2021. Classification of plant diseases using machine and deep learning. Open Computer Science 11 (1):491–508. doi:10.1515/comp-2020-0122.

 Web of Science ®Google Scholar

Toda, Y., and F. Okura. 2019. How convolutional neural networks diagnose plant disease. Plant Phenomics 2019. doi:10.34133/2019/9237136.

 PubMedGoogle Scholar

Abdu, A. M., M. Mohd Mokji, and U. Ullah Sheikh. 2020. Automatic vegetable disease identification approach using individual lesion features. Computers and Electronics in Agriculture 176:105660. doi:10.1016/j.compag.2020.105660.

 Web of Science ®Google Scholar

Gokulnath, B. V., and G. Usha Devi. 2021. Identifying and classifying plant disease using resilient LF-CNN. Ecological Informatics 63:101283. doi:10.1016/j.ecoinf.2021.101283.

 Web of Science ®Google Scholar

Zhao, H.H., and H. Liu. 2020. Multiple classifiers fusion and CNN feature extraction for handwritten digits recognition. Granular Computing 5 (3):411–18. doi:10.1007/s41066-019-00158-6.

 Google Scholar

Hassan, S. M., A. Kumar Maji, M. Jasiński, Z. Leonowicz, and E. Jasińska. 2021. Identification of plant-leaf diseases using CNN and transfer-learning approach. Electronics 10 (12):1388. doi:10.3390/electronics10121388.

 Web of Science ®Google Scholar

Mohameth, F., C. Bingcai, and K. Amath Sada. 2020. Plant disease detection with deep learning and feature extraction using plant village. Journal of Computer and Communications 8 (6):10–22. doi:10.4236/jcc.2020.86002.

 Google Scholar

Mahum, R., S. Ur Rehman, T. Meraj, H. Tayyab Rauf, A. Irtaza, A. M. El-Sherbeeny, and M. A. El-Meligy. 2021. A novel hybrid approach based on deep cnn features to detect knee osteoarthritis. Sensors 21 (18):6189. doi:10.3390/s21186189.

 PubMed Web of Science ®Google Scholar

Vijayalakshmi, M., and V. Joseph Peter. 2021. CNN based approach for identifying banana species from fruits. International Journal of Information Technology 13 (1):27–32. doi:10.1007/s41870-020-00554-1.

 Google Scholar

Li, T., J. Leng, L. Kong, S. Guo, G. Bai, and K. Wang. 2019. DCNR: Deep cube CNN with random forest for hyperspectral image classification. Multimedia Tools and Applications 78 (3):3411–33. doi:10.1007/s11042-018-5986-5.

 Web of Science ®Google Scholar

Jadhav, S. B., V. R. Udupi, and S. B. Patil. 2019. Convolutional neural networks for leaf image-based plant disease classification. IAES International Journal of Artificial Intelligence 8 (4):328. doi:10.11591/ijai.v8.i4.pp328-341.

 Google Scholar

Kumar, S., V. Chaudhary, and S. Khaitan Chandra. 2021. Plant disease detection using CNN. Turkish Journal of Computer and Mathematics Education (TURCOMAT) 12 (12):2106–12.

 Google Scholar

Yang, S., L. Gu, X. Li, T. Jiang, and R. Ren. 2020. Crop classification method based on optimal feature selection and hybrid CNN-RF networks for multi-temporal remote sensing imagery. Remote Sensing 12 (19):3119. doi:10.3390/rs12193119.

 Web of Science ®Google Scholar

Li, Y., J. Nie, and X. Chao. 2020. Do we really need deep CNN for plant diseases identification? Computers and Electronics in Agriculture 178:105803. doi:10.1016/j.compag.2020.105803.

 Web of Science ®Google Scholar

Singh, J., D. Thakur, F. Ali, T. Gera, and K. Sup Kwak. 2020. Deep feature extraction and classification of android malware images. Sensors 20 (24):7013. doi:10.3390/s20247013.

 PubMed Web of Science ®Google Scholar

Scarpa, G., M. Gargiulo, A. Mazza, and R. Gaetano. 2018. A CNN-based fusion method for feature extraction from sentinel data. Remote Sensing 10 (2):236. doi:10.3390/rs10020236.

 Web of Science ®Google Scholar

Barbedo, J. G. A. 2019. Plant disease identification from individual lesions and spots using deep learning. Biosystems Engineering 180:96–107. doi:10.1016/j.biosystemseng.2019.02.002.

 Web of Science ®Google Scholar

Kwak, G.H., C.W. Park, K.D. Lee, S. -I. Na, H.Y. Ahn, and N.W. Park. 2021. Potential of hybrid CNN-RF model for early crop mapping with limited input data. Remote Sensing 13 (9):1629. doi:10.3390/rs13091629.

 Web of Science ®Google Scholar

Simonyan, K., and A. Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv: 14091556 1 1–14 .

 Google Scholar

Xie, W., Z. Li, Y. Xu, P. Gardoni, and W. Li. 2022. Evaluation of different bearing fault classifiers in utilizing CNN feature extraction ability. Sensors 22 (9):3314. doi:10.3390/s22093314.

 PubMed Web of Science ®Google Scholar