Adaptive fusion of K-means region growing with optimized deep features for enhanced LSTM-based multi-disease classification of plant leaves

Abstract

The manual process of plant leaves disease detection takes more time to perform. To achieve successful classification results, a flawless feature extraction process is required for a detection model. Aiming at the localization of diseased-plant leaves, this paper performs complex tasks like segmentation, and multi-disease classification of plant leaves using ‘improved segmentation, feature extraction, and classification’ models. Here, the Adaptive Fusion of K-Means Region Growing (AFKMRG) accomplishes the abnormality segmentation of leaves. The extracted features are subjected to the Enhanced Long short-term memory (LSTM) for performing the multi-disease classification. Here, the segmentation, classification, and feature extraction are improved by the Fitness Sorted Jaya-Forest Optimization Algorithm (FSJ-FOA). From the empirical results, the accuracy and precision of the designed method is attains 98.35% and 98.40% for all kinds of leaves. Results show that the designed multi-disease classification of plant leaves method provides elevated performance with diverse performance metrics.

Keywords:

• Multi-disease classification
• adaptive fusion of K-means region growing
• deep learning
• fitness sorted-based Jaya and forest optimization algorithm

1. Introduction

The development of the country is mainly based on its agricultural practices in it. So, it is necessary to maintain the crops as healthier as possible (Hang et al. 2019). The seasonal variations may cause some defects to the plant leaves, which affect the growth of plants. Multiple types of diseases can affect the plant leaves and the most common types of diseases among them are dwarf mosaic, brown spot, curvulria leaf spot, gray leaf spot, northern leaf bright, round spot, rust, and southern leaf bright (Taujuddin et al. 2020). Initially, these diseases gradually affect the plant leaves and further affect the entire plant thereby reducing the quality and quantity of the crops. The identification and classification of these diseases of each plant leaf are hard for the normal human eye since there are an enormous amount of plants on a single farm (Wiesner-Hanks and Nelson 2016). However, it is essential to identify each plant since a single infected leaf of a plant can spread the disease gradually to the whole plant. The use of pesticides without awareness may spoil the land, and hiring professionals may be expensive for the farmers (Prasad et al. 2016). It is essential to detect the exact diseases for preventing them from further spreading. Therefore, the primary source of disease detection is routine monitoring of plant leaves by using automated plant leaf disease detection technology (Min and Htun 2018; Liang et al. 2019).

The multi-disease detection and classification of plant leaves should be done accurately in a short period to prevent unnecessary waste of resources. Advanced analysis is not possible when the disease affected the plant leaves without any explicit symptoms. Still, the maximum number of plants reveal explicit symptoms, which are diagnosed by the plant pathologist based on the optical observations of the affected leaves of plants (Sileshi et al. 2016; Simmi et al. 2020). However, plant pathologists face some difficulties in the accurate prediction of plant disease due to the availability of an excessive variety of plants and the earlier spread of diseases during seasonal changes. The detection of characteristic symptoms of plant leaves diseases is hard for pathologists due to a lack of skills in observation and unawareness of certain diseases (Singh and Misra 2017; Goodridge et al. 2017).

In (Dai et al. 2020) have developed a Generative Adversarial Network with Dual-Attention and Topology-Fusion mechanisms termed DATFGAN. In (Sharma et al. 2020) have proposed a new approach with artificial intelligence for the detection and classification of leaf diseases in an automatic manner. In (Atila et al. 2021) have developed an architecture based on deep learning called EfficientNet for the classification of disease leaves. In (Zhang et al. 2018) have developed two improved deep learning-based models such as improved GoogLeNet and Cifar10 models for highly accurate maize leaf disease detection. In (Liu et al. 2020) have developed a new approach based on GANs named LeafGAN to generate four dissimilar grape leaf disease images for training the detection models. The proposed model has provided a feasible solution for data augmentation of images and generated adequate leaf images. In (Chen et al. 2020) have proposed a new approach based on image processing techniques for the automatic detection and classification of plant leaf diseases. In (Singh et al. 2018) have introduced a method namely Bacterial foraging optimization based on Radial Basis Function Neural Network (BRBFNN). In (Pham et al. 2020) have proposed a wrapper-based feature selection approach using an adaptive particle-grey wolf hybrid meta-heuristic algorithm.

In recent decades, the automated detection and classification of multiple diseases of plant leaves have been executed by various classical machine learning models. However, the models are designed through limited-resolution images. Therefore, deep learning methods provide great benefits in multi-disease detection and classification of plant leaves with less time and high accuracy. The deep learning methods are used to extract optimal feature representations from a large image dataset and also ensure the scope of computer vision in the precision agriculture field (Vellaichamy et al. 2021; Krishnamoorthy et al. 2021). CNN-based model is trained for identifying plant leaf diseases (Ramesh and Vydeki 2020). The CNN-based models improve the identification accuracy based on modifying the ratio of the size of the training set to the testing set al.though the CNN models have obtained better results, the recognition rate is reduced by the usage of more parameters and longer training convergence times (Hang et al. 2019; Gokulnath and Devi 2021). AlexNet model is implemented to combine global pooling and batch normalization for the identification of various leaf diseases, and these studies based on CNN models define the effectiveness of Deformable Convolution Neural Networks (DCNNs) to identify leaf diseases (Naz et al. 2019). However, the images obtained from the farms have poor quality, which leads to a reduced accuracy rate of pre-trained classifiers that are already trained based on high-quality image datasets (Sachdeva et al. 2021). Hence, the multi-disease leaf detection and classification uses several digital image processing techniques. However, these methods cannot achieve a high recognition rate when dealing with a large number of samples (Sachdeva et al. 2021). Therefore, it is essential to introduce improved methodologies for the efficient detection and classification of plant leaves that are affected by multiple diseases.

The multi-disease classification of plant leaves using traditional methods is time-consuming and needs more resources. The features of the numerous techniques for enhanced prediction of plant-leaves disease are reviewed below. This model enhances classification accuracy. It is used in real-world benefits by minimizing the number of network constraints. This technique attains high accuracy even for large data sets and more types of diseases. It provides remedies for the classified disease. This method gives high accuracy even for lower-resolution images. This model has improved recognition and training efficiency. This technique has effectively generated sufficient leaf images with prominent lesions. This model recognizes plant diseases well as well as monitors the growth of plants for future use. It provides great computational efficiency for identification and classification. This model can be implemented in resource-constrained devices like smartphones. The challenges for the various existing multi-disease classification of plant leaves are given below. This model is harder to train for very larger image datasets. It suffers from complexity to train the neural network since this model requires large datasets for training. This model has slowed down when processing max-pool-like operations. This model is not suitable for rapid plant disease identification using smartphones. This model has overfitting problems while using a large number of datasets. The accuracy rate downs slightly when it deals with high-resolution images. This model causes deviations in the result in the case of slight and moderate disease due to a negligible gap. This model is not improved to detect bacterial or viral diseases in plant leaves. It requires more trainable parameters for large-size images. The research based on optimal weighted features for existing and proposed work is described given below. The optimization of the weight function is done in such a way that the precision and recall should be maximized after image retrieval. (Darapureddy et al. 2021). The Radiomics-based strategies, the histogram, and Gray-Level Co-Occurrence Matrix (GLCM)-based radionics features were extracted from cancerous (Zhang et al. 2017). The T1-weighted images extract the local features of each brain tissue. This pre-processing is done using Statistical Parametric Mapping (SPM) and Voxel-Based Morphometry (VBM) (Kondo et al. 2015). In the proposed work, the deep features are acquired using the pooling layer of the CNN technique by optimizing the hidden neurons of CNN using a new heuristic algorithm for maximizing the performance.

The major contribution of the multi-disease classification of plant leaves is represented as follows:

• To employ the classification of multiple diseases of different types of plant leaves like strawberry, grape, apple, peach, cherry, pepper, potato, citrus, and tomato leaves by adopting enhanced abnormality segmentation, deep feature extraction, and enhanced classification model with the purpose of hybridized optimization-based algorithm.

• To implement the leaf segmentation using binary thresholding and abnormality segmentation process by AFKMRG technique using FSJ-FOA algorithm. The abnormality-segmented image is produced by combining the outcomes of both K-means and RG after being multiplied with fusion weights that are optimized by the FSJ-FOA algorithm. It increases the quality of segmented images.

• To develop a novel FSJ-FOA model for improving the quality of segmented images by optimizing the fusion constants in segmentation and to improve the efficiency of classification by optimizing the hidden neurons of CNN with the aim of significant feature extraction from the pooling layer and by optimizing the hidden neurons of LSTM for classifying the multi-disease in plant leaves.

• To evaluate the effectiveness of the recommended multi-disease classification of plant leaves model on trained datasets using different performance measures through comparing with conventional classifiers and meta-heuristic algorithms.

2. Literature review

2.1. Related works

(Dai et al. 2020) have developed a Generative Adversarial Network with Dual-Attention and Topology-Fusion mechanisms termed DATFGAN. This network was used to attain clear and high-resolution images from unclear images. Moreover, the count of parameters was reduced using the weight-sharing scheme. The proposed network has attained better results. The performance evaluation on the proposed network has provided robust performance for real-world applications.

(Sharma et al. 2020) have proposed a new approach with artificial intelligence for the detection and classification of leaf diseases in an automatic manner. This approach was used for easy and quick detection of disease and then it performed efficient classification, thereby observing the essential remedies for curing that plant diseases (Sreedevi and Manike 2022). The primary goal of this approach was to improve the yield of crops. In this approach, various steps like image collection, pre-processing, segmentation, and classification were done based on several machine learning techniques. The experimental analysis of this approach has shown improved detection and classification performance.

(Atila et al. 2021) have developed an architecture based on deep learning called EfficientNet for the classification of disease leaves. The transfer learning approach was utilized to learn the proposed method and other deep-structured architectures. The experimental results have shown that EfficientNet architecture has attained better values than the original and augmented datasets of other deep learning models owing to accuracy and precision.

(Zhang et al. 2018) have developed two improved deep learning-based models such as improved GoogLeNet and Cifar10 models for highly accurate maize leaf disease detection. The experimental results have shown that the improved models have attained better efficiency than the VGG and AlexNet structures in terms of accuracy, and it has also provided a fast convergence rate.

(Liu et al. 2020) have developed a new approach based on GANs named LeafGAN to generate four dissimilar grape leaf disease images for training the detection models. The proposed model has provided a suitable solution for augmentation with data of images and generated adequate leaf images. This data augmentation method has solved the overfitting problems and enhanced the accuracy of the proposed disease detection model.

(Chen et al. 2020) have proposed a new approach based on image processing techniques for the automatic detection and classification of plant leaf diseases. The segmentation process was performed by the grey variance method to spot the diseased regions without any background images. The features such as texture, color, area, and pixels from the diseased part were extracted as feature indicators. Finally, the classification performance was done. The offered method gives better outcomes. The results of the comparison have revealed that the novel method was effective in terms of detecting the diseased plant.

(Chohan et al. 2018) have introduced a method namely Bacterial foraging optimization based Radial Basis Function Neural Network (BRBFNN). The feature extraction step was employed by the region-growing algorithm to improve the development of the proposed network. The experimental results of the proposed method have attained higher identification and classification accuracy.

(Pham et al. 2020) have proposed a wrapper-based feature selection approach using an adaptive particle-grey wolf hybrid meta-heuristic algorithm. At first, the pre-processing method was employed and the feature extraction process was done by a wrapper-based algorithm. The selected features were fed into Artificial Neural Network (ANN) for the effective process of detection. The classification approach of the proposed model was compared with other approaches like CNN models and the results have shown that the ANN was better than the CNN in terms of classification accuracy.

(Islam et al. 2020) have presented an online machine-learning aid for detecting papaya disease. Here, images were gathered via a mobile app and transferred into the system for disease detection. (Islam et al. 2021) have worked on 4 kinds of diseases and one healthy leaf class of the paddy. In the end, it has been revealed that the suggested method achieves a higher accuracy rate. (Hossen et al. 2020) have employed the improved deep structured method to detect and categorize papaya disease. It was reliable with fully connected, where categorization was completed with the help of deeply structured architecture.

Krishnamoorthi and Vijayaganth (2022) have developed a new ensemble learning method with the combination of soft computing and deep structured architectures for the multi-disease classification of plant leaves. In this work, the suggested structure of ensemble learning approaches has been customized for optimizing the constraints of classifiers for attaining accurately categorized outcomes. In the end, the simulation findings have demonstrated that the offered method attains high accuracy with diverse efficiency metrics.

(Tiwari et al. 2022) has implemented automated and more efficient plant disease detection approaches for diagnosing early symptoms to avoid economic loss. The categorization and detection of plant diseases from leaf images have been done with the help of a deep-learning-based technique. In the end, the experimental results have revealed the efficient performance of the offered method.

Hyeon et al. (2022) have introduced an enhanced deep learning-based multi-plant disease recognition method that integrates deep features acquired by deep convolutional neural networks and k-nearest neighbors to get the disease images with the help of query images. The fine-tuning approach was utilized to leverage the deep features. The experimentation has been done for achieving a better accuracy rate.

2.2. Problem formulation

The multi-disease classification of plant leaves with the help of traditional methods is time-consuming and needs more resources. Therefore, various approaches for enhanced prediction of plant-leaves disease are reviewed in Table 1.

Table 1. Features and challenges of multi-disease classification of plant leaves.

Author [citation]	Approaches	Upper Hands	Downsides
(Dai et al. 2020)	DATFGAN	This model enhances the classification accuracy It is utilized in real-world benefits by minimizing the number of network constraints.	This model is harder to train for very larger image datasets.
(Sharma et al. 2020)	CNN	This technique attains high accuracy even for large data sets and more types of diseases. It provides remedies for the classified disease.	It suffers from complexity to train the neural network since this model requires large datasets for training. This model has slowed down when processing max-pool-like operations.
(Atila et al. 2021)	EfficientNet deep learning	This method gives high accuracy even for lower-resolution images.	This model is not suitable for rapid plant disease identification using smartphones.
(Zhang et al. 2018)	GoogLeNet and Cifar10	This model has improved recognition and training efficiency.	This model has overfitting problems while using a large number of datasets.
(Liu et al. 2020)	Leaf GAN	This technique has effectively generated sufficient leaf images with prominent lesions.	The accuracy rate downs slightly when it deals with high-resolution images.
(Chen et al. 2020)	GMDH-Logistic method	This model recognizes plant diseases well as well as monitors the growth of plants for future use.	This model causes deviations in the result in the case of slight and moderate disease due to a negligible gap.
(Chohan et al. 2018)	BRBFNN	It provides great computational efficiency for identification and classification.	This model is not improved to detect bacterial or viral diseases in plant leaves.
(Pham et al. 2020)	ANN	This model can be developed in resource-constrained devices.	It requires more trainable parameters for large-size images.

3. Proposed method for multi-disease classification of plant leaves: meta-heuristic improvement

3.1. Designed architecture

The schematic representation of the recommended scheme is shown in Figure 1. The proposed multi-disease classification of the plant leaves model consists of three major phases 1. Pre-processing, 2(a). Leaf segmentation, 2(b) Abnormality segmentation, and 3. Classification. The acquired images of plant leaves from the different datasets are fed to the pre-processing phase to suppress the undesired distortions and enhance the relevant features of the image for further processing of classification and segmentation. The pre-processing of images is executed by using scaling and threshold development approaches. The pre-processed images are further subjected to leaf segmentation, thereby providing a simpler analysis with the proper region of interest.

The leaf segmentation process (Shanthakumari et al. 2022; Yeong et al. 2022) is performed by the binary thresholding method and the abnormality segmentation phase is carried out by a combined method namely AFKMRG, which consists of K-means and the Region growing method. The outputs produced from both methods are multiplied with the fusion constants separately, which are attained by FSJ-FOA. The enhanced AFKMRG-based abnormality segmented images are then forwarded to the classification stage, which is done by CNN-based feature extraction and enhanced LSTM classifier. The feature extraction process using CNN is improved by optimizing the hidden neurons through FSJ-FOA.

Figure 1. Presentation of an offered multi-disease classification method for plant leaves.

The CNN-based acquired features are taken as input to the LSTM classifier, in which the number of suitably hidden neurons is tuned by FSJ-FOA. The classified outputs from the enhanced LSTM classification are attained from different types of leaf diseases. Here, the enhanced detection (Ramesh et al. 2018) is performed by optimizing the hidden neurons of both CNN and LSTM using FSJ-FOA by solving the objective function with maximization of accuracy. Therefore, the process of classification, segmentation, and feature extraction is enhanced by a hybrid optimization algorithm called FSJ-FOA.

3.2. Dataset description

The analysis of multi-disease detection of plant leaves is performed by using various leaf images gathered from two diverse datasets like Mendeley and Kaggle. The Kaggle dataset contains the plant leaves like strawberry, grape, tomato, cherry, potato, peach, pepper, corn, and apple. The Mendeley dataset contains citrus plant leaves.

3.2.1. Dataset 1

It is collected from the link, ‘https://www.kaggle.com/vipoooool/new-plant-diseases-dataset’. The Kaggle dataset contains 38 various classes of disease-affected and unaffected leaves. The average of images comprises 87K RGB. The whole dataset is split into validation and training datasets in the proportion of 20 and 80, respectively. Additionally, the detection process uses a new directory, which contains 33 test images.

The sample images of plant leaves collected from the Kaggle dataset are represented in Figure 2.

Figure 2. Sample images are taken from dataset 1 with categories of different leaf diseases.

3.2.1. Dataset 2

The Mendeley dataset composes a set of images of healthy and unhealthy citrus leaves, which has 609 numbers of total images. To prevent the plants from disease, his dataset may be used for future research.

Finally, citrus (Iqbal et al. 2018) consists of classes like citrus black spot, citrus canker, citrus greening, citrus melanosis, and citrus healthy. The sample images of plant leaves gathered from the Mendeley dataset are represented in Figure 3.

Figure 3. Sample images are taken from dataset 2 for citrus leaves.

The input images of plant leaves are collected from the different datasets presented as $P{L}_{in},$ where $in=1,2,\dots ,IN$ and $IN$ refer to the average count of leaf images. The statistics of the whole dataset are given in Table 2.

Table 2. The statistics of the whole dataset for the suggested method.

Different types of plant leaves	Number of Classes	Classes names	Number of Images
Dataset 1
Apple	4	Apple cedar apple rust, apple healthy, apple Blackrot, and apple scab.	1943
Cherry	2	Cherry-powdery mildew and healthy.	877
Corn	5	Corn healthy, corn northern leaf blight, corn common rust, Corn Cerco sporal leaf spot, gray leaf spot, and.	1829
Grape	4	Grape leaf blight, Blackout, grape esca, and grape healthy.	1805
Peach	2	Peach healthy and peach bacterial spot.	891
Pepper	2	Bacterial spots and healthy leaves.	975
Potato	3	Early blight, late blight, and healthy leaves.	1426
Strawberry	2	Healthy and leaf scorch.	900
Tomato	10	Tomato Early blight, tomato mosaic virus, Tomato late blight, tomato spider mites two-spotted spider mite, tomato target spot, Bacterial spot, Tomato healthy, Tomato septoria leaf spot, Tomato leaf Mold, and tomato yellow leaf curl virus.	4585
Dataset 2
Citrus	5	citrus canker, citrus melanosis, citrus greening, Citrus black spot, and citrus healthy	609

3.3. Meta-heuristic FSJ-FOA

The classification performance of the multi-disease plant leaves detection model is enhanced by developing the hybrid algorithm namely FSJ-FOA, which is a combination of FOA and JA optimization algorithms. The FOA algorithm is generally used to resolve non-linear optimization issues and the JA algorithm solves constrained and unconstrained problems. Therefore, both algorithms are combined and used in the proposed model. Since the FOA algorithm works on the tree-based mechanism, it takes more computational time for network training which degrades the classification efficiency of the offered method. Therefore, the JA algorithm is merged with the FOA algorithm to resolve the existing issues of the FOA algorithm. Hence, the proposed FSJ-FOA algorithm enhances the performance of the offered scheme.

The developed FSJ-FOA algorithm finds the best solution for improving the development of the recommended scheme. This algorithm works according to the fitness values of old and newly updated solutions. Then, the mean value is calculated for the fitness function of the previous solution and the recently updated solution. Consider, $e$ is the average of the fitness function of the prior solution (old fitness function) and $g$ the average of the fitness function of the updated solution (new fitness function). It $g<e$ is attained, then the JA algorithm is considered, or else the FOA algorithm is taken into consideration for upgrading the solutions. Thus, the fitness function of both algorithms is sorted to determine the best solution.

JA algorithm (Ramesh and Vydeki 2020): The main objective of this algorithm is finding the best solution and avoiding the worst solution by moving the obtained solutions for the given problem towards the best solution. This process is defined by the following equation.

The fitness function to be decreased or increased is considered as $f(a).$ Consider for any iteration $t,$ there are $v$ several design variables, were $i=1,2,\dots ,v$ and $c$ several candidate solutions that are the population size, where $j=1,2,\dots ,c.$ In the whole solution, the best search agent $bs$ achieves the accurate value of $f(a)$ that is $f{(a)}_{bs}$ and the worst solution $ws$ achieves the worst value of $f(a)$ that is $f{(a)}_{ws}.$ It $A_{k,i,t}$ is the value of the term $k,$ for the $j{}^{th}$ solution during the iteration $t,$ then the equation is customized as Equation (1)(1) $$\begin{array}{c}{A^{\prime }}_{k,i,t}=A_{k,i,t}+rd{1}_{k,t}\left(A_{k,bs,t}-\left|A_{k,i,t}\right|\right)\\ -rd{2}_{k,t}\left(A_{k,ws,t}-\left|A_{k,i,t}\right|\right)\end{array}$$(1) . (1) $$\begin{array}{c}{A^{\prime }}_{k,i,t}=A_{k,i,t}+rd{1}_{k,t}\left(A_{k,bs,t}-\left|A_{k,i,t}\right|\right)\\ -rd{2}_{k,t}\left(A_{k,ws,t}-\left|A_{k,i,t}\right|\right)\end{array}$$(1)

The term $k$ for the best solution is denoted by $A_{k,bs,t},$ the ‘two arbitrary numbers for the $k^{th}$ constraint during the $t^{th}$ iteration are indicated by $rd{1}_{k,t}$ and $rd{2}_{k,t}$’, and the value of the term $k$ for the worst solution is denoted by $A_{k,ws,t}.$ The value $A_{k,i,t}$ is updated as ${A^{\prime }}_{k,i,t}$ and which lies in the limit [0,1]. The objective of the solution to move closer to the accurate solution is represented by $rd{1}_{k,t}(A_{k,bs,t}-\left|A_{k,i,t}\right|)$ and the worst solution to be avoided is indicated by $-rd{2}_{k,t}(A_{k,ws,t}-\left|A_{k,i,t}\right|).$ If it provides a better function value, then ${A^{\prime }}_{k,i,t}$ is accepted. The accepted function values of the previous iterations are given as input to the next iteration.

FOA algorithm (Naz et al. 2019): The FOA algorithm works mainly based on the concept of the seeding procedure of trees. The population of trees is initialized and the values of the variable are denoted in addition to the age of the tree. At first, the age of a tree is considered as 0. Consider a $D$ dimensional problem in which each tree is assumed as a $D+1$ dimensional vector $dv,$ where $dv=(\mathit{age},d{v}_1,d{v}_2,\dots ,d{v}_D).$ A parameter value is assigned for choosing the candidate population trees. The trees which are reached the parameters are added to the candidate population for a better process of local search. Once, the initialization process is completed, the new young trees or seeds are generated by the local seeding operator from the trees, which have an age of 0. When new trees are added to the forest and all the ages of trees are raised by ‘1 except the new ones. In the local seeding step, a parameter called Local Seeding Changes (LSC) refers to the new ones, which become the neighbor trees. The local seeding operator value is decreased at the next iterations since the age of the trees is increased by 1 and the non-zero-aged trees are not considered in the local seeding stage. The operation of local seeding is performed by selecting a random variable. A small random value $vr\in \left[-\Delta b,\Delta b\right]$ and this step are repeated for each zero-aged tree for LSC times. The global seeding operation is performed based on the randomly selected variables of each tree. Then, the value of a chosen variable is interchanged with another value of an arbitrarily created variable. Therefore, a global search is performed by this operator in the whole search space. Next, the process of updating the best tree is performed by categorizing the trees corresponding to their fitness value. The best tree is selected based on the better fitness value of the related tree. The age of the better tree age updated to 0 to ignore the aging process in the local seeding stage. Finally, the stop conditions of the algorithm are checked and the best tree is returned.

The pseudo-code of the proposed FSJ-FOA is shown in Algorithm 1.

Algorithm 1:

Designed FSJ-FOA

Population Initialization

Calculate the fitness function for every iteration

Determine $e$ and $g$

if $g<e$

 Solution update with JA algorithm using Equation (1)(1) $$\begin{array}{c}{A^{\prime }}_{k,i,t}=A_{k,i,t}+rd{1}_{k,t}\left(A_{k,bs,t}-\left|A_{k,i,t}\right|\right)\\ -rd{2}_{k,t}\left(A_{k,ws,t}-\left|A_{k,i,t}\right|\right)\end{array}$$(1)

else

 Solution update with FOA algorithm

 Determine Local seeding

 Determine Global seeding

end

Return the best solution

End

Therefore, the developed FSJ-FOA optimization algorithm enhances the development of the offered method by updating the best solutions according to the sorted value of old and new fitness functions. The flow diagram of the offered FSJ-FOA algorithm is given in Figure 4.

Figure 4. Flow chart of the proposed FSJ-FOA algorithm.

4. Processing steps for developed multi-disease classification of plant leaves

4.1. Pre-processing of leaf images

The process of automated detection of plant leaves diseases with the help of a hybridized classifier is initially started with the pre-processing stage. This technique is employed to enlarge the quantity of the image by eliminating the distortions present in the images. The resolution-enhanced images are used further in the efficient process of segmentation and classification. In this paper, image scaling and contrast enhancement approaches are used to perform the pre-processing step.

Image scaling (Roomi and Maragatham 2015): This process is also known as image resizing. It is mainly used for adjusting the pixels. The size of an image increases or decreases based on pixel adjustments. In image processing, the modification of digital objects is called scaling. This scaling approach is utilized for making the images look better using geometric variations, without losing the rate of the image. The input image after the scaling process is referred to as $P{L}_{in}^{sc}.$

Contrast Enhancement (Roomi and Maragatham 2015): This technique is applied for improving the fidelity and contrast of input images by reducing the distortions in them. The two steps included in the contrast enhancement technique are histogram equalization and image intensity adjustment. These phases also offer detailed content regarding the image. That is, the contrasts between the intensity values are increased to make the considered content of the leaves visible. (2) $$H_{eq}\left(P{L}_{in}^{sc}{}_{\left(x,y\right)}\right)=\mathit{round}\left(\frac{I{n}_{px}^{cr}\left(P{L}_{in}^{sc}{}_{\left(x,y\right)}\right)-I{n}_{px}^{cr}{}_{\min }}{\left(PR\times PC\right)-I{n}_{px}^{cr}}\times NI-1\right)$$(2)

The variable $I{n}_{px}^{cr}$ denotes the cumulative frequency of the gray level, the intensity of the present pixel is denoted as $I{n}_{px}^{cr}(P{L}_{in}^{sc}{}_{(x,y)})$ the term $NI$ refers to the number of intensities, the minimum value of CDF is represented by $I{n}_{px}^{cr}{}_{\min }$ and the product of several pixels in rows and columns is correspondingly indicated by $PR$ & $PC.$ The contrast-enhanced images using the histogram equalization are represented as $P{L}_{in}^{HE}.$

4.2. Binary thresholding-based leaf segmentation

After the pre-processing, the pre-processed images are fed to the segmentation stage. Image segmentation is a procedure of segmenting images into several image objects or segments. In this paper, the binary threshold method is applied for segmenting the leaf regions. This threshold method converts the grayscale images into binary images to separate the objects. This method is also used to split the image into foreground and background images. The pixel intensity of the original image has a threshold value, based on which a bitonal image is generated. This process is termed thresholding and it is equated in Equation (3)(3) $$P{L}_{in}^{ls}\left(x,y\right)=\{\begin{array}{cc}1& if{I}_{P{L}_{in}^{HE}}\left(x,y\right)>TR\\ 0& if{I}_{P{L}_{in}^{HE}}\left(x,y\right)\le TR\end{array}$$(3) . (3) $$P{L}_{in}^{ls}\left(x,y\right)=\{\begin{array}{cc}1& if{I}_{P{L}_{in}^{HE}}\left(x,y\right)>TR\\ 0& if{I}_{P{L}_{in}^{HE}}\left(x,y\right)\le TR\end{array}$$(3)

Here, the term $I_{P{L}_{in}^{HE}}(x,y)$ indicates the image intensity of the divided image and the term $TR$ denotes the threshold value. Therefore the segmented images using the binary thresholding approach are called as $P{L}_{pr}^{ls}.$

4.3. Proposed AFKMRG-based abnormality segmentation

In the abnormality segmentation phase, an adaptive fusion technique is introduced to segment the defects or abnormalities present in the plant leaves. In this paper, the K-means clustering and Region Growing (RG) algorithm is used for abnormality segmentation. Clustering is based on the concept of partitioning an image or data into a set of objects. The integration of K-means clustering and RG algorithms is utilized for segmenting the abnormalities of the plant leaves. The segmented images $P{L}_{pr}^{ls}$ are given as input to the abnormality segmentation phase. The outcomes from both K-means clustering and RG models are multiplied by optimal constants produced using the FSJ-FOA algorithm. After optimizing the parameters, the resultant images obtained from both models are fused to attain the abnormality segmented images.

K-means clustering (Isa et al. 2009): K-means clustering separates the input image into the k-number of objects. The Euclidean method is utilized to describe the nearest centroid distance. A new centroid is recalculated from each cluster depending on the related centroid after the grouping process. As a result, Euclidean distance is measured in every data center. Additionally, the cluster has assigned some points depending on the lowest Euclidean distance. Assume an input image $P{L}_{in}^{ls}$ with the decision of $r\times s$ and then $P{L}_{in}^{ls}(r,s)$ is the input pixels to the cluster and $C{L}_{ww}$ denotes the cluster centers, in which the images are partitioned into $ww$ several clusters. The K-means algorithm works based on the following steps:

• Initialize the count of clusters and centers.

• The Euclidean distance $\mathit{EUD}$ calculation is given in Equation (4)(4) $$\mathit{EUD}=\Vert P{L}_{in}^{ls}\left(r,s\right)-C{L}_{ww}\Vert$$(4)

(4) $$\mathit{EUD}=\Vert P{L}_{in}^{ls}\left(r,s\right)-C{L}_{ww}\Vert$$(4)

• The Euclidean distance has assigned the pixels to the nearest center.

• The new position of the center is calculated in Equation (5)(5) $$C{L}_{ww}=\frac{1}{ww}\sum _{s\in C{L}_{ww}}\sum _{r\in C_{ww}}P{L}_{in}^{ls}\left(r,s\right)$$(5)

(5) $$C{L}_{ww}=\frac{1}{ww}\sum _{s\in C{L}_{ww}}\sum _{r\in C_{ww}}P{L}_{in}^{ls}\left(r,s\right)$$(5)

• This process is replicated until the error or tolerance value is attained.

• The pixel of a cluster is then reshaped into an image.

This simpler implementation of K-means clustering makes the process an effective one. However, the more count of iterative and clusters of the K-means algorithm makes some difficulties and may attain less segmentation performance. The output produced using the K-means clustering algorithm is termed as $L{S}_{km}.$

RG (Borges et al. 2018): This algorithm uses seed pixels for segmenting the image into salient image regions. The seed pixels are added with new homogeneous pixels in a repeated manner until the segmentation criterion is satisfied. The segmentation process of an entire image area into sub-regions requires the following conditions:

1. ${\cup }_{p=1}^{nx}E{I}_p=EI;$

2. For $E{I}_1,E{I}_2,\dots ,E{I}_n,$ $EI$ is a connected region

3. For any $p,q,p\ne q,$ $E{I}_p\cap E{I}_q=\phi$

4. $Z(E{I}_p)=\mathit{true};$

5. $EI(E{I}_p\cup E{I}_q)=\mathit{false}.$

Here, the entire image area is denoted by $EI$ and it is split into $n$ sub-regions. The gray value of the set is denoted by $E{I}_i$ and $nx$ denotes the average of the pixels of an image. When the pixel value contains the following requirements, the seed point is selected initially and a pixel is added in each step of the algorithm. This process is denoted in Equation (6)(6) $$\mu =\frac{1}{n}\sum _{\left(u,w\right)\in EI}f\left(u,w\right)$$(6) (6) $$\mu =\frac{1}{n}\sum _{\left(u,w\right)\in EI}f\left(u,w\right)$$(6) (7) $${\left|f\left(u,w\right)-\mu \right|}_{\left(u,w\right)\in EI}<\Psi$$(7)

Here, the mean of the image is denoted by $\mu ,$ the value at the correlate axis $(u,w)$ is represented by $f(u,w)$ and the threshold value is indicated by $\Psi .$ The threshold value is close to the mean of the image. Therefore, the output produced from the RG algorithm is termed as $L{S}_{rg}.$

The AFKMRG fusion model is adopted for segmenting the abnormalities of the image. This method is an integration of K-means and RG algorithms. The segmented images $P{L}_{pr}^{ls}$ are further processed through the abnormality segmentation step and the output produced by K-means and RG is termed as $L{S}_{km}$ and $L{S}_{rg}.$ An optimal constant is multiplied with each output by the FSJ-FOA algorithm. The resultant image obtained by the AFKMRG fusion model produces $P{L}_{ls}^{\mathit{abs}}$ segmented regions. This abnormality segmentation process is formulated in Equation (8)(8) $$P{L}_{ls}^{\mathit{abs}}=\alpha \cdot L{S}_{km}+\beta \cdot L{S}_{rg}$$(8) . (8) $$P{L}_{ls}^{\mathit{abs}}=\alpha \cdot L{S}_{km}+\beta \cdot L{S}_{rg}$$(8)

Here, the term $\alpha$ $\beta$ denotes the constants selected optimally by the FSJ-FOA algorithm. After the abnormality segmentation process, the results obtained are represented as $P{L}_{ls}^{\mathit{abs}}$ where $ls=\left\{l{s}_1,l{s}_2,\dots l{s}_{ns}\right\}$ and $ns$ denoted by the number of segmented images. The abnormality segmentation process by AFKMRG fused model is depicted in Figure 5.

Figure 5. Abnormality segmentation using the AFKMRG fusion model.

The accurate solution is attained by computing the fitness value for the individual solutions. The best solution is termed according to the solution that has a minimum fitness value. The fitness function of the FSJ-FOA for deriving the abnormality segmentation constants is represented as shown in Equation (9)(9) $$\mathit{ftn}=\underset{\alpha ,\beta }{\arg }\min \left(\sum _{dk=1}^{is}\sum _{dc=1}^{sp}\left(i{m}^{dk}-in{p}_{dc}\right)\right)$$(9) . (9) $$\mathit{ftn}=\underset{\alpha ,\beta }{\arg }\min \left(\sum _{dk=1}^{is}\sum _{dc=1}^{sp}\left(i{m}^{dk}-in{p}_{dc}\right)\right)$$(9)

Here, the size of the input image is indicated by $sp,$ and the center of the image segment $C{R}_{dk}$ is indicated by $i{m}^{dk}$ and $in{p}_{dc}$ denotes the $d{c}^{th}$ pixel that belongs to the $d{k}^{th}$ segment.

Assume the image with $TP$ the number of pixels from the $d{k}^{th}$ segment and each pixel is represented as $i{n}_{dc}.$ The center for the segmented image is computed as given in Equation (10)(10) $$i{m}^{dk}=\frac{1}{TP}\sum _{dc=1}^{TP}in{p}_{dc}\in C{R}_{dk}$$(10) . (10) $$i{m}^{dk}=\frac{1}{TP}\sum _{dc=1}^{TP}in{p}_{dc}\in C{R}_{dk}$$(10)

The use of optical constants by the FSJ-FOA algorithm enhances the process of abnormality segmentation. The enhanced segmented images are used to improve the further classification performance. The solution encoding diagram for optimal constants is depicted in Figure 6.

Figure 6. Solution encoding for optimal constants in segmentation.

5. Multi-disease classification of plant leaves using optimal deep features with LSTM

5.1. Optimized CNN-aided deep features extraction

In the offered method, the classification of multi-diseases of plant leaves is done by two methods as CNN and Enhanced LSTM method. The categorization enlargement of the recommended scheme is enhanced by selecting the optimal features from the segmented images using the CNN technique. Here, the segmented images $P{L}_{ls}^{\mathit{abs}}$ from the abnormality segmentation phase are forwarded to the CNN in the classification stage to attain the optimal features. The number of suitably hidden neurons in CNN is developed by the FSJ-FOA algorithm to enhance the efficiency of the feature extraction process.

CNN (Rawat and Wang 2017): Generally, the CNN architecture consists of several layers. In the proposed scheme, the deep features are acquired from convolutional and pooling layers. The description of each layer is represented as follows:

5.1.1. Convolution layer

The input segmented images $P{L}_{ls}^{\mathit{abs}}$ are subjected to the convolutional layer to acquire the deep features. Each field is associated with the neuron of the previous layer. The extraction process of the feature map is formulated in Equation (11)(11) $$E{F}_{cl}=gy\left(F{M}_{cl}\ast P{L}_{ls}^{\mathit{abs}}\right)$$(11) . (11) $$E{F}_{cl}=gy\left(F{M}_{cl}\ast P{L}_{ls}^{\mathit{abs}}\right)$$(11)

The variable $P{L}_{ls}^{\mathit{abs}}$ denotes the output abnormality segmented image, the term $E{F}_{cl}$ is the $c{l}^{th}$ resultant feature map in the convolution layer. The interior attributes of the input image at every position are calculated using the two-dimensional operator and it is denoted by the symbol $\ast$ and the non-linear activation function is called as $gy(\cdot )$. This function is applied for extracting the non-linear features.

5.1.2. Pooling layer

The higher rate value is transmitted by most pooling layers. The respective value is propagated to the next later inside a receptive field, in which this layer selects the higher element. This process is formulated in Equation (12)(12) $$E{F}_{\mathit{clpg}}=\underset{(su,sv)\in L_{pg}}{\max }gyP{L}_{\mathit{clsusv}}$$(12) . (12) $$E{F}_{\mathit{clpg}}=\underset{(su,sv)\in L_{pg}}{\max }gyP{L}_{\mathit{clsusv}}$$(12)

In the aforementioned equation, the variable $E{F}_{\mathit{clpg}}$ denotes the outcome of the pooling process with the $c{l}^{th}$ feature map. In the element $P{L}_{\mathit{clsusv}},$ the location is denoted by $(su,sv),$ which is enclosed by the polling area $L_{pg}.$

Thus, the deep features acquired using the pooling layer of the CNN technique are indicated as $E{F}_{cl},$ where $cl=1,2,\dots ,CL,$ and $CL$ indicates the average of extracted deep features. The feature extraction performance of CNN is then improved by tuning the hidden neurons using FSJ-FOA, which lies in the range of 5 to 255. The extracted features are further processed through the classification phase, which is done by enhanced LSTM.

5.2. Enhanced LSTM-based classification

The final phase of the proposed multi-disease classification of plant leaves model is the classification phase (Manne et al. 2020). This phase is used to sort out the different diseases of plant leaves. The deep features acquired by the CNN technique are given as input for the LSTM classification. LSTM network is a specific form of RNN network and the past data can be stored in the memory unit using LSTM. LSTM is also used to reduce the exploding and gradient vanishing problems of RNN. In many recent prediction models, LSTM has been showing effective performance in time series data prediction.

LSTM (Akilan et al. 2020): A LSTM network generally contains four components cell, an input gate, forget gate, and an output gate. The cell in the LSTM transfers the information at randomly chosen time intervals. The input and output data flow is traced by the gates. The computation of nodal outputs of an LSTM network is shown in the following equations: (13) $$in{g}_{tm}=\sigma \left(w{m}_{\mathit{ing}}\cdot \left[h{s}_{tm-1},E{F}_{cl}\right]+b{v}_{\mathit{ing}}\right)$$(13) (14) $$h{s}_{tm}=o{g}_{tm}\ast \tanh \left(c{e}_{tm}\right)$$(14)

Here, the term $E{F}_{cl}$ denotes the input variable at a time $tm.$ The weight matrices are indicated by $w{m}_{\mathit{ing}}.$ The input gates are represented by $in{g}_{tm}.$ The symbol $\sigma$ represents the sigmoid activation function and the hidden calculation for state outputs at a time $tm$ is denoted by $h{s}_{tm}.$ The biased values of different gates are represented by $b{v}_{\mathit{ing}}.$

5.3. Objective function

The proposed multi-disease plant leaves detection model is performed using optimized CNN and enhanced LSTM by adopting FSJ-FOA. The fitness function is calculated for the improved classification model as shown in Equation (19)(19) $$\mathit{ftn}=\underset{\left\{\mathit{chn},\mathit{lhn}\right\}}{\arg \min }\left(\frac{1}{\mathit{acy}}\right)$$(19) . (19) $$\mathit{ftn}=\underset{\left\{\mathit{chn},\mathit{lhn}\right\}}{\arg \min }\left(\frac{1}{\mathit{acy}}\right)$$(19)

The CNN feature extraction and LSTM classification model are outlined to deliver high-precision forecasting. The classification efficiency of both models is improved by resolving the fitness function with accuracy. The performance of the CNN-based optimal features with LSTM classification is further improved by tuning the suitably hidden neurons of both CNN $\mathit{chn}$ and LSTM $\mathit{lhn}$ networks using the FSJ-FOA algorithm, which is in the range of 5 to 255. This optimization enhances the classification effectiveness of the designed model concerning with maximization of accuracy. The solution encoding for the offered classification method is shown in Figure 7.

Figure 7. Optimization of constraints in the proposed classification model.

Accuracy: The proportion of the study exactly measured to the entire observations. This is formulated in Equation (20)(20) $$\mathit{acy}=\frac{\left(PT+NT\right)}{\left(PT+NT+PF+NF\right)}$$(20) . (20) $$\mathit{acy}=\frac{\left(PT+NT\right)}{\left(PT+NT+PF+NF\right)}$$(20)

The true positives, false negatives, false positives, true negatives and are called as $PT,$ $NF,$ $PF,$ $NT$ and. Thus, the CNN-based optimal features with enhanced LSTM classification provide multi-disease classified plant leaves with high accuracy. The classification method is expressed in Figure 8.

Figure 8. Proposed CNN-based feature extraction and enhanced LSTM-aided classification method.

6. Calculation of results

6.1. Experimental Evaluation

The offered FSJ-FOA-CNN-LSTM-based multi-disease classification of plant leaves was implemented in Python. Evaluation of the designed system was executed by comparing it to the existing systems with performance metrics. The performance analysis was executed for the proposed FSJ-FOA algorithm by comparing various optimization algorithms such as Grey Wolf Optimization GWO (Sreedharan et al. 2018), Deer Hunting Optimization Algorithm (DHOA) (Brammya et al. 2019), FOA (Naz et al. 2019) and JA (Ramesh and Vydeki 2020), and CNN based enhanced LSTM classifier by various machine learning algorithms like CNN (Rawat and Wang 2017), Support Vector Machine (SVM) (Martens et al. 2009), Neural Networks (NN) (Wu et al. 2016), LSTM (Akilan et al. 2020) and CNN-LSTM (Nagda and Poovammal 2018).

6.2. Efficiency metrics

The performance metrics used for the designed method are elaborated here.

1. F1 score:

(21) $$F1-\mathit{score}=\frac{\mathit{Sen}•\mathit{pre}}{\mathit{pre}+\mathit{Sen}}$$(21)

• MCC:

(22) $$\mathit{MCC}=\frac{PT\times NT-PF\times NF}{\sqrt{\left(PT+PF\right)\left(PT+NF\right)\left(NT+PF\right)\left(NT+NF\right)}}$$(22)

• FPR:

(23) $$\mathit{FPR}=\frac{PF}{PF+NT}$$(23)

• NPV:

(24) $$\mathit{NPV}=\frac{NT}{NF+NT}$$(24)

• FNR:

(25) $$\mathit{FNR}=\frac{NF}{NT+PT}$$(25)

• FDR:

(26) $$\mathit{FDR}=\frac{PF}{PF+PT}$$(26)

• Precision:

(27) $$\mathit{pre}=\frac{PT}{PT+PF}$$(27)

• Specificity:

(28) $$\mathit{Spe}=\frac{NT}{NT+PF}$$(28)

• Sensitivity:

(29) $$\mathit{Sen}=\frac{PT}{PT+NF}$$(29)

6.3. Time complexity

The big oh notation of the existing and proposed algorithms is given here. For GWO $O\left[t_{\max }\ast \left[c+(c\ast \lg )\right]\right],$ DHOA $O\left[t_{\max }\ast \left[c+(c\ast \lg )\right]\right],$ FOA $O\left[t_{\max }\ast \left[c\ast 2+1\left[\left(c\ast \frac{tr}{100}\right)\ast \frac{\lg }{10}\right]+c\right]\right],$ and JA $O\left[c+\left[t_{\max }\ast (c\ast \lg )+(c+c+c)\right]\right]$F the proposed model is FSJ-JOA: $O\left[c+\left[t_{\max }\ast (c\ast \lg )+(c+c+c)\right]\right].$

Here, iteration $t,$ $c$ several candidate solutions that are the population size, $\lg$ channel length, and the transfer rate is termed as $tr$= 15.

6.4. Experimental results

Some of the original, pre-processed, segmented, and abnormality-segmented images are shown in Figure 9.

Figure 9. Experimental results of proposed FSJ-FOA-CNN-LSTM-based multi-disease classification of plant leave with original, pre-processed, segmented, and abnormality segmented images for diverse types of plant.

6.5. Analysis of accuracy over meta-heuristic algorithms

The FSJ-FOA algorithm of the offered multi-leaves disease classification model is compared by varying the learning percentages regarding accuracy as shown in Figure 10. While considering the accuracy of apple plant leaves for the learning percentage of 55, the FSJ-FOA-CNN-LSTM performed 2.88% better than GWO-CNN-LSTM, 5.01% better than DHOA-CNN-LSTM, 2.12% better than FOA-CNN-LSTM, and 1.58% better than JA-CNN-LSTM. Similarly, for all kinds of plant leaves the proposed FSJ-FOA algorithm provides better accuracy than the other existing optimization algorithms. In certain cases, JA and FOA algorithms are also trying to attain the same performance as the proposed algorithm. For example, during the analysis of peach leaves, the FOA algorithm reaches the same level as the proposed model for the learning percentage 55. Likewise, while analyzing the citrus leaves, the JA reaches an equal level to the proposed model corresponding with a learning percentage of 55. Therefore, the offered model has used the combination of FOA and JOA algorithm namely FSJ-FOA for achieving the superior performance of classification.

Figure 10. Validation of the proposed FSJ-FOA-CNN-LSTM-based multi-disease classification of plants model on ‘metaheuristic algorithms in terms of accuracy’ for diverse types of leaves like ‘(a) apple, (b) cherry, (c) corn, (d) grape, (e) peach, (f)pepper, (g) potato, (h) strawberry, (i) tomato and (j) citrus’.

6.6. Accuracy analysis of the proposed classifiers

The enhanced CNN-LSTM classification model using the FSJ-FOA algorithm for multi-leaves disease classification is compared with other existing classifiers regarding accuracy and learning percentages as given in Figure 11. While considering the F1-score of corn plant leaves for the learning percentage of 65, the FSJ-FOA-CNN-LSTM performed 11.76%, 10.46%, 13.09%, 15.85%, and 5.55% more advanced performance than NN, SVM, CNN, LSTM, CNN-LSTM. Likewise, for all kinds of plant leaves the FSJ-FOA-CNN-LSTM classification method gives maximum accuracy. In an overview, the proposed FSJ-FOA-CNN-LSTM attains maximum performance in evaluating all the performance measures for all types of plant leaves with various learning percentages. Therefore, the enhanced CNN-based LSTM classification using FSJ-FOA provides a better classification of multi-diseased plant leaves.

Figure 11. Evaluation of the proposed FSJ-FOA-CNN-LSTM-based multi-disease classification of plant leaves model on existing deep structured architectures regarding accuracy for different types of leaves like ‘(a) apple, (b) cherry, (C) corn, (d) grape, (e) peach, (f)pepper, (g) potato, (h) strawberry, (i) tomato and (j) citrus’.

6.7. F1-score evaluation of the offered scheme on optimization algorithms

Validation of the F1-score of the offered FSJ-FOA-CNN-LSTM-based multi-disease classification of plant leaves model using the FSJ-FOA algorithm is ‘compared with other meta-heuristic algorithms in terms of F1-score and learning percentages’ as shown in Figure 12. The F1-score of the offered FSJ-FOA algorithm for cherry plant leaves is 1.02%, 0.71%, 0.50%, and 0.71% enriched performance than GWO-CNN-LSTM, DHOA-CNN-LSTM, FOA-CNN-LSTM, JA-CNN-LSTM while considering 65 as a learning percentage. Similarly, for the plant leaves, the proposed algorithm achieves enhanced performance in F1-score by varying the learning percentages. All the other algorithms are also competing to reach the high value of the F1 score in certain cases. For example, in the analysis of peach leaves for the learning percentage of 75, all the other meta-heuristic algorithms are tried to reach the value of 98. But the proposed FSJ-FOA model reaches the maximum value of the F1-score as 99. Therefore, the proposed classification using the FSJ-FOA algorithm provides superior performance to all the existing meta-heuristic algorithms.

Figure 12. Calculation of the proposed FSJ-FOA-CNN-LSTM-based multi-disease classification of plant leaves on ‘metaheuristic algorithms in terms of’ F1-score for diverse types of leaves like ‘(a) apple, (b) cherry, (C) corn, (d) grape, (e) peach, (f) pepper, (g) potato, (h) strawberry, (i) tomato and (j) citrus’.

6.8. Estimation of F1-score with classifiers

The suggested FSJ-FOA-CNN-LSTM classification method is validated regarding F1-score by testing with other classifiers as shown in Figure 13. While taking the analysis of pepper leaves, the F1-score of the offered FSJ-FOA-CNN-LSTM-based classification model is 3.15%, 5.37%, 2.08%, 3.15%, 1.03% more advanced performance than the NN, SVM, CNN, LSTM, CNN-LSTM. Likewise, the proposed FSJ-FOA-CNN-LSTM classification model attains better performance.

Figure 13. Evaluation of the proposed FSJ-FOA-CNN-LSTM-based multi-disease classification of plant leaves models on existing classifiers regarding F1-score for different types of leaves like ‘(a)apple, (b)cherry, (C) corn, (d) grape, (e) peach, (f)pepper, (g) potato, (h) strawberry, (i) tomato and (j)citrus’.

6.9. Parameters of sensitive analysis

The sensitive analysis of parameters considered as population size and area limit for the proposed FSJ-FOA-CNN-LSTM-based multi-disease classification model of plant leaves is shown in Figure 14. The cost function analysis is performed by differing the number of iterations to reveal the lower cost function attained from the FSJ-FOA-CNN-LSTM model. From Figure 14(a), while varying the iterations, when the population size is 100, the cost function of the value starts from 0.92 and gradually decreased to 0.65. Accordingly, From Figure 14(b), when the area limit is 25, the cost function of the value starts from 0.88 and is gradually reduced to 0.70.

Figure 14. Sensitive analysis on the parameters considered as (a) Population size and (b) Area-limit of the proposed model.

6.10. Empirical analysis of offered method

The empirical analysis of the proposed FSJ-FOA-CNN-LSTM-based multi-disease classification of plant leaves model on optimization algorithms and classifiers for different types of plant leaves is shown in Tables 3 and 4. From Table 3, the average of the FSJ-FOA-CNN-LSTM model is secured at 4.11%, 4.08%, 4.64%, and 1.35% progressed than the GWO-CNN-LSTM, DHOA-CNN-LSTM, FOA-CNN-LSTM, and JA-CNN-LSTM respectively for apple leaves. Similarly, the standard deviation of the FSJ-FOA-CNN-LSTM model is attained at 85.20%, 87.59%, 58.75%, 98.38%, and 59.75% enhanced than the GWO-CNN-LSTM, DHOA-CNN-LSTM, FOA-CNN-LSTM, and JA-CNN-LSTM respectively for apple leaves. Hence, it is proved that the suggested FSJ-FOA-CNN-LSTM method achieved better results.

Table 3. Calculation of the offered scheme on optimization algorithms for different types of plant leaves.

Algorithms	GWO-CNN-LSTM (Sreedharan et al. 2018)	DHOA-CNN-LSTM (Brammya et al. 2019)	FOA-CNN-LSTM (Naz et al. 2019)	JA-CNN-LSTM (Ramesh and Vydeki 2020)	FSJ-FOA-CNN-LSTM
Apple
Average	0.9332	0.9334	0.928	0.96	0.9732
Standard deviation	0.007386	0.014773	0.008602	0.006481	0.0036
Cherry
Average	0.944	0.9424	0.9388	0.9454	0.9788
Standard deviation	0.015925	0.013245	0.01087	0.018906	0.0024
Corn
Average	0.9424	0.9502	0.9432	0.938	0.9754
Standard deviation	0.013306	0.01574	0.013526	0.018974	0.003323
Grape
Average	0.9326	0.9396	0.9628	0.941	0.976
Standard deviation	0.010151	0.016181	0.008495	0.016346	0.003847
Peach
Average	0.9306	0.9618	0.948	0.9432	0.9752
Standard deviation	0.012468	0.006554	0.01858	0.019385	0.004915
Pepper
Average	0.9488	0.9422	0.9522	0.944	0.9786
Standard deviation	0.019208	0.011873	0.010515	0.009633	0.003826
Potato
Average	0.9398	0.9422	0.949	0.9426	0.977
Standard deviation	0.012205	0.009368	0.016912	0.021077	0.003033
Strawberry
Average	0.9378	0.961	0.9344	0.934	0.9742
Standard deviation	0.014538	0.009757	0.016305	0.007457	0.002227
Tomato
Average	0.9424	0.9378	0.9428	0.9394	0.9788
Standard deviation	0.012274	0.020576	0.016881	0.015615	0.002926
Citrus
Average	0.9336	0.9526	0.9424	0.9384	0.9714
Standard deviation	0.011943	0.01758	0.016181	0.017738	0.002728

Table 4. Examination of the proposed multi-disease classification of plant leaves model on classifiers for different types of plant leaves.

Classifiers	NN (Martens et al. 2009)	SVM (Wu et al. 2016)	CNN (Rawat and Wang 2017)	LSTM (Akilan et al. 2020)	CNN-LSTM (Nagda and Poovammal 2018)	FSJ-FOA-CNN-LSTM
Apple
Average	0.913	0.8948	0.9518	0.8998	0.9444	0.9732
Standard deviation	0.020219	0.01828	0.018777	0.019009	0.013351	0.0036
Cherry
Average	0.915	0.921	0.9372	0.9136	0.9424	0.9788
Standard deviation	0.024585	0.018547	0.014274	0.023252	0.021341	0.0024
Corn
Average	0.9052	0.9084	0.9304	0.8998	0.9558	0.9754
Standard deviation	0.022337	0.026635	0.008015	0.020498	0.008232	0.003323
Grape
9Average	0.9062	0.9272	0.9458	0.9162	0.9384	0.976
Standard deviation	0.022194	0.016412	0.013422	0.034184	0.018757	0.003847
Peach
Average	0.9044	0.9172	0.944	0.914	0.9486	0.9752
Standard deviation	0.027215	0.019783	0.014199	0.031235	0.011792	0.004915
Pepper
Average	0.9188	0.9144	0.9384	0.9152	0.9566	0.9786
Standard deviation	0.020827	0.018789	0.013937	0.028519	0.014108	0.003826
potato
Average	0.9002	0.9002	0.9366	0.901	0.9444	0.977
Standard deviation	0.023138	0.026977	0.012532	0.022865	0.014678	0.003033
Strawberry
Average	0.9042	0.9104	0.9518	0.9108	0.9602	0.9742
Standard deviation	0.02801	0.03025	0.013948	0.018659	0.00598	0.002227
Tomato
Average	0.8924	0.8988	0.9344	0.8914	0.9352	0.9788
Standard deviation	0.006184	0.024506	0.012241	0.021001	0.012921	0.002926
Citrus
Average	0.8988	0.9138	0.9312	0.9048	0.9422	0.9714
Standard deviation	0.032536	0.023836	0.014824	0.024498	0.019467	0.002728

6.11. Runtime analysis

The running time analysis of the offered method for various types of plant leaves is given in Table 5. From Table 5, when considering the peach leaves, holds minimum time when compared with the other different types of leaves. In the same manner, when considering the Tomato leaves it can obtain maximum time. Thus, this analysis shows the runtime taken by each plant.

Table 5. Calculation of running time for the offered scheme for different types of plant leaves.

Different types of plant leaves	Running time (Sec)
Apple	108.804
Cherry	43.735
Corn	100.493
Grape	100.591
Peach	36.948
Pepper	47.727
Potato	74.001
Strawberry	44.504
Tomato	527.465
Citrus	54.813

6.12. Total images of both datasets

The total image of both datasets for the offered scheme is given in Table 6.

Table 6. Total image images of both datasets for the designed model.

Different types of plant leaves	Number of images
Peach	891
Cherry	877
Strawberry	900
Corn	1829
Grape	1805
Potato	1426
Apple	1943
Pepper	975
Tomato	4585
Citrus	609

6.13. Examination of confusion matrix

Evaluation of the confusion matrix for the offered model with the baseline approaches is given in Table 7 and Table 8. The graph result for the confusion matrix on the designed method is shown in Figure 15.

Figure 15. Graph result for confusion matrix on the suggested plant leaves classification with multiple diseases model.

Table 7. Evaluation of confusion matrix with conventional optimization algorithms for the offered method.

Algorithms	GWO-CNN-LSTM (Sreedharan et al. 2018)	DHOA-CNN-LSTM (Brammya et al. 2019)	FOA-CNN-LSTM (Naz et al. 2019)	JA-CNN-LSTM (Ramesh and Vydeki 2020)	FSJ-FOA-CNN-LSTM
Apple
TP	908	940	905	936	957
TN	19	16	17	17	12
FP	15	16	15	16	14
FN	58	28	63	31	17
Cherry
TP	923	934	920	943	961
TN	16	17	18	16	14
FP	16	19	18	15	14
FN	45	30	44	26	11
Corn
TP	924	924	937	911	964
TN	17	18	18	17	12
FP	15	16	15	16	12
FN	44	42	30	56	12
Grape
TP	902	941	953	922	969
TN	19	15	16	18	12
FP	16	19	18	18	13
FN	63	25	13	42	6
Peach
TP	939	947	943	941	967
TN	16	17	19	19	12
FP	16	18	16	18	12
FN	29	18	22	22	9
Pepper
TP	953	923	937	940	967
TN	19	16	15	15	14
FP	18	19	18	18	14
FN	10	42	30	27	5
Potato
TP	936	910	941	953	968
TN	18	17	19	17	13
FP	19	18	19	17	12
FN	27	55	21	13	7
Strawberry
TP	939	952	903	921	964
TN	17	16	15	17	14
FP	19	17	17	15	13
FN	25	15	65	47	9
Tomato
TP	921	929	943	906	961
TN	19	17	18	18	13
FP	17	17	17	15	12
FN	43	37	22	61	14
Citrus
TP	913	947	907	901	957
TN	16	18	15	17	14
FP	15	16	15	16	13
FN	56	19	63	66	16

Table 8. Estimation of confusion matrix with conventional classifiers for the designed method.

Algorithms	NN (Martens et al. 2009)	SVM (Wu et al. 2016)	CNN (Rawat and Wang 2017)	LSTM (Akilan et al. 2020)	CNN-LSTM (Nagda and Poovammal 2018)	FSJ-FOA-CNN-LSTM
Apple
TP	932	910	905	855	919	957
TN	18	17	19	19	17	12
FP	18	16	15	18	19	14
FN	32	57	61	108	45	17
Cherry
TP	862	917	900	869	903	961
TN	17	18	15	16	19	14
FP	17	15	17	16	15	14
FN	104	50	68	99	63	11
Corn
TP	851	893	921	911	934	964
TN	17	19	17	18	15	12
FP	17	19	16	15	15	12
FN	115	69	46	56	36	12
Grape
TP	850	897	919	858	910	969
TN	17	18	17	15	15	12
FP	16	19	18	19	15	13
FN	117	66	46	108	60	6
Peach
TP	935	895	936	925	941	967
TN	18	16	19	17	18	12
FP	19	18	17	17	16	12
FN	28	71	28	41	25	9
Pepper
TP	890	897	907	936	954	967
TN	17	18	16	18	19	14
FP	17	15	16	16	17	14
FN	76	70	61	30	10	5
Potato
TP	871	918	905	888	913	968
TN	16	19	15	17	19	13
FP	19	16	17	18	16	12
FN	94	47	63	77	52	7
Strawberry
TP	871	876	917	905	946	964
TN	15	19	18	19	18	14
FP	18	17	18	17	16	13
FN	96	88	47	59	20	9
Tomato
TP	876	895	919	858	915	961
TN	15	15	17	15	16	13
FP	19	18	16	15	15	12
FN	90	72	48	112	54	14
Citrus
TP	862	871	908	900	910	957
TN	17	15	16	19	17	14
FP	15	16	17	17	18	13
FN	106	98	59	64	55	16

6.14. Estimation of the offered method

Examination of the offered FSJ-FOA-CNN-LSTM-based multi-disease classification model of plant leaves model given in Table 9 and Table 10. In Table 9, the specificity of the designed FSJ-FOA-CNN-LSTM for grape leaves is correspondingly secured at 5.87%, 2.33%, 3.22%, and 3.99% more advanced performance than GWO-CNN-LSTM, DHOA-CNN-LSTM, FOA-CNN-LSTM, JA-CNN-LSTM. Accordingly, from Table 10 the FPR of the developed FSJ-FOA algorithm is correspondingly secured at 45.34%, 23.56%, 31.77%, 38.55%, and 26.41% enriched performance than the NN, SVM, CNN, LSTM, and CNN-LSTM. Hence, the given validation reveals that the suggested FSJ-FOA-CNN-LSTM classification method offers a better classification of plant leaves affected by multiple diseases.

Table 9. Examination of the offered FSJ-FOA-CNN-LSTM-based multi-disease classification method of plant leaves model.

Metrics	GWO-CNN-LSTM (Sreedharan et al. 2018)	DHOA-CNN-LSTM (Brammya et al. 2019)	FOA-CNN-LSTM (Naz et al. 2019)	JA-CNN-LSTM (Ramesh and Vydeki 2020)	FSJ-FOA-CNN-LSTM
Accuracy	95.42	95.75	96.42	97.02	98.35
Sensitivity	95.40	95.77	96.34	97.04	98.36
Specificity	95.44	95.73	96.51	96.99	98.35
Precision	95.58	95.86	96.61	97.09	98.40
FPR	4.56	4.27	3.49	3.01	1.65
FNR	4.60	4.23	3.66	2.96	1.64
NPV	95.44	95.73	96.51	96.99	98.35
FDR	4.42	4.14	3.39	2.91	1.60
F1-Score	95.49	95.82	96.47	97.06	98.38
MCC	90.84	91.50	92.84	94.03	96.71

Table 10. Validation of the offered FSJ-FOA-CNN-LSTM-based multi-disease classification method of plant leaves model with classifiers.

Metrics	NN (Martens et al. 2009)	SVM (Wu et al. 2016)	CNN (Rawat and Wang 2017)	LSTM (Akilan et al. 2020)	CNN-LSTM (Nagda and Poovammal 2018)	FSJ-FOA-CNN-LSTM
Accuracy	95.49	96.18	96.71	97.35	97.71	98.35
Sensitivity	95.49	96.15	96.71	97.32	97.70	98.36
Specificity	95.49	96.22	96.70	97.38	97.72	98.35
Precision	95.63	96.33	96.80	97.46	97.79	98.40
FPR	4.51	3.78	3.30	2.62	2.28	1.65
FNR	4.51	3.85	3.29	2.68	2.30	1.64
NPV	95.49	96.22	96.70	97.38	97.72	98.35
FDR	4.37	3.67	3.20	2.54	2.21	1.60
F1-Score	95.56	96.24	96.76	97.39	97.74	98.38
MCC	90.98	92.36	93.41	94.70	95.42	96.71

6.15. Training accuracy graph of the offered scheme

The training accuracy graph of the suggested plant leaves classification with multiple disease representation is shown in Figure 16. In Figure 16(a), the accuracy of the offered method is differed according to the number of epochs. Similarly, in Figure 16(b), the failure of the offered approach is differing according to the suitable number of epochs. Finally, the given graph analysis shows that the suitable number of epochs improved then the model accuracy also increases. Accordingly, if the number of epochs decreases then the model loss also decreases.

Figure 16. Training accuracy graph of the suggested plant leaves classification with multiple diseases model (a) Model accuracy (b) Model loss.

6.16. Discussion

The existing approaches contain certain demerits like the DATFGAN (Dai et al. 2020) enhance the classification accuracy. However, it is harder to train for very larger image datasets. CNN (Sharma et al. 2020) attains high accuracy even for large data sets and more types of diseases and also provides remedies for the classified disease. But, it is hard to train the neural network since this model requires large datasets for training and slows down when processing max-pool-like operations. EfficientNet deep learning (Atila et al. 2021) gives high accuracy even for lower-resolution images. However, it is not suitable for rapid plant disease identification using smartphones. GoogLeNet and Cifar10 (Zhang et al. 2018) have improved recognition and training efficiency. Nevertheless, it has overfitting problems while using a large number of datasets. Leaf GAN (Liu et al. 2020) has effectively generated sufficient leaf images with prominent lesions. However, the accuracy rate is downing slightly when it deals with high-resolution images. GMDH-Logistic method (Chen et al. 2020) recognizes plant diseases well as well as monitors the growth of plants for future use. However, it causes deviations in the result in the case of slight and moderate disease due to a negligible gap. BRBFNN (Chohan et al. 2018) provides great computational efficiency for identification and classification. On the other hand, it is not improved to detect bacterial or viral diseases in plant leaves. ANN (Pham et al. 2020) (Giridhar et al. 2020) can be implemented in resource-constrained devices like smartphones. Yet it requires more trainable parameters for large-size images. GWO enlarges the fault rates. The proposed FSJ-FOA-CNN-LSTM model achieves a better convergence rate. Hence, it is proved that the offered FSJ-FOA-CNN-LSTM approach is better than the other conventional optimization algorithms and classifiers. The above outcomes suggested that the offered techniques can detect the multi-disease of plant leaves effectively with reasonable accuracy. In addition, improved deep learning techniques may be applied to the offered method with better accuracy in the future.

7. Conclusion

In this task, a novel approach was adopted for better plant leaf classification with multiple diseases. The original images of various kinds of plant leaves were given as input to the contrast enhancement approaches and scaling for pre-processing the images. Then, the binary thresholding technique was utilized for segmenting the leaf region from the pre-processed images. Next, the abnormality segmentation process was made by the developed AFKMRG approach through the FSJ-FOA algorithm. Finally, the classifications of plant leaves were performed by CNN-based optimal features and enhanced the LSTM classification technique, which was improved by the FSJ-FOA algorithm. This enhanced classification model has provided a better classification of plant leaves. The overall analysis of the offered FSJ-FOA-CNN-LSTM model has provided promising results over the conventional models. For example, the accuracy of the proposed FSJ-FOA-CNN-LSTM classification model has attained 10.46% superior performance to NN, 10% superior performance to SVM, 5.08% superior performance to CNN, 5.65% superior performance to LSTM and 4.74% superior performance to CNN-LSTM for citrus plant leaves. Hence, it was concluded that the designed FOA-CNN-LSTM method had achieved more advanced efficiency in the multi-disease classification of plant leaves. Future work will be focused on expanding the dataset of plant leaves by increasing the number of plants with diverse classes. This expansion leads the proposed model to make many accurate predictions even in complex environments. The applications of applied soft computing are given by, automotive systems and manufacturing, handwritten script recognition, image processing, data compression, decision support system, and soft computing-based architecture. The practical implications of plant leaf classification with multiple diseases are listed below, livestock health monitoring, insect and plant disease detection, crop and soil monitoring, and product grading and sorting. The limitations of the proposed work are it does not solve optimization problems and does not consider multiple leaves in a single image. In the future, the work will concentrate on implementing intelligent hybridized algorithms to enlarge the recognition rate of the final classification procedure by considering the above two problems. Moreover, future works can be focused on automatically evaluating the severity of the detected diseases.

Disclosure statement

No potential conflict of interest was reported by the authors.

Data availability statement

The analysis of multi-disease detection of plant leaves is performed by using different leaf images gathered from two diverse datasets such as Kaggle and Mendeley.