Transfer Learning-Based Framework for Classification of Pest in Tomato Plants

ABSTRACT

Pest in the plant is a major challenge in the agriculture sector. Hence, early and accurate detection and classification of pests could help in precautionary measures while substantially reducing economic losses. Recent developments in deep convolutional neural network (CNN) have drastically improved the accuracy of image recognition systems. In this paper, we have presented a transfer learning of pre-trained deep CNN-based framework for classification of pest in tomato plants. The dataset for this study has been collected from online sources that consist of 859 images categorized into 10 classes. This study is first of its kind where: (i) dataset with 10 classes of tomato pest are involved; (ii) an exhaustive comparison of the performance of 15 pre-trained deep CNN models has been presented on tomato pest classification. The experimental results show that the highest classification accuracy of 88.83% has been obtained using DenseNet169 model. Further, the encouraging results of transfer learning-based models demonstrate its effectiveness in pest detection and classification tasks.

Introduction

The economy of a country can be assessed by the role of agriculture. However, there are various challenges in agriculture such as huge requirements due to population growth, climate change, insufficient resource, and plant disease. One of the major hindrances in the cultivation of the crop is pest (Manoja and Rajalakshmi 2014). It is estimated that approximately 18% of the crop production lost every year due to animal pest (Oerke 2006). It causes substantial loss to farmers and a threat to food security (Food and Agriculture Organization of the United Nations 2017). Thus, it is urgently needed to find efficient pest management strategies. Pest may be controlled by applying physical (cultivation, mechanical weeding), biological (cultivar choice, crop rotation, antagonists, predators), and chemical measures (pesticides) (Ehler 2006). The integration of biological and chemical control put forth a concept of Integrated Pest Management which involves multiple tactics to control all classes of pests (Ehler 2006). Further, there are few traditional methods of pest control like blacklight traps (Mutwiwa and Tantau 2005) and sticky traps (Pinto-Zevallos and Vänninen 2013). Since backlight traps and sticky traps need to be replaced at a regular time interval, so it is not economical and less effective. Spraying pesticides is one of the solutions but ample and random use of pesticides causes health hazards in human beings by consuming the food (Prathibha et al. 2014). Therefore, early detection and classification of pests play a vital role in crop management, but it is a challenging task and needs to be treated with special attention. Due to the advancement in digital technology, image processing, and artificial intelligence can play a significant role in agricultural research which becomes a catalyst for the researchers to solve the pest detection and classification problem.

Along this line, various methods have been presented in the literature for image-based pest detection and classification on different crops. This includes greenhouse crops like a rose (Boissard, Martin, and Moisan 2008) and agricultural-based crops like rice (Faithpraise et al. 2013), cotton (He et al. 2013), maize (Sena Jr et al. 2003), soybean (Souza et al. 2011), and teagarden (Samanta and Ghosh 2012). The image processing technique based on morphological characteristics has been implemented for the detection of pests like honey bees (Cho et al. 2007) and wasps (Watson, O’Neill, and Kitching 2004). The three most common pests like whiteflies, thrips, and aphids were identified using the thresholding method in YUV color space (Cho et al. 2007). In Qing et al. (2014), an approach has been presented to count rice plant hopper using features like Haar and Histogram of Gradient and classifiers like AdaBoost and Support Vector Machine (SVM).

This paper is focused on the classification of pests in tomato plants. Tomato is one of the most important vegetable crops all around the world. The major tomato-producing countries are China, the European Union, India, the USA, and Turkey (Atherton and Rudich 2012). India has got 2nd position in the agricultural area of tomato as well as in the production of tomato (Rupanagudi et al. 2015). Further, it ranks third in priority after Potato and Onion in India and second after Potato in the world (Atherton and Rudich 2012). However, the cultivation of these crops has undergone a crisis due to pest attacks. In literature, a handful number of research works have been presented on tomato pest classification. Prathibha et al. (2014) have presented an approach to segment the tomato region from the captured images using thresholding and morphological techniques and then the number of Borer (Helicoverpa Armigera) insect has been counted. Similarly, the Borer insect has been detected using k-means clustering and morphological techniques in Rupanagudi et al. (2015). Recently, deep learning (Gautam and Singh 2020) has attracted the researchers because of its superior performance in image recognition tasks including pest detection on tomato plants. In Fuentes et al. (2017), the authors have presented a deep learning-based approach for the detection and classification of tomato plant diseases and pests. They have experimented with three architectures: faster region-based convolutional neural network (Faster R-CNN), region-based fully convolution network (R-FCN), and single-shot multiplex detector (SSD) with various CNN-based feature extractors such as Virtual Geometry Group (VGGNet) and Residual Network (ResNet). It has been reported that the best average precision of 85.98% has been achieved using R-FCN with ResNet-50. Shijie, Peiyi, and Siping (2017) presented a transfer learning approach using VGG16 for detection and classification of tomato plant diseases and pests. Further, they have experimented with VGG16 as a feature extractor and SVM as a classifier. The transfer learning using VGG16 approach performed better than the VGG16+ SVM approach with an average accuracy of 89%. In Nieuwenhuizen, Hemming, and Suh (2018), an approach has been proposed to detect tomato whitefly and its predatory bugs using deep CNN model. The result has been compared with hand-counted insects using the yellow sticky trap method. The average classification accuracy was obtained as 87.40%. In Gutierrez et al. (2019), a comparative study of KNN (K-Nearest Neighbor), SVM, MLP (Multilayer Perceptron), Faster R-CNN, and SSD classifiers has been presented in distinguishing Bemisia Tabacii egg and Trialeurodes Vaporariorum egg tomato pest classes. It has been reported that the best classification accuracy of 82.51% has been obtained using Faster-RCNN. A transfer learning of pre-trained models AlexNet and GoogleNet was used in Brahimi, Boukhalfa, and Moussaoui (2017) for classifying nine tomato diseases. The accuracy of these deep model was reported higher than shallow model like SVM and Random Forest. The following observations have been made from the literature review on tomato pest detection and classification: (i) a handful number of research works have been done and hence, there is a need to explore the image-based tomato pest classification tasks; (ii) The dataset used in most of the research works is mix of tomato plant diseases and pests, which may not result in robust and reliable model for tomato pest classification; (iii) number of pest classes in the considered dataset used in literature are limited to 2–4, which need to be increased.

In this paper, a transfer learning-based approach has been presented for the classification of tomato pest with the objective to minimize aforementioned limitations. In deep neural network, a fundamental problem is the requirement of a large dataset for efficient training of the model. One of the promising solutions to this problem is transfer learning where a pre-trained model on a large dataset is used. Here, we have explored 15 pre-trained deep CNN models and presented an exhaustive comparison of the performance of these models on tomato pest classification. The experiments have been performed on 859 tomatopest images belonging to 10 classes. This study is first of its kind where 10 classes of tomato pests are involved.

The rest of the paper is organized as follows. In Section 2, we have described the methodology which consists of dataset description, transfer learning, and pre-trained deep CNN models. In Section 3, experimental results have been presented followed by discussion in Section 4. Finally, Section 5 concludes our work.

Methodology

Dataset Description

The dataset used in this study has been downloaded from online sources (Flickr 2018; Insect Images 2018; IPM Images 2018; The National Bureau of Agricultural Insect Resources (NBAIR) 2013; The Tamil Nadu Agricultural University [TNAU] 2019). The dataset consists of 859 tomato pest images belonging to 10 classes. All the images are in RGB color space. The details of the dataset have been provided in Table 1. The first class, Bactrocera Litifrons (Shimizu et al. 2007) is one of the damaging insects often affects tomato plant including other plants such as brinjal, bell pepper, and cucurbits. However, the damage is less due to this pest. Bemisia Tabaci (2012) species is another kind of pest which is difficult to predict due to the more affected area. Chrysodeixis Chalcites is an extremely polyphagous pest which feeds on many fruits, vegetables, ornamental crops along with tomato plant. Epilachna Vigintiopunctata (Rajagopal and Trivedi 1989) and Spodoptera Litura (2012) are the major pests that attack solanaceous plants such as tomato and potato. Another fruit borer named Helicoverpa Armigera (2012) bores into fruit which leads to rotting in fruits. Icerya Aegyptiaca (Meena et al. 2012) is a pest having a wide host range. It sucks the cell sap from the leaves and upper soft portion of the plant which results in damage of leaf. One more hazardous pest named Liriomyza Trifolii (2012) reduces production due to its damaging effect. Tuta Absoluta (Desneux et al. 2010) is one of the most devastating pest tomato leaf miners that affects tomato in all growing stages of egg, larvae, pupa, and adult. On the contrary, a pest like Nesidiocoris Tenuis (Pérez-Hedo, Arias-Sanguine, and Urbaneja 2018) is a useful pest capable of inducing plant defenses in tomato due to its phytophagous behavior.

Table 5. Benchmarking of our approach with literature on tomato pest classification.

Authors (year)	Data Size	#Classes	Methodology	Classification Accuracy (%)
Fuentes et al. (2017)	5,000	9 (7 diseases and 2 pest)	Faster R-CNN, R-FCN, SSD	85.98 (R-FCN)
Shijie et al. (2017)	7,040	10 (7 diseases and 3 pest)	VGG16 with transfer learning	89
Nieuwenhuizen et al. (2018)	6,900	4 (pest)	Faster R-CNN	87.4
Gutierrez et al. (2019)	4,331	4 (2 insects and 2 their eggs)	KNN,MLP, Faster R-CNN, SSD	82.51 (Faster-RCNN)
Our Approach	859	10 (pests)	Transfer Learning (DenseNet169)	88.83

Table 1. Details of pest dataset.

Transfer Learning

Deep learning models, especially CNN, have shown tremendous success in classifying images. However, training deep CNN model from scratch usually requires a large amount of data and high computation power. The collection of a large dataset is not always possible, specifically in the domain like agriculture due to local weather condition, indiscrimination of insect pests, and uncontrolled experimental field of invasive species. Further, there may be a problem of overfitting (Skalski 2018), if the deep CNN model is trained with small dataset. Hence, the transfer learningtechnique (Talo et al. 2019) is a solution to this problem. Transfer learning is a technique where the model uses the knowledge gained during the training of a relatively large dataset. The concept of transfer learning has been depicted in Figure 1. The figure shows general deep CNN model pre-trained on large dataset. While training, all the layers of this model are frozen (non-trainable layers) except last layer (trainable layer). Due to this, only the weights of last layer are updated while training. Hence it reduces the computational cost while achieving adequate performance.

Figure 1. The concept of transfer learning.

Pre-trained Deep CNN Model

Here, we have explored 15 pre-trained deep CNN models which are VGG16 (Simonyan and Zisserman 2014), VGG19 (Simonyan and Zisserman 2014), ResNet50V2 (He et al. 2016), ResNet101V2 (He et al. 2016), ResNet152V2 (He et al. 2016), InceptionV3 (Szegedy et al. 2016), Xception (Chollet 2017), InceptionResNetV2 (Szegedy et al. 2017), MobileNet (Howard et al. 2017), DenseNet121 (Huang et al. 2017), DenseNet169 (Huang et al. 2017), DenseNet201 (Huang et al. 2017), NASNetMobile (Zoph et al. 2018), NASNetLarge (Zoph et al. 2018), and MobileNetV2 (Sandler et al. 2018). All the above models are trained on ImageNet dataset (Krizhevsky, Sutskever, and Hinton 2012) which has 1.2 million images belonging to 1,000 categories. Each model has some unique characteristics. VGG16 and VGG19 are a sequential convolutional neural network using 3 × 3 filters. Max-pooling was performed on 2 × 2 pixel window with stride of 2. After each maxpool layer, the number of convolution filters gets doubled in VGG16 and VGG19. As the name specifies, VGG16 has 16 layers whereas VGG19 has 19 layers. The ResNet model (ResNet50V2, ResNet101V2, and ResNet152V2) has the skip connections from earlier layer along with direct connection from the immediate previous layer. InceptionV3 works in blocks and each block consists of parallel existence of convolution filters and pooling layer. It handles the computing resources in a better way. InceptionResNetV2 is the combination of Inception architecture and residual connections. This model has three ensembles residual and one InceptionV3connection. Xception model is the result of depthwise separable convolution implying a complete separation of spatial convolution and cross channel convolution. MobileNet is built from depthwise separable convolutions and followed Inception models to reduce complications in initial few layers. Another model, MobileNetV2, is based on a flip of residual structure where the shortcut connections are between the thin bottleneck layers. In DenseNet, each layer is connected to every other layer in a dense connectivity pattern. It introduces $L\left(L+1\right)/2$ direct connections instead of L layers in other networks. NASNet introduced a new regularization technique called scheduled drop path which improves the performance. Further details of all 15 pre-trained models are provided in Table 2. The discussion on non-trainable and trainable parameters can be found in Section 3.1.

Table 2. Details of pre-trained deep CNN models.

Deep CNN Model	Input Shape	#Convolution Layers	#Pooling Layers	Non-trainable Parameter	Trainable Parameter
VGG16	(224,224,3)	13	5	134,260,544	40,970
VGG19	(224,224,3)	16	5	139,570,240	40,970
ResNet50V2	(224,224,3)	51	1	23,564,800	20,490
ResNet101V2	(224,224,3)	105	1	42,626,560	20,490
ResNet152V2	(224,224,3)	151	4	58,331,648	20,490
Inception	(299,299,3)	83	12	21,802,784	20,490
Xception	(299,299,3)	37	4	20,861,480	20,490
InceptionResNetV2	(299,299,3)	240	6	54,336,736	15,370
MobileNet	(224,224,3)	14	1	4,253,864	10,010
DenseNet121	(224,224,3)	114	4	7,037,504	10,250
DenseNet169	(224,224,3)	152	5	12,642,880	16,650
DenseNet201	(224,224,3)	196	5	18,321,984	19,210
NASNet Mobile	(224,224,3)	183	57	4,269,716	10,570
NASNetLarge	(224,224,3)	227	76	84,916,818	40,330
MobileNetV2	(224,224,3)	34	1	2,257,984	12,810

Results

Experimental Setup

In this subsection, we have provided all the experimental setup to train 15 pre-trained models for tomato pest classification. It can be observed from Table 2 that the input shape varies for each model. Therefore, we have reshaped our tomato pest images to the desired shape as per the requirement of each model. For example, the tomato pest images have been reshaped to $224\times 224\times 3$ for VGG16 model and $299\times 299\times 3$ for Inception model. The second experimental setup is replacing the last fully connected layer which consists of 1,000 neurons to the fully connected layer with 10 neurons. This is done because all the 15 pre-trained models considered here were trained on ImageNet dataset which is having 1,000 classes and hence last layer consists of 1,000 neurons. Whereas, the tomato pest dataset used in this study is having 10 classes and hence last fully connected layer should have 10 neurons. Further, all the layers were frozen while training except last layer based on the concept of transfer learning as shown in Figure 1, i.e., the weights obtained from training of ImageNet dataset were remained intact while training for tomato pest dataset and only the weights of last layer will be updated. Consequently, the number of trainable parameters was drastically reduced as shown in Table 2. Then, we have randomly partitioned the tomato pest dataset into 70% training set, 10% validation set, and 20% test set. Each model has been trained for 100 epochs with a mini-batch size of 8 and learning rate of 0.01. The experiment was performed with Adam (Adaptive Moment Estimation) optimizer. Moreover, we have run our model for five trials(T) to reduce the variability obtained in classification accuracy due to random partitioning of train, validation, and test dataset. Finally, the overall accuracy (OA) has been calculated by averaging the accuracy of five trials. In addition, we have shown standard deviation (STD) of accuracy in five trials which demonstrate the robustness of the model. All experiments have been performed in Python 3.6 with Keras framework having Tensorflow backend. Simulation was carried out in Google Colaboratory that provides Intel(R) Xeon(R) CPU @ 2.30 GHz, 13GB RAM, and NVIDIA Tesla K80 GPU.

Experimental Results

The classification accuracy obtained using 15 pre-trained models on test set of tomato pest dataset has been shown in Table 3. We have shown the classification accuracy for each trial along with OA and STD of five trials. It can be observed that the highest OA of 88.83% with STD of 1.48% has been obtained by applying DenseNet169 model. Further, Figure 2 depicts the graphical view of performance comparison of 15 pre-trained models on tomato pest dataset. For a more detailed analysis, we have calculated the following parameters: Class-wise Accuracy, Precision, Sensitivity, Specificity, and F1 Score using DenseNet169 model as it has produced the highest OA (Table 4).

Table 3. Classification results obtained using 15 pre-trained models on tomato pest dataset.

Deep CNN model	T1	T2	T3	T4	T5	OA±STD
VGG16	79.65	82.55	79.65	83.72	84.30	81.97 ± 2.21
VGG19	85.46	84.30	83.13	83.72	86.04	84.53 ± 1.20
ResNet50V2	79.06	84.88	86.62	83.72	88.37	88.37 ± 3.52
ResNet101V2	82.55	87.79	88.95	88.37	90.11	87.55 ± 2.92
ResNet152V2	80.81	87.20	86.62	87.79	88.37	86.16 ± 3.05
InceptionV3	84.88	86.04	79.65	86.04	81.97	83.72 ± 2.81
Xception	84.30	82.55	88.95	81.39	85.46	84.53 ± 2.92
InceptionResNetV2	81.97	85.46	86.04	81.97	84.88	84.06 ± 1.95
MobileNet	79.06	81.39	86.62	83.13	88.95	83.83 ± 3.97
DenseNet121	87.79	83.13	84.88	86.04	76.74	88.48 ± 2.67
DenseNet169	87.79	90.69	90.11	86.62	88.95	88.83 ± 1.48
DenseNet201	84.30	83.13	75.58	83.72	87.79	82.90 ± 4.47
NASNetLarge	79.65	88.37	77.32	82.55	84.30	82.44 ± 4.26
NASNetMobile	74.41	74.41	74.41	76.16	80.81	76.04 ± 2.77
MobileNetV2	76.16	75.58	72.09	73.25	83.13	76.04 ± 4.29

Table 4. Other performance parameters obtained using denseNet169 model for tomato pest dataset.

Classes	Class Accuracy (%)	Precision (%)	Sensitivity (%)	Specificity (%)	F1-Score (%)
Pest 1	97.3	87.0	89.7	98.4	88.0
Pest 2	95.6	74.2	92.2	96.3	81.9
Pest 3	97.3	86.6	82.0	98.8	84.1
Pest 4	100	100	100	100	100
Pest 5	96.5	85.0	85.6	97.9	85.1
Pest 6	96.8	94.4	70.9	99.6	80.0
Pest 7	97.0	87.8	86.9	98.3	87.3
Pest 8	98.3	85.3	95.6	98.5	90.0
Pest 9	98.9	100	88.6	100	93.3
Pest 10	98.8	93.3	95.4	99.2	94.2

Figure 2. Performance comparison of 15 pre-trained deep CNN models on tomato pest dataset.

Discussion

To highlight the performance of transfer learning approach adopted in this paper for classification of tomato pest dataset, we have presented a benchmarking of our approach with literature (Table 5). From Table 5, it can be observed that the presented transfer learning approach has obtained the highest classification accuracy of 88.83% using DenseNet169 model. Further, it has been observed that the training is done only for single trial probably because of high computation cost and then classification accuracy is computed. But, the model with single trial may not be reliable because of random partition of train, validation, and test set. For different set of train, validation, and test set, the classification accuracy may vary. In this paper, we have used transfer learning approach where computation cost is drastically reduced because of reduction in trainable parameters. Hence, we run our model for five trials and classification accuracy is computed by averaging the classification accuracies of five trials. Consequently, we have computed STD of five trials and the low STD shows the reliability of the system. The main advantages of presented work are summarized as follows:

• Most of the studies are focused on the classification of tomato leaf diseases or mix dataset of diseases and pests. In this study, we have focussed on classification of tomato pests only.

• To the best of our knowledge, this is the first study where 10 tomato pest classes are involved.

• The exhaustive comparison of 15 pre-trained deep CNN models for tomato pest classification has been presented.

Conclusion

Tomato pest detection and classification have been performed with images obtained from online resources using transfer learning of deep CNN models. In this study, we employed 15 pre-trained models to classify tomato pest dataset. Our results showed that the DenseNet169 model obtained the highest classification accuracy of 88.83%±1.48% among the 15 models. The presented transfer learning approach shows the encouraging results and demonstrates its ability to classify tomato pests. In the future, we intent to work on data augmentation to generate a large dataset and train the deep CNN model from scratch for tomato pest classification.

Declaration of Interest Statement

Authors declare no conflict of interest.

Additional information

Funding

No funding has been received for this research.