Detecting diseases in apple tree leaves using FPN–ISResNet–Faster RCNN

Figures & data

Figure 1. Apple leaf diseases considered in this study: (a) powdery mildew, (b) Alternaria, and (c) rust disease.

Figure 2. Framework of the FPN–ISResNet–Faster RCNN model, where C2–C5 represent the predicted feature layers, in represents the inception module, P2–P5 represent the predicted outputs of the feature graphs from the FPN, ROI represents the region of interest, and Cls_score represents the classification output by softmax.

Table 1. ISResNet structure.

Output size	Structure name	Kernel size/Stride
112 × 112	Conv1	7 × 7/2，3 × 3/2
56 × 56	Max Pooling	3 × 3/2
	Conv2_x	$\left[\begin{array}{c}1\times 1,64\\ 3\times 3,64\\ 1\times 1,256\end{array}\right]\times 3$
	SE block_1	FC, [16,256]
28 × 28	Conv3_x	$\left[\begin{array}{c}1\times 1,128\\ 3\times 3,128\\ 1\times 1,512\end{array}\right]\times 4$
	Inception block_1	\
14 × 14	Conv4_x	$\left[\begin{array}{c}1\times 1,256\\ 3\times 3,256\\ 1\times 1,1024\end{array}\right]\times 6$
	Inception block_2	\
7 × 7	Conv5_x	$\left[\begin{array}{c}1\times 1,512\\ 3\times 3,512\\ 1\times 1,2048\end{array}\right]\times$3
	SE block_2	FC,[16,2048]
1 × 1	Average pooling	Stride 1
	SoftMax	3

Table 2. Different reduction coefficients (r) obtained from SE-Resnet-50 to determine its best value (J. Hu et al., 2020).

r	2	4	8	16	32
Top-1 error (%)	22.29	22.25	22.26	22.28	22.72
Top-5 error (%)	6.00	6.09	5.99	6.03	6.20
Parameter size (M)	45.7	35.7	30.7	28.1	26.9

Table 3. Comparison of the accuracies and speeds of different models for detecting apple leaf diseases.

Method	SSD	Faster RCNN					FPN-ISResNet -Faster RCNN
Feature extraction network	ResNet50	VGG16	GoogleNet	ResNet50	ResNeXt	ISResNet	ISResNet
Iterations	100	100	100	100	100	100	100
Powdery mildew (AP50, %)	84.77	86.68	81.73	88.12	88.42	90.86	90.43
Alternaria leaf spot (AP50, %)	81.69	82.36	79.61	83.94	81.38	87.96	93.34
Rust disease (AP50, %)	80.72	81.07	75.49	83.08	86.27	89.14	96.94
AP (%)	50.28	50.04	51.46	55.78	56.41	58.17	62.71
AP50 (%)	82.56	83.37	78.95	85.04	85.35	89.32	93.68
AP75 (%)	53.22	52.93	55.73	63.19	63.73	64.36	70.94
Detection speed (FPS)	46.89	16.42	24.42	18.40	15.03	18.87	14.87

Figure 3. Visualized detection results of (A) powdery mildew, (B) Alternaria leaf spots, (C) rust disease, and (D) Alternaria leaf spots and rust disease in apple tree leaves using the FPN–ISResNet–Faster RCNN model. (E) Identification errors in the left image labeled with (1) and omissions in the right image labelled with (2).

Figure 4. A UAV image of the apple orchard (e) with four sampled apple trees (a, b, c, and, d) in Shatan village, Enhe Town, Zhongning County, Ningxia, China acquired at 16:00 on July 8, 2021.

Figure 5. Distribution of powdery mildew, Alternaria leaf spots, and rust disease in apple tree leaves in magnified images of the (a) northern and (b) southern orchard.

Table 4. Precision rates of feature extraction networks used to detect apple leaf diseases.

Feature extraction network	Input size	Precision rate (%)
VGG16	224 × 224	96.25
GoogleNet	224 × 224	96.84
ResNet18	224 × 224	93.66
ResNet34	224 × 224	96.44
ResNet50	224 × 224	97.35
ResNet101	224 × 224	95.76
ResNeXt	224 × 224	97.46
SE-ResNet	224 × 224	98.10
ISResNet	224 × 224	98.79

Table 5. Accuracies of ResNet50 with different positions and numbers of SENet modules after the convolution layer for the detection of apple leaf diseases.

Position of SENet	Quantity of SENet	Accuracy (%)
Conv2_x+Conv3_x	2	97.09
Conv2_x+Conv4_x	2	97.41
Conv2_x+Conv5_x	2	98.10
Conv3_x+Conv4_x	2	96.68
Conv3_x+Conv5_x	2	96.78
Conv4_x+Conv5_x	2	97.01
Conv2_x+Conv3_x+Conv4_x	3	96.93
Conv2_x+Conv3_x+Conv5_x	3	96.76
Conv3_x+Conv4_x+Conv5_x	3	97.01
All blocks	4	96.49

Table 6. Accuracies of ResNet50 with different positions and numbers of inception modules after the convolution layers for the detection of apple leaf diseases.

Position of Inception	Quantity of Inception	Accuracy (%)
Conv2_x+Conv3_x	2	96.76
Conv2_x+Conv4_x	2	96.71
Conv2_x+Conv5_x	2	96.93
Conv3_x+Conv4_x	2	98.79
Conv3_x+Conv5_x	2	96.93
Conv4_x+Conv5_x	2	96.93
Conv2_x+Conv3_x+Conv4_x	3	97.09
Conv2_x+Conv3_x+Conv5_x	3	97.01
Conv3_x+Conv4_x+Conv5_x	3	96.69
All Blocks	4	96.68

Table 7. Comparison of detection accuracies of apple leaf diseases obtained from models with and without FPNs.

Disease and accuracy	ISResNet-Faster RCNN	FPN-ISResNet -Faster RCNN	Increment (+) or decrement (−)
Powdery mildew (AP50, %)	90.86	90.43	−0.43
Alternaria leaf spot (AP50, %)	87.96	93.34	+5.38
Rust disease (AP50, %)	89.14	96.94	+7.8
AP (%)	58.17	62.71	+4.54
AP50 (%)	89.88	93.68	+3.80
AP75 (%)	64.36	70.94	+6.58

Table 8. Comparison of detection accuracies of apple leaf diseases obtained by the proposed model with different scales of anchor frames.

Anchor frame	Anchor scale	Aspect ratio	AP (%)	AP50 (%)	AP75 (%)
3 Scales and 3 ratios	{128^2, 256^2, 512^2}	{1:2, 1:1, 2:1}	48.87	78.79	57.19
5 Scales and 3 ratios	{32^2, 64^2, 128^2, 256^2, 512^2}	{1:2, 1:1, 2:1}	53.36	83.88	59.86
6 Scales and 3 ratios	{16^2, 32^2, 64^2, 128^2, 256^2, 512^2}	{1:2, 1:1, 2:1}	62.71	93.68	70.94
9 Scales and 3 ratios	{16^2, 32^2, 48^2, 64^2, 96^2, 128^2, 192^2, 256^2, 512^2}	{1:2, 1:1, 2:1}	55.57	81.29	59.65

Hu, J., Shen, L., Albanie, S., Sun, G., & Wu, E. (2020). Squeeze-and-excitation networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(8), 2011–2023. https://doi.org/10.1109/TPAMI.2019.2913372

 PubMed Web of Science ®Google Scholar

Data availability statement

Dataset(s) derived from public resources and made available with the article. https://www.kaggle.com/competitions/plant-pathology-2021-fgvc8/data