Fine-grained Potato Disease Identification Based on Contrastive Convolutional Neural Networks

Figures & data

Figure 1. CLCNN’s model architecture.

Figure 2. Examples of potato early blight and late blight in each period.

Table 1. Definition of disease period of potato early blight and late blight.

	Early_1	Early_2	Early_3	Early_4	Late_1	Late_2	Late_3	Late_4
number of lesions	0-3	4-10	10-20	>20	1	1-2	3-5	>5
total lesion area	<5%	5%-15%	15%-30%	>30%	<5%	5%-10%	10%-20%	>20%

Table 2. Network structure of CLCNN’s encoder. $Vgg{16}^{\ast }$ represents the feature extractor and max pooling layer of Vgg16. Please refer to (Simonyan and Zisserman 2014) for the specific network structure of Vgg16.

Module	Network	Input size	Output size
Encoder	$Vgg{16}^{\ast }$	650*65	25088

Table 3. Network structure of CLCNN’s classifier. Vgg16** represents the classifier of Vgg16 that lacks the last fully connected layer. Please refer to (Simonyan and Zisserman 2014) for the specific network structure of Vgg16.

Module	Network	Input size	Output size
Classifier	$Vgg{16}^{\ast \ast }$	25088	1000
	Fully connected layer	1000	8
	Softmax	8	8

Table 4. Network structure of CLCNN’s projection head.

Module	Network	Input size	Output size
Projection head	Fully connected layer	25088	512
	BatchNorm	512	512
	ReLU	512	512
	Fully connected layer	512	128

Table

Algorithm 1:CLCNN’s main learning algorithm.
Input: Potato disease images $X$, batch size $n$, structure of $\mathbb{E}$,$\mathbb{C}$,$\mathbb{P}$, function of $\mathrm{\Gamma }$,$f$,$g$, constant $\tau$
1 for sampled minibatch $\{x_i{\}}_{i=1}^n$ do
2 for $i\in \{1,\dots ,n\}$ do
3 $\{a_{7i-6},\dots ,a_{7i}\}=\mathrm{\Gamma }(x_i)$	# Data augmentation
4 define $P=\{\}$	# Positive examples
5 define $N=\{\}$	# Negative examples
6 for $j\in \{1,\dots ,7n\}$ do
7 $e_j=\mathbb{E}(a_j)$	# Encoder
8 $y_j=\mathbb{C}(e_j)$	# Classifier
9 $z_j=\mathbb{P}(e_j)$	# Projection head
10 # get category function $f$
11 # get most category function $g$
12 if $f(z_i)=g(Z)$ then
13 push $z_i$ to $P$
14 else
15 push $z_i$ to $N$
16 define $\alpha =0$# Positive examples’ similarity
17 define $\beta =0$# Negative examples’ similarity
18 for $k\in \{1,\dots ,7n\}$ do
19 for $m\in \{1,\dots ,7n\}$ do
20 if $z_k\in P$ and $z_m\in P$ and $z_k\ne z_m$ then
21 $\alpha =\alpha +z_k\cdot z_m$	# Update $\alpha$
22 if $z_k\in N$ and $z_m\in N$ and $z_k\ne z_m$ then
23 $\beta =\beta +z_k\cdot z_m$	# Update $\beta$
24 # CrossEntropy loss
25 define ${\mathrm{\ell }}_{ce}={\sum }_{j\in 7n}lable(y_j)log(y_j)$
26 # Contrastive loss
27 define ${\mathrm{\ell }}_c=-log\frac{\mathrm{e}\mathrm{x}\mathrm{p}(\alpha /\tau )}{\mathrm{e}\mathrm{x}\mathrm{p}(\alpha /\tau )+\mathrm{e}\mathrm{x}\mathrm{p}(\beta /\tau )}$
28 # Joint loss
29 define $\mathcal{L}={\mathrm{\ell }}_{ce}+{\mathrm{\ell }}_c$
30 Update networks’ parameters to minimize $\mathcal{L}$
Output:Predicted potato diseases $Y$

Figure 3. Process of CLCNN’s experiments.

Figure 4. The experimental process of using Mask-RCNN to segment plant diseased leaves.

Figure 5. Six methods of data augmentation.

Table 5. Results of data augmentation.

	Early_1	Early_2	Early_3	Early_4	Late_1	Late_2	Late_3	Late_4	total
original	10	10	28	25	6	25	32	33	169
augmentated	70	70	196	175	42	175	224	231	1183

Table 6. Hyperparameters of CLCNN. $\tau$ refers to Equation 13(13) ${\mathrm{\ell }}_c=-log\frac{\mathrm{e}\mathrm{x}\mathrm{p}(\alpha /\tau )}{\mathrm{e}\mathrm{x}\mathrm{p}(\alpha /\tau )+\mathrm{e}\mathrm{x}\mathrm{p}(\beta /\tau )}$(13) .

model	hyperparameters	value
CLCNN	learn_rate	1e-05
	$\tau$	0.8
	batch_size	10
	epochs	200
	rand_seed	42

Figure 6. Confusion matrix of the classification results.

Figure 7. Accuracy curves for training and testing of each model.

Table 7. The highest accuracy of the 6 models on test set.

Models	Resnet50	Resnet101	AlexNet	Inception V3	Vgg16	CLCNN
accuracy	90.27	90.62	93.05	94.44	94.79	97.24

Figure 8. The highest accuracy of CLCNN on different hyperparameters. lr denotes learn_rate. $\tau$ refers to Equation 13(13) ${\mathrm{\ell }}_c=-log\frac{\mathrm{e}\mathrm{x}\mathrm{p}(\alpha /\tau )}{\mathrm{e}\mathrm{x}\mathrm{p}(\alpha /\tau )+\mathrm{e}\mathrm{x}\mathrm{p}(\beta /\tau )}$(13) .

Table 8. Precision, recall, specificity, and F1 score of Vgg16 and CLCNN in each period.

Vgg16	Early_1	Early_2	Early_3	Early_4	Late_1	Late_2	Late_3	Late_4
precision	1.0	1.0	0.961	1.0	0.933	0.93	0.902	0.942
recall	0.667	1.0	1.0	0.886	1.0	0.952	0.982	0.98
specificity	1.0	1.0	0.992	1.0	0.996	0.988	0.974	0.987
F1 score	0.8	1.0	0.98	0.9394	0.965	0.9412	0.9402	0.9608
CLCNN	Early_1	Early_2	Early_3	Early_4	Late_1	Late_2	Late_3	Late_4
precision	1.0	0.84	1.0	1.0	0.933	1.0	0.949	1.0
recall	0.667	1.0	1.0	1.0	1.0	0.976	1.0	1.0
specificity	1.0	0.985	1.0	1.0	0.996	1.0	0.987	1.0
F1 score	0.8	0.913	1.0	1.0	0.965	0.988	0.973	1.0

Simonyan, K., and A. Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:14091556. doi:10.48550/arXiv.1409.1556.

 Google Scholar