Applicability of UAV-based optical imagery and classification algorithms for detecting pine wilt disease at different infection stages

Figures & data

Figure 1. Location of the study areas. (a) represents the location of the study areas; (b) and (c) denote the true color composite images from 639 nm, 550 nm, and 470 nm of Site a and Site B used in this study; (d) and (e) are the live photo of the study area.

Table 1. The segmentation indices and thresholds of each dataset.

Dataset	Segmentation indices	Thresholds
HSI	$\mathrm{N}\mathrm{D}\mathrm{V}\mathrm{I}\left(860,640\right)=\left(R_{860}-R_{640}\right)/\left(R_{860}+R_{640}\right)$	−1≤NDVI(860,640)≤0.7
	$\mathrm{S}\mathrm{i}\mathrm{m}\mathrm{p}\mathrm{l}\mathrm{e}\mathrm{I}\mathrm{n}\mathrm{d}\mathrm{e}\mathrm{x}=R_{550}+R_{766}-R_{679}$	$0\le \mathrm{S}\mathrm{i}\mathrm{m}\mathrm{p}\mathrm{l}\mathrm{e}\mathrm{I}\mathrm{n}\mathrm{d}\mathrm{e}\mathrm{x}\le 0.2$
MSI/MSI&TIR	$\mathrm{R}\mathrm{V}\mathrm{I}=\mathrm{R}\mathrm{e}\mathrm{d}/\mathrm{N}\mathrm{I}\mathrm{R}$	$\mathrm{m}\mathrm{i}\mathrm{n}\le \mathrm{R}\mathrm{V}\mathrm{I}\le 0.105\hspace{0.17em}$*

*indicates the effective values of RVI are between the minimum value and 0.105.

Figure 2. UAV-based remote-sensing data analysis processing framework for a comprehensive assessment of different PWD infection stages.

Figure 3. Typical images of different PWD infection stages and corresponding categorized standards of each stage in this study.

Table 2. The detailed information of training, validation, and test sets for different classification algorithms.

Algorithms	Training set construction criterion	Detailed information*
		Stages	Training set	Validation set	Test set
2D-CNN & 3D-CNN	Taking each pixel as the center, constructing the space-spectral cube of 19 × 19 × 270 (for HSI) and 43 × 43 × 270 (for MSI and MSI&TIR) size as the sliding window size (for 3D-CNN) and setting 1 pixel as the sliding step size, all extracting space-spectral cube and the corresponding label as sample data of each dataset. The space-spectral cube size of 2D-CNN are 19 × 19 × 3 (for HSI), 43 × 43 × 5 (for MSI) and 43 × 43 × 6 (for MSI&TIR).	Healthy	39582	21983	21072
		Chlorosis	39582	21567	20055
		Red	74156	40562	34101
		Gray	31247	13795	15444
RF & SVM	Tacking the center point of each ROI as the center, selecting all pixels points within 3 × 3 size and the corresponding label as sample data of each data set.	Healthy	2394	21983	21072
		Chlorosis	2394	21567	20055
		Red	5427	40562	34101
		Gray	1476	13795	15444
Spectral Analysis	Extracting the mean spectral reflectance of each ROI and the corresponding label as sample data of the training data set, tacking the center point of each ROI and the corresponding label as sample data of the test data set.	Healthy	266	21983	21072
		Chlorosis	266	21567	20055
		Red	603	40562	34101
		Gray	163	13795	15444

*The samples number of training sets for different algorithms has different sample size statistical basis. 2D-CNN and 3D-CNN are based on the space-spectral cube; RF and SVM are based on the pixel number; and spectral analysis is based on the canopy crown ROI number. The validation and test set were counted according to the actual surveyed pixels of diseased trees.

Figure 4. Architecture of the 2D- and 3D-CNN models for HSI dataset. (A) is the architecture of 2D-CNN and (B) is for 3D-CNN.

Figure 5. SPA bands selection result. (a) Variation in RMSE as the number of selected bands increase. (b) Final optimal selected bands.

Figure 6. Scatterplot of different simple indices and the classification performance and thresholds of three construction spectral indices. (a) and (b) are the simple indices for the HSI dataset; (c) and (d) are the simple indices for the MSI dataset; (d), (e) and (f) are the classification performance of each spectral indices for HSI, MSI and MSI&Tir dataset. The different color dots are the spectral index values of four corresponding infection stages.

Table 3. The overall accuracy and kappa coefficient of validation set under different classification algorithms for different datasets.

Algorithms		HSI	MSI	MSI&TIR
Spectral Analysis	Model	$\mathrm{P}\mathrm{H}\mathrm{S}\mathrm{I}=\hspace{0.17em}5.5\times {\mathrm{R}}_{911}-33\times {\mathrm{R}}_{413}+\mathrm{N}\mathrm{D}\mathrm{S}\mathrm{I}\left(691,720\right)$	$\mathrm{P}\mathrm{M}\mathrm{S}\mathrm{I}=8.4\times {\mathrm{R}}_{840}-36\times {\mathrm{R}}_{475}+\mathrm{N}\mathrm{D}\mathrm{S}\mathrm{I}\left(668,717\right)$	$\mathrm{P}\mathrm{T}\mathrm{S}\mathrm{I}=8.4\times {\mathrm{R}}_{840}-36\times {\mathrm{R}}_{475}+\mathrm{N}\mathrm{D}\mathrm{S}\mathrm{I}\left(668,717\right)-0.1\times {\mathrm{R}}_{11000}$
	OA/%	73.54	84.37	85.15
	Kappa	0.68	0.72	0.86
SVM	Model	SVM(20,0.5)^a	SVM(100,0.2)	SVM(100,0.17)
	OA/%	87.94	86.14	89.65
	Kappa	0.86	0.77	0.88
RF	Model	RF-1000^b	RF-100	RF-100
	OA/%	93.63	87.20	97.22
	Kappa	0.91	0.83	0.97
2D-CNN	Model	three convolutional layers and two pooling layers with the fixed kernel of 5 × 5	three convolutional layers and two pooling layers with the fixed kernel of 5 × 5	three convolutional layers and two pooling layers with the fixed kernel of 5 × 5
	OA/%	89.62	86.45	94.91
	Kappa	0.81	0.81	0.92
3D-CNN	Model	Five 3D convolutional layers with the fixed kernel of 3 × 3 × 7	Two 3D convolutional layers with the fixed kernel 3 × 3 × 3	Two 3D convolutional layers with the fixed kernel 3 × 3 × 3
	OA/%	92.62	86.32	97.58
	Kappa	0.90	0.81	0.96

^aSVM(i,j) indicates that the SVM classification is performed using a parameter combination of (penalty coefficient, the kernel parameter) in the G-RBF kernel.

^bRF-n indicates that the fandom forest classification is performed using the n trees.

Figure 7. The classification confusion matrices (percent) of different algorithms with different datasets. The horizontal axis is the ground truth and the vertical axis is the classification result.

Table 4. The overall accuracy and kappa coefficient of test set under different classification algorithms for different datasets.

Algorithms		HSI	MSI	MSI&TIR
Spectral Analysis	OA/%	63.10	81.22	81.60
	Kappa	0.48	0.74	0.75
SVM	OA/%	84.90	81.54	92.32
	Kappa	0.79	0.75	0.90
RF	OA/%	87.60	84.79	94.26
	Kappa	0.83	0.79	0.92
2D-CNN	OA/%	85.27	81.33	90.48
	Kappa	0.79	0.75	0.87
3D-CNN	OA/%	87.46	81.39	93.30
	Kappa	0.83	0.74	0.90

Table 5. The two-way ANOVA results.

Source	SS	df	MS	F	P-value	F crit
Methods	605.26	4	151.32	12.82	<0.001	3.06
Data sets	358.82	2	179.41	15.20	<0.001	3.68
Interaction	292.32	8	36.54	3.10	0.03	2.64
Internal	177.05	15	11.80
Total	1433.46	29

Figure 8. The classification results of different algorithms using different datasets.

Figure 9. Comparison mapping with investigation results. Circles and crosses indicate the remote sensing classification results and investigation results, respectively.

Table 6. Comparisons of classification accuracy among different data sources.

Evaluating Indicators	TIR	Composited RGB	RGB&TIR	MSI	MSI&TRI
Healthy stage (%)	68.17	90.32	90.62	87.27	100.00
Chlorosis Stage (%)	45.78	56.98	71.83	69.27	92.73
Red Stage (%)	49.72	65.86	83.45	99.23	91.97
Gray Stage (%)	52.12	84.12	89.33	95.05	97.38
OA (%)	53.22	72.54	81.64	84.79	94.26
Kappa	0.50	0.70	0.74	0.79	0.92

Figure 10. Classification results and infected tree crown delineation maps based on the MSI&Tir dataset.

Table 7. Comparisons of classification accuracy among different spatial resolution datasets.

Evaluating Indicators	Original 0.053 m	Resampled 0.3m	Resampled 0.8m	Resampled 1.5m
Healthy stage (%)	100	95.12	89.32	80.32
Chlorosis Stage (%)	92.73	90.21	74.78	59.33
Red Stage (%)	91.97	90.89	84.93	72.18
Gray Stage (%)	97.38	97.22	86.64	74.54
OA (%)	94.26	92.83	86.98	69.93
Kappa	0.92	0.90	0.86	0.65

Data availability statement

The data that support the findings of this study are available from the first author, [Ning Zhang, [email protected]], upon reasonable request.