Long-short-term-memory-based crop classification using high-resolution optical images and multi-temporal SAR data

Figures & data

Figure 1. Flowchart of the LSTM-based time series analysis method for crop classification.

Figure 2. Farmland parcels and field survey samples for crop classification.

Table 1. Sentinel-1 SAR data list used in this study (the data between DOY 154 and DOY 178 is missing).

Path/Frame	Day of Year	Path/Frame	Day of Year	Path/Frame	Day of Year	Path/Frame	Day of Year
11/94	−2(2016)	11/94	10	11/94	22	11/94	34
11/94	46	11/94	58	11/91	70	11/91	82
11/91	94	11/91	106	11/91	118	11/91	130
11/91	142	11/91	154	11/91	178	11/91	190
11/91	202	11/91	214	11/91	226	11/91	238
11/91	250	11/91	262	11/91	274	11/91	286
11/91	298	11/91	310	11/91	322	11/91	334
11/91	346	11/91	358	11/91	370(2018)

Figure 3. Sample augmentation for LSTM-based classification.

Figure 4. Network structure of the LSTM-based classifier used for time series analysis.

Figure 5. The study area in northern Hunan Province, China. The right-hand section provides an overview of the SAR data (R: 23 May 2017 VH polarization, G: 26 October 2017 VH polarization, B: 3 August 2017 VV polarization), roadmaps and sample spots of the field survey.

Figure 6. Results of parcel-based crop classification for Hunan using the proposed method.

Figure 7. Local details of parcel-based crop classification for Hunan using the proposed method.

Table 2. Comparison of overall accuracy, kappa coefficient and F1 measure values conducted using the SVM-, RF-, and LSTM-based classifiers for Hunan.

	SVM			RF			LSTM
crops	UA(%)	PA(%)	F1	UA(%)	PA(%)	F1	UA(%)	PA(%)	F1
rice	76.53	79.35	77.91	65.31	69.63	67.29	80.64	81.59	81.11
double rice	72.71	75.48	74.07	77.68	79.26	78.46	82.76	83.92	83.34
rice-rape	79.82	82.99	81.37	80.78	82.56	81.66	86.82	86.74	86.78
rape-cotton	76.63	74.95	75.78	83.82	85.43	84.62	81.42	83.23	82.32
rape-rice	74.12	77.70	75.87	74.62	75.14	74.88	84.57	85.76	85.16
rape-rice-rape	73.71	72.40	73.05	78.63	80.39	79.50	86.71	88.12	87.52
other	68.76	72.19	70.43	69.94	72.54	71.22	77.15	79.38	78.25
OA	75.50			76.82			83.67
kappa	71.79			73.58			80.91

Figure 8. VH intensity time-series curves for crops (the upper section) and their corresponding crop calendars (the lower section) for Hunan.

Table 3. Classification performance using multivariate and univariable time series for Hunan.

		Variables
classifier	Accuracy(%)	VH+VV+R	VH+VV	VH	VV
SVM	OA	75.38	75.50	71.68	72.94
	Kappa	72.07	71.79	66.83	68.96
RF	OA	77.37	76.82	74.09	72.41
	Kappa	73.13	73.58	70.57	69.12
LSTM	OA	84.21	83.67	78.43	77.06
	Kappa	81.40	80.91	74.36	72.64

Figure 9. Classification performance as a function of the number of neurons in hidden layers.

Figure 10. Classification performance as a function of time series.

Table 4. Comparison of overall accuracy values for Guizhou using the SVM-, RF-, and LSTM-based classifiers.

classifier	SVM	RF	LSTM
OA (%)	72.64	74.19	80.71

Figure 11. Results of the parcel-based crop classification for Guizhou using the proposed method.

Figure 12. Local details of the parcel-based crop classification for Guizhou using the proposed method.