Deep learning for processing and analysis of remote sensing big data: a technical review

Figures & data

Table 1. Some common data sources of remote sensing big data.

Category	Data Source	Spatial (m)	Duration (year) Temporal (day)	Spectral
Satellite (optical)	Landsat 1-8	15 – 120	1972 – now 16 – 18	4 – 11
	MODIS	250 – 1000	1999 – now 1 – 2	36
	SPOT 1-7	1.5 – 20	1987 – now 26	4 – 5
	Sentinel-2	10 – 60	2015 – now 5 – 10	13
	Worldview 1-4	0.31 – 1.64	2007 – now 1 – 3	1 – 17
	HJ-1 (China)	30 – 300	2008 – now 4	4 – 8^a
	GaoFen-1/6 (China)	2 – 16	2013 – now 4	4 – 9
Satellite (SAR)	Sentinel-1	5 – 20	2014 – now 6 – 12	Single & Dual
	COSMO-SkyMed	1 – 15	2007 – now 1 – 16
	RADASAT 1-2	3 – 100	1995 – now 1 – 24	Single & Dual & Quad
	ALOS-1-2	3 – 100	2006 – now 14 – 46
	TerraSAR-X	1 – 16	2007 – now 11
	GanFen-3 (China)	1 – 500	2016 – now <3
UAV	Multispectral camera	0.02 – 0.2 m	on demand	3 – 4
	LiDAR	/		1
Sensor Networks	Hydrology (level, flow, sediment, precipitation, evaporation)	/	on demand	/
	Meteorology (temperature, humidity, pressure, wind, cloud)	/	on demand	/

^aNo consideration on the hyperspectral sensor on HJ-1 satellite

Table 2. List of popular deep learning architectures in remote sensing tasks.

Model	Features and advantages	Variants or implementation	Tasks and references
CNN	Image processing and analysis	LeNet, R-CNN, VggNet, ResNet, U-Net	geometric processing (Wang et al., 2018d), cloud masking (Wu et al., 2020a), object extraction (Abdollahi, Pradhan, Shukla, Chakraborty, & Alamri, 2020), semantic segmentation (Waldner & Diakogiannis, 2020)
RNN	Sequential data and time-series analysis	LSTM, BiLSTM, GRU	sequential prediction (Zhou, Yang, & Feng et al., 2020), sequential classification (Zhou et al., 2019), change detection (Lyu, Lu, & Mou, 2016)
AE	Feature extraction and representation	Stacked AE, Spare AE, Variational AE, Denoising AE	feature extraction (Shao et al., 2017; Zhou et al., 2020; Xing, Wang, Yang, & Jiao, 2018; Huang, Xiao, Wei, Liu, & Tang, 2015)
GAN	Data generation, unsupervised learning	Conditional GAN, f-divergence GAN, Cycle-consistent GAN	data fusion (Liu et al., 2018c), data reconstruction (Bermudez, Happ, Feitosa, & Oliveira, 2019)
GNN	network analysis of irregular data	RGNN, GCN, GAT, GAE, STGNN	image classification (Qin et al., 2018; Mou, Lu, Li, & Zhu, 2020; Wan et al., 2020)

Figure 1. The general technical pipeline of processing and analysis for remote sensing big data.

Wang, S., Quan, D., Liang, X., Ning, M., Guo, Y., & Jiao, L. (2018d). A deep learning framework for remote sensing image registration. ISPRS Journal of Photogrammetry and Remote Sensing, 145, 148–164.

 Web of Science ®Google Scholar

Wu, W., Gao, X., Fan, J., Xia, L., Luo, J., & Zhou, Y. (2020a). Improved mask R-CNN-based cloud masking method for remote sensing images. International Journal of Remote Sensing, 41(23), 8910–8933.

 Web of Science ®Google Scholar

Abdollahi, A., Pradhan, B., Shukla, N., Chakraborty, S., & Alamri, A. (2020). Deep learning approaches applied to remote sensing datasets for road extraction: A state-of-the-art review. Remote Sensing, 12(9), 1444.

 Web of Science ®Google Scholar

Waldner, F., & Diakogiannis, F. I. (2020). Deep learning on edge: Extracting field boundaries from satellite images with a convolutional neural network. Remote Sensing of Environment, 245, 111741.

 Web of Science ®Google Scholar

Zhou, Y. N., Yang, X., Feng, L., Wu, W., Wu, T., Luo, J., & Zhang, X. (2020). Superpixel-based time-series reconstruction for optical images incorporating SAR data using autoencoder networks. GIScience & Remote Sensing, 57(8), 1005–1025

 Google Scholar

Zhou, Y., Luo, J., Feng, L., Yang, Y., Chen, Y., & Wu, W. (2019). Long-short-term-memory-based crop classification using high-resolution optical images and multi-temporal SAR data. GIScience & Remote Sensing, 56(8), 1170–1191.

 Web of Science ®Google Scholar

Lyu, H., Lu, H., & Mou, L. (2016). Learning a transferable change rule from a recurrent neural network for land cover change detection. Remote Sensing, 8(6), 506.

 Web of Science ®Google Scholar

Shao, Z., Deng, J., Wang, L., Fan, Y., Sumari, N., & Cheng, Q. (2017). Fuzzy autoencode based cloud detection for remote sensing imagery. Remote Sensing, 9(4), 311.

 Web of Science ®Google Scholar

Xing, Y., Wang, M., Yang, S., & Jiao, L. (2018). Pan-sharpening via deep metric learning. ISPRS Journal of Photogrammetry and Remote Sensing, 145, 165–183.

 Web of Science ®Google Scholar

Huang, W., Xiao, L., Wei, Z., Liu, H., & Tang, S. (2015). A new pan-sharpening method with deep neural networks. IEEE Geoscience and Remote Sensing Letters, 12(5), 1037–1041.

 Web of Science ®Google Scholar

Liu, Y., Chen, X., Wang, Z., Wang, Z. J., Ward, R. K., & Wang, X. (2018b). Deep learning for pixel-level image fusion: Recent advances and future prospects. Information Fusion, 42, 158–173.

 Web of Science ®Google Scholar

Bermudez, J. D., Happ, P. N., Feitosa, R. Q., & Oliveira, D. A. B. (2019). Synthesis of multispectral optical images from SAR/optical multitemporal data using conditional generative adversarial networks. IEEE Geoscience and Remote Sensing Letters, 16(8), 1220–1224.

 Web of Science ®Google Scholar

Qin, A., Shang, Z., Tian, J., Wang, Y., Zhang, T., & Tang, Y. Y. (2018). Spectral–spatial graph convolutional networks for semisupervised hyperspectral image classification. IEEE Geoscience and Remote Sensing Letters, 16(2), 241–245.

 Web of Science ®Google Scholar

Mou, L., Lu, X., Li, X., & Zhu, X. X. (2020). Nonlocal graph convolutional networks for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 58(12), 8246–8257.

 Web of Science ®Google Scholar

Wan, S., Gong, C., Zhong, P., Du, B., Zhang, L., & Yang, J. (2020). Multiscale dynamic graph convolutional network for Hyperspectral Image Classification. IEEE Transactions on Geoscience and Remote Sensing, 58(5), 3162–3177.

 Web of Science ®Google Scholar

Data availability statement

Data sharing is not applicable to this article as no new data were created or analysed in this study.