Cloud detection using sentinel 2 imageries: a comparison of XGBoost, RF, SVM, and CNN algorithms

Figures & data

Figure 1. Process of texture feature extraction.

Table 1. Combination of texture features of Sentinel-2.

S. No.	Features	Combination Operation
1	S + B	Concatenate (S, B)
2	S + G	Concatenate (S, G)
3	S + M	Concatenate (S, M)
4	B + G	Concatenate (B, G)
5	B + M	Concatenate (B, M)
6	G + M	Concatenate (G, M)
7	S + B+G	Concatenate (S, B, G)
8	S + B+M	Concatenate (S, B, M)
9	S + G+M	Concatenate (S, G, M)
10	B + G+M	Concatenate (B, G, M)
11	S + B+G + M	Concatenate (S, B, G, M)

Here, S: Spectral; B: Bilateral Filtering; G: GLCM, and M: Morphological profile analysis.

Figure 2. Process of deep feature extraction.

Table 2. Setup of considered Residual Network (Resnet).

Layer/Block	Output	Connected to
Input Layer	(None, 5, 5, 4)	[ ]
Convolution 2D	(None, 5, 5, 64)	Input Layer
Batch_Normalization	(None, 5, 5, 64)	Convolution 2D
Activation	(None, 5, 5, 64)	Batch_Normalization
Convolution Block	(None, 5, 5, 32)	Activation
Identity Block	(None, 5, 5, 32)	Convolution Block
Convolution Block	(None, 5, 5, 32)	Identity Block
Identity Block	(None, 5, 5, 32)	Convolution Block
Average Pooling	(None, 3, 3, 32)	Identity Block
Flatten	(None, 288)	Average Pooling

Table 3. Manual Cloud Mask Description of Baetens-Hagolle Sentinel-2 dataset.

Label	Class Name	Label	Class Name
0	No Fill	4	Cloud Shadow
1	No Data	5	Ground
2	Low Cloud	6	Water
3	High Cloud	7	Snow

Table 4. Spectral Band Information of Sentinel 2.

S. No.	Band Name	Band Index	Resolution (meter)	Wavelength (µ-meter)
1	Coastal	Band 1	60	0.433–0.453
2	Blue	Band 2	10	0.458–0.523
3	Green	Band 3	10	0.543–0.578
4	Red	Band 4	10	0.650–0.680
5	Red Edge 1	Band 5	20	0.698–0.713
6	Red Edge 2	Band 6	20	0.733–0.748
7	Red Edge 3	Band 7	20	0.765–0.785
8	Wide NIR	Band 8	10	0.785–0.900
9	Narrow NIR	Band 8A	20	0.855–0.875
10	Water Vapor	Band 9	60	0.930–0.950
11	Cirrus	Band 10	60	1.365–1.385
12	SWIR1	Band 11	20	1.565–1.655
13	SWIR2	Band 12	20	2.100–2.280

Here, NIR is Near Infrared, and SWIR is Shortwave Infrared

Figure 3. Process of cloud detection (mask).

Table 6. Results of XGBoost, RF and SVM classifiers using Baetens-Hagolle dataset. The best input feature combination is highlighted for all three classifiers.

S. No	Features	Accuracy (%)			Kappa
		XGBoost	RF	SVM	XGBoost	RF	SVM
1	S	85.8	86.1	78.5	0.665	0.678	0.437
2	B	87.5	88.6	79.5	0.710	0.741	0.469
3	G	80.3	80.1	79.7	0.451	0.446	0.434
4	M	89.0	90.8	82.6	0.746	0.807	0.561
5	S + B	88.1	89.9	79.6	0.723	0.770	0.470
6	S + G	87.7	87.7	83.3	0.711	0.714	0.573
7	S + M	89.7	94.1	82.9	0.766	0.862	0.569
8	B + G	88.9	89.6	83.9	0.744	0.763	0.593
9	B + M	90.3	93.1	83.2	0.779	0.860	0.577
10	G + M	89.4	92.6	84.1	0.757	0.832	0.598
11	S + B+G	89.1	90.1	84.0	0.749	0.772	0.596
12	S + B+M	90.3	94.2	83.2	0.780	0.870	0.578
13	S + G+M	90.0	92.3	84.4	0.771	0.825	0.610
14	B + G+M	90.4	93.4	84.4	0.783	0.850	0.614
15	S + B+G + M	90.5	92.8	84.9	0.784	0.836	0.624

Table 7. Numerical results of considered cloud detection methods for Baetens-Hagolle dataset.

Metrics		XGBoost		RF		SVM
		S + B+G + M	Deep	S + B+M	Deep	S + B+G + M	Deep
UA (%)	Low Cloud	77.7	80.2	86.8	82.4	60.3	75.4
	High Cloud	57.7	65.5	76.6	66.8	37.3	56.9
	Cloud Shadow	63.9	63.6	72.5	61.7	9.8	53.1
	Ground	98.3	98	98.9	98.6	99.3	98.4
	Water	88.2	89.1	90.8	87.9	74.1	84.2
	Snow	72.1	64.4	74.2	56.3	28	54.7
PA (%)	Low Cloud	77.1	80.6	87.6	82.6	70.1	77.4
	High Cloud	77.3	78.5	89.9	83.4	68.7	74.7
	Cloud Shadow	77.1	75	85.4	81.3	64.3	72.2
	Ground	94.6	95.2	96.2	95	88.4	93.8
	Water	92.9	91.5	95.6	94.2	76	90.8
	Snow	85.6	82.8	91.5	91.5	82.7	84.4
Accuracy (%)		90.5	91.3	94.2	92.1	84.9	89.7
Kappa		0.784	0.80	0.87	0.82	0.62	0.76
Prediction Time (sec)		23.28	34.24	176.58	1540.98	2148.96	9129.65
Size of Classifier (kb)		4,871	5,007	13,966,954	12,372,374	10	36

Table 8. Conversion of multiple classes to binary classes for cloud detection.

Binary Class	Machine Learning Based		Fmask		Sen2Cor
	Label	Class	Label	Class	Label	Class
Contaminated	2 3 4	Low cloud High cloud Cloud shadow	2 4	Cloud shadow Cloud	3 8 9 10	Cloud shadows Cloud medium probability Cloud high probability thin cirrus
Clear	5 6 7	Ground Water Snow	0 1 3	Clear land Water Snow	2 4 5 6 7 11	Dark area pixels Vegetation Bare soils Water Unclassified snow

Table 9. Numerical results of considered cloud detection methods for Baetens-Hagolle dataset.

Classifier		Metrics	Railroad Valley (USA)	Alta Floresta (Brazil)	Alta Floresta (Brazil)	Marrakech (Morocco)	Arles (France)	Orleans (France)	Ispra (Italy)	Gobabeb (Namibia)	Mongu (Zambia)	Pretoria (South Africa)	Munich (Germany)	Munich (Germany)	Average
XGBoost	S + B+G + M	Accuracy	91.4	96.8	90	83.8	88.9	93.3	90.9	86.2	94.3	92	97.1	88.2	91.1
		F1 Score	92.5	49.2	70.6	73.9	92.7	74.4	41.6	79.5	71.8	78.2	61.2	91.5	73.1
	Deep	Accuracy	92	96.7	88.7	83.1	90	90.2	92	83	92.4	88.8	96.5	88.1	90.1
		F1 Score	93.1	50.5	68.7	73.5	93.5	67.6	53.4	75.6	66.6	73.6	55.2	91.4	71.9
RF	S + B+M	Accuracy	91.6	96.7	89.9	84.2	88.7	92.8	91.1	85.5	94.3	91.3	96.9	88.5	90.9
		F1 Score	92.8	47	71.3	75	92.5	73.3	46.7	78.8	71.8	76.6	58.4	91.7	73
	Deep	Accuracy	92.4	96.8	89.6	82.6	90.4	91.9	91.3	83.4	93	90.6	97.3	88.1	90.6
		F1 Score	93.4	48.1	70.2	71.4	93.7	71.8	48.9	76	67.2	75.3	61	91.3	72.3
SVM	S + B+G + M	Accuracy	90.2	95.4	88.2	71.1	91.9	81.4	89.2	80.5	92.1	89.7	96.6	63.4	85.8
		F1 Score	91.6	28.8	63.3	40.5	94.6	53.4	13.2	73.4	60.3	72.9	48.9	67.8	59.1
	Deep	Accuracy	91.7	97	89.1	82.6	90.4	92.5	91.3	83.8	94.4	90.8	97	87.2	90.7
		F1 Score	92.7	49.5	67.1	70.6	93.7	73	44.1	76.8	71.4	74.6	60.1	90.6	72
Fmask [54]		Accuracy	89.5	97.4	78.8	89.1	95.1	94.4	91.7	91.9	76.3	97.1	98	88.8	90.7
		F1 Score	91.2	63.3	61.4	70.2	80.2	96.3	86.1	47.3	70.2	88.2	69.5	92.3	76.3
Sen2cor [54]		Accuracy	80.1	95.9	89.2	87.9	91.8	78.5	85.9	93	89.6	97	98.2	85.1	89.4
		F1 Score	82	15.2	83.3	51.8	61.8	84.1	75.1	56.9	89.2	85.7	67	89	70.1
MAJA [54]		Accuracy	89.9	97.1	90	93.7	95.2	93.8	89.3	94.6	85.7	96.4	97.8	91.6	92.9
		F1 Score	91.6	60	84.9	81.7	80.2	95.9	82.4	71	84	82.8	63.2	94.4	81

Table 10. Numerical results of considered cloud detection methods for WHUS2-CD dataset.

Metrics	XGBoost		RF		SVM		Resnet	CDFM3SF-4 (Li et al. 2021)	Fmask	Sen2Cor
	Texture	Deep	Texture	Deep	Texture	Deep
UA%	74.16	84.29	76.16	83.18	56.07	83.97	84.69	82.50	59.90	64.20
PA%	78.03	88.27	80.07	89.78	79.40	87.91	87.22	97.91	95.14	69.36
Accuracy (%)	96.41	98.11	97.16	98.23	95.23	98.13	98.00	98.68	92.62	90.89

Table 5. Description of Baetens-Hagolle Sentinel-2 dataset.

	Scene ID	Scene information
Phase 1	S2A_MSIL1C_20170501T183311_N0205_R127_T11SPC_20170501T183311	Small cumulus over bright soil
	S2B_MSIL1C_20170827T182909_N0205_R027_T11SPC_20170827T183954	Large cumulus over bright soil
	S2A_MSIL1C_20180505T141051_N0206_R110_T21LWK_20180505T160911	Scattered small cumulus
	S2B_MSIL1C_20180609T141049_N0206_R110_T21LWK_20180609T173247	Thin cirrus
	S2A_MSIL1C_20160417T110652_N0201_R137_T29RPQ_20160417T111159	Scattered cumulus and thin cirrus
	S2A_MSIL1C_20170621T110651_N0205_R137_T29RPQ_20170621T111222	Clear image with snow and thin cirrus
	S2A_MSIL1C_20170917T103021_N0205_R108_T31TFJ_20170917T103018	Large cloud cover
	S2B_MSIL1C_20171002T103009_N0205_R108_T31TFJ_20171002T103209	Thick and thin clouds
	S2A_MSIL1C_20170516T105031_N0205_R051_T31UDP_20170516T105322	Thick and thin cirrus clouds
	S2B_MSIL1C_20170819T105029_N0205_R051_T31UDP_20170819T105403	Large mid-altitude cloud cover
	S2A_MSIL1C_20170815T102021_N0205_R065_T32TMR_20170815T102513	Clouds over mountains with snow
	S2B_MSIL1C_20171009T102009_N0205_R065_T32TMR_20171009T102008	Clouds over mountains with snow and bright soil
	S2A_MSIL1C_20161221T085352_N0204_R107_T33KWP_20161221T091140	Thick clouds above the desert
	S2B_MSIL1C_20170909T084009_N0205_R064_T33KWP_20170909T090455	Small and low clouds
	S2A_MSIL1C_20161112T082202_N0204_R121_T34LGJ_20161112T083553	Large thick cloud cover and some cirrus
	S2B_MSIL1C_20170804T081559_N0205_R121_T34LGJ_20170804T083419	Clear image and a few mid-altitude clouds
	S2A_MSIL1C_20170313T074931_N0204_R135_T35JPM_20170313T081339	Diverse cloud types
	S2A_MSIL1C_20170820T074941_N0205_R135_T35JPM_20170820T081632	Scattered small clouds
Phase 2	S2B_MSIL1C_20180213T183449_N0206_R027_T11SPC_20180213T214845	Large stratus and some cumulus
	S2A_MSIL1C_20180714T141051_N0206_R110_T21LWK_20180714T155753	Mid-altitude small clouds
	S2A_MSIL1C_20180813T141051_N0206_R110_T21LWK_20180813T174352	Thin cirrus
	S2A_MSIL1C_20171218T111451_N0206_R137_T29RPQ_20171218T132926	Scattered cumulus and snow
	S2B_MSIL1C_20171221T103429_N0206_R108_T31TFJ_20171221T123944	Mid-altitude thick clouds and snow
	S2B_MSIL1C_20180218T110109_N0206_R094_T31UDP_20180218T162525	Stratus cloud
	S2B_MSIL1C_20171111T103239_N0206_R108_T32TMR_20171111T124420	Clouds over mountains and mist
	S2B_MSIL1C_20180209T085109_N0206_R107_T33KWP_20180209T123639	High and thin clouds
	S2B_MSIL1C_20171013T081959_N0205_R121_T34LGJ_20171013T083556	Large thin cirrus cover
	S2B_MSIL1C_20171213T075319_N0206_R135_T35JPM_20171213T112619	Altostratus and small scattered clouds
	S2A_MSIL1C_20180422T102031_N0206_R065_T32UPU_20180422T141352	Mostly cloud-free with a few small clouds
	S2B_MSIL1C_20180424T101029_N0206_R022_T32UPU_20180424T124227	Large cloud cover with cumulus and cirrus

Figure 4. Color code for cloud mask.

Figure 5. Railroad Valley (USA) (a) True Color image, (b) Manual reference mask, generated cloud mask by: (c) RF with traditional texture features (d) RF with deep features (e) XGBoost with traditional texture features (f) XGBoost with deep features, (g) SVM with traditional texture features, and (h) SVM with deep features.

Figure 6. Alta Floresta (Brazil) (a) True Color image, (b) Manual reference mask, generated cloud mask by: (c) RF with traditional texture features (d) RF with deep features (e) XGBoost with traditional texture features (f) XGBoost with deep features, (g) SVM with traditional texture features, and (h) SVM with deep features.

Figure 7. Alta Floresta (Brazil) (a) True Color image, (b) Manual reference mask, generated cloud mask by: (c) RF with traditional texture features (d) RF with deep features (e) XGBoost with traditional texture features (f) XGBoost with deep features, (g) SVM with traditional texture features, and (h) SVM with deep features.

Figure 8. Marrakech (Morocco) (a) True Color image, (b) Manual reference mask, generated cloud mask by: (c) RF with traditional texture features (d) RF with deep features (e) XGBoost with traditional texture features (f) XGBoost with deep features, (g) SVM with traditional texture features, and (h) SVM with deep features.

Figure 9. Arles (France) (a) True Color image, (b) Manual reference mask, generated cloud mask by: (c) RF with traditional texture features (d) RF with deep features (e) XGBoost with traditional texture features (f) XGBoost with deep features, (g) SVM with traditional texture features, and (h) SVM with deep features.

Figure 10. Orleans (France) (a) True Color image, (b)Manual reference mask, generated cloud mask by: (c) RF with traditional texture features (d) RF with deep features (e) XGBoost with traditional texture features (f) XGBoost with deep features, (g) SVM with traditional texture features, and (h) SVM with deep features.

Figure 11. Ispra (Italy) (a) True Color image, (b) Manual reference mask, generated cloud mask by: (c) RF with traditional texture features (d) RF with deep features (e) XGBoost with traditional texture features (f) XGBoost with deep features, (g) SVM with traditional texture features, and (h) SVM with deep features.

Figure 12. Gobabeb (Namibia) (a) True Color image, (b) Manual reference mask, generated cloud mask by: (c) RF with traditional texture features (d) RF with deep features (e) XGBoost with traditional texture features (f) XGBoost with deep features, (g) SVM with traditional texture features, and (h) SVM with deep features.

Figure 13. Mongu (Zambia) (a) True Color image, (b) Manual reference mask, generated cloud mask by: (c) RF with traditional texture features (d) RF with deep features (e) XGBoost with traditional texture features (f) XGBoost with deep features, (g) SVM with traditional texture features, and (h) SVM with deep features.

Figure 14. Pretoria (South Africa) (a) True Color image, (b) Manual reference mask, generated cloud mask by: (c) RF with traditional texture features (d) RF with deep features (e) XGBoost with traditional texture features (f) XGBoost with deep features, (g) SVM with traditional texture features, and (h) SVM with deep features.

Figure 15. Munich (Germany) (a) True Color image, (b) Manual reference mask, generated cloud mask by: (c) RF with traditional texture features (d) RF with deep features (e) XGBoost with traditional texture features (f) XGBoost with deep features, (g) SVM with traditional texture features, and (h) SVM with deep features.

Figure 16. Munich (Germany) (a) True Color image, (b) Manual reference mask, generated cloud mask by: (c) RF with traditional texture features (d) RF with deep features (e) XGBoost with traditional texture features (f) XGBoost with deep features, (g) SVM with traditional texture features, and (h) SVM with deep features.

Table 11. Numerical results of considered cloud detection methods for WHUS2-CD dataset.

Methods	Features	Metrics	Zhanjiang	Changzhou and Wuxi	Linshui	Yilan	Baotou	Altay	Tibet	Tibet	Average
XGboost	Texture	Accuracy	99.61	98.74	99.63	92.48	98.63	94.25	94.64	93.31	96.41
		F1 score	74.81	72.13	84.27	91	83.04	51.6	62.33	78.58	74.72
	Deep	Accuracy	99.76	99.04	99.78	95.10	99.09	97.27	98.62	96.26	98.11
		F1 score	84.73	77.71	89.86	94.41	89.79	71.70	88.9	88.9	85.75
RF	Texture	Accuracy	99.63	98.68	99.63	93.65	98.59	93.79	99.59	93.68	97.16
		F1 score	76.1	72.03	76.10	92.58	83.07	49.25	82.6	82.04	76.72
	Deep	Accuracy	99.75	99.06	99.76	95.32	99.08	97.27	98.78	96.78	98.23
		F1 score	84.16	77.27	89.4	94.61	89.49	71.61	90.02	90.15	85.84
SVM	Texture	Accuracy	99.46	98.61	99.32	89.52	98.28	93.69	91.44	91.51	95.23
		F1 score	55.65	57.57	62.76	86.82	77.6	35.28	48.25	74.44	62.30
	Deep	Accuracy	99.74	99.01	99.75	95.16	99.11	97.09	98.54	96.6	98.13
		F1 score	83.94	77.38	89.05	94.41	89.99	70.5	88.22	89.71	85.40
Resnet		Accuracy	99.77	99.10	99.77	95.08	99.12	96.89	98.37	95.9	98.00
		F1 score	85.31	79.13	89.75	94.31	90.11	69.11	87.21	87.94	85.36
CD-FM3SF-4 (Li et al. 2021)		Accuracy	99.76	99.38	99.74	95.87	99.29	99.05	98.92	97.38	98.67
		F1 score	84.71	85.01	88.70	95.23	91.82	87.41	90.78	91.68	89.41

Figure 17. Zhanjiang (China) (a) True Color image, (b) Manual reference mask, generated cloud mask by: (c) RF with traditional texture features (d) RF with deep features (e) XGBoost with traditional texture features (f) XGBoost with deep features, (g) SVM with traditional texture features, and (h) SVM with deep features, (i) Resnet, and (j) CD-FM3SF-4.

Figure 18. Changzhou and Wuxi (China) (a) True Color image, (b) Manual reference mask, generated cloud mask by: (c) RF with traditional texture features (d) RF with deep features (e) XGBoost with traditional texture features (f) XGBoost with deep features, (g) SVM with traditional texture features, and (h) SVM with deep features, (i) Resnet, and (j) CD-FM3SF-4.

Figure 19. Linshui (China) (a) True Color image, (b) Manual reference mask, generated cloud mask by: (c) RF with traditional texture features (d) RF with deep features (e) XGBoost with traditional texture features (f) XGBoost with deep features, (g) SVM with traditional texture features, and (h) SVM with deep features, (i) Resnet, and (j) CD-FM3SF-4.

Figure 20. Yilan (China) (a) True Color image, (b) Manual reference mask, generated cloud mask by: (c) RF with traditional texture features (d) RF with deep features (e) XGBoost with traditional texture features (f) XGBoost with deep features, (g) SVM with traditional texture features, and (h) SVM with deep features, (i) Resnet, and (j) CD-FM3SF-4.

Figure 21. Baotou (China) (a) True Color image, (b) Manual reference mask, generated cloud mask by: (c) RF with traditional texture features (d) RF with deep features (e) XGBoost with traditional texture features (f) XGBoost with deep features, (g) SVM with traditional texture features, and (h) SVM with deep features, (i) Resnet, and (j) CD-FM3SF-4.

Figure 22. Altay (China) (a) True Color image, (b) Manual reference mask, generated cloud mask by: (c) RF with traditional texture features (d) RF with deep features (e) XGBoost with traditional texture features (f) XGBoost with deep features, (g) SVM with traditional texture features, and (h) SVM with deep features, (i) Resnet, and (j) CD-FM3SF-4.

Figure 23. Tibet (a) True Color image, (b) Manual reference mask, generated cloud mask by: (c) RF with traditional texture features (d) RF with deep features (e) XGBoost with traditional texture features (f) XGBoost with deep features, (g) SVM with traditional texture features, and (h) SVM with deep features, (i) Resnet, and (j) CD-FM3SF-4.

Figure 24. Tibet (a) True Color image, (b) Manual reference mask, generated cloud mask by: (c) RF with traditional texture features (d) RF with deep features (e) XGBoost with traditional texture features (f) XGBoost with deep features, (g) SVM with traditional texture features, and (h) SVM with deep features, (i) Resnet, and (j) CD-FM3SF-4.

Li J, Wu Z, Hu Z, Jian C, Luo S, Mou L, Zhu XX, Molinier M. 2021. A lightweight deep learning-based cloud detection method for Sentinel-2A imagery fusing multiscale spectral and spatial features. IEEE Trans Geosci Remote Sensing. 60:1–19.

 Web of Science ®Google Scholar