Urban land-use classification using machine learning classifiers: comparative evaluation and post-classification multi-feature fusion approach

Figures & data

Figure 1. Location of the test study area: greater gaborone planning area (GGPA) (reprinted with permission from Y. Ouma et al. (2022). Copyright 2022 ISPRS archives).

Table 1. Characteristics of Landsat MSS, TM, ETM+ and Landsat OLI sensors.

Landsat sensors	Spectral wavelength (μm)	Spectral bands	Spatial resolution (m)	Temporal period (5-year)
Landsat MSS	0.50–0.60 0.60–0.70 0.70–0.80 0.80–1.11	Green Red Near IR1 Near IR2	60	1984
Landsat TM	0.45–0.52 0.53–0.60 0.63–0.69 0.75–0.90 1.55–1.75 2.09–2.35	Blue Green Red Near IR1 SWIR1 SWIR2	30	1990, 1995, 2000, 2005
Landsat ETM+	0.45–0.52 0.52–0.60 0.63–0.69 0.77–0.90 1.55–1.75 2.09–2.35	Blue Green Red Near IR1 SWIR1 SWIR2	30	2010
Landsat OLI	0.45–0.51 0.53–0.59 0.64–0.67 0.85–0.88 1.57–1.65 2.11–2.29	Blue Green Red Near IR2 SWIR1 SWIR2	30	2015, 2020

Figure 2. False color composite LULC classes and the spectral reflectances in L8-OLI (part of figure reprinted with permission from Y. Ouma et al. (2022). Copyright 2022 ISPRS archives).

Figure 3. Visualizing gradient tree decision boosting (reprinted with permission from Y. Ouma et al. (2022). Copyright 2022 ISPRS archives).

Figure 4. Illustration of the random forest classification structure.

Figure 5. Maximum margin-minimum norm classifier in support vector machine with optimal hyperplane for linearly non-separable classes (reprinted with permission from Y. Ouma et al. (2022). Copyright 2022 ISPRS archives).

Figure 6. Topology of a three-layer MLP-ANN.

Figure 7. The feature in-feature out (FEI-FEO) fusion for post-classification fusion.

Figure 8. Variation of number of training samples and classifier overall average accuracy.

Figure 9. Average yearly PA and UA metrics for LULC classes.

Table 2. Classifier-class average accuracy.

Built-up	OA (%)	F1-score	TPR	FPR	AUC
GTB	97.2	0.848	0.861	0.018	0.921
RF	98.6	0.912	0.923	0.008	0.957
MLP-ANN	99.0	0.928	0.931	0.004	0.963
SVM	98.5	0.901	0.919	0.009	0.955
Water
GTB	99.0	0.962	0.960	0.953	0.974
RF	99.6	0.989	0.970	0.997	0.997
MLP-ANN	99.8	0.994	0.995	0.997	0.998
SVM	99.5	0.984	0.987	0.997	0.996
Grass
GTB	97.2	0.633	0.604	0.008	0.798
RF	97.8	0.811	0.826	0.010	0.908
MLP-ANN	97.9	0.772	0.755	0.006	0.874
SVM	97.0	0.720	0.728	0.010	0.859
Shrubs
GTB	90.6	0.817	0.920	0.146	0.887
RF	94.7	0.867	0.939	0.076	0.931
MLP-ANN	92.8	0.844	0.941	0.102	0.920
SVM	93.9	0.857	0.921	0.079	0.921
Forest
GTB	91.7	0.740	0.668	0.020	0.824
RF	95.4	0.872	0.816	0.010	0.903
MLP-ANN	94.2	0.833	0.754	0.012	0.871
SVM	95.0	0.860	0.799	0.013	0.893
Bare-soil
GTB	97.2	0.795	0.813	0.017	0.898
RF	97.9	0.826	0.839	0.012	0.914
MLP-ANN	98.1	0.858	0.881	0.014	0.933
SVM	97.0	0.791	0.837	0.022	0.907

Figure 10. Average ROC curves for the classifiers.

Figure 11. Classifier-class-year F1-scores.

Figure 12. Average overall accuracy performance of the classifiers and the FEI-FEO fusion.

Table 3. Average kappa coefficients for the classifiers per year.

Year	Kappa index
	GTB	RF	SVM	MLP-ANN
1984	0.761	0.864	0.862	0.891
1990	0.696	0.799	0.801	0.806
1995	0.767	0.921	0.898	0.872
2000	0.868	0.936	0.923	0.922
2005	0.768	0.900	0.855	0.846
2010	0.893	0.875	0.905	0.930
2015	0.717	0.845	0.751	0.788
2020	0.884	0.907	0.890	0.856

Table 4. Average z-scores and p-values for ML model pairs.

Classifier pairs	z-score	p-value	Significance level
GTB-RF	−1.687	0.092	No
GTB-SVM	−1.308	0.191	No
GTB-ANN	−1.476	0.140	No
RF-SVM	0.362	0.717	No
RF-ANN	0.150	0.881	No
SVM-ANN	−0.194	0.846	No

Figure 13. LULC year-class areas and change detection using GTB, RF, SVM and MLP-ANN.

Figure 14. Multiclass FEI-FEO post classification fusion.

Figure 15. Image-based ground-truth comparison of classification results for different LULC classes and at different locations within the study area for 2020 (reprinted with permission from Y. Ouma et al. (2022). Copyright 2022 ISPRS archives).

Table 5. Summary of best class classifier per year.

LULC class	Best machine learning classifier and ML-fusion accuracy (%)
	1984	1990	1995	2000	2005	2010	2015	2020
Built-up	MLP-ANN	GTB	MLP-ANN	MLP-ANN	MLP-ANN	MLP-ANN	MLP-ANN	RF
Water	MLP-ANN	MLP-ANN	SVM	SVM	MLP-ANN	MLP-ANN	RF	RF
Grass	MLP-ANN	SVM	RF	MLP-ANN	MLP-ANN	RF	MLP-ANN	RF
Shrubs	RF	MLP-ANN	RF	RF	RF	MLP-ANN	RF	RF
Forest	RF	RF	RF	RF	RF	GTB	RF	RF
Bare soil	MLP-ANN	MLP-ANN	RF	SVM	GTB	SVM	RF	RF
ML-Fusion Accuracy	93.7%	93.4%	96.5%	97.1%	95.5%	96.2%	90.1%	93.4%

Ouma, Y., Nkwae, B., Moalafhi, D., Odirile, P., Parida, B., Anderson, G., & Qi, J. (2022). Comparison of machine learning classifiers for multitemporal and multisensor mapping of urban LULC features. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLIII-B3-2022, 681–689. https://doi.org/10.5194/isprs-archives-XLIII-B3-2022-681-2022

 Google Scholar

Data availability statement

The data used in this study was obtained from the United States Geological Survey (USGS): https://earthexplorer.usgs.gov/. The rest of the data are as presented in this paper. The image data classification was carried out within the Google Earth Engine (GEE).