Evaluating the impact of sampling designs on the performance of machine learning techniques for land use land cover classification using Sentinel-2 data

ABSTRACT

In today’s world, by integrating remote sensing technology and modern state-of-the-art machine learning techniques, obtaining Land Use Land Cover (LULC) maps has become easier in comparison to traditional manual methods. The performance of a Machine Learning classifier is influenced by various factors. The objective of this study is to evaluate the impact of sampling design in rough complex terrain located in the Northern Himalayan region in Uttarakhand state, India, where reference data is often limited due to the geographical characteristics of the study area. Three sampling design strategies have been incorporated in this study, namely, stratified random sampling with a proportional number of samples (SRS)_proportional, stratified random sampling with an equal number of samples (SRS)_equivalent and stratified systematic sampling with an equal number of samples with a minimum distance of 10 m between the consecutive samples (SSS)_{D = 10 m} for the LULC classification. In this study, Sentinel-2 data of 10 m spatial resolution for the study area of Dehradun district, Uttarakhand, India, has been selected. The following conclusions can be drawn from the results of this study (i) (SRS)_proportional achieved the highest Overall Accuracy (OA) among all the three sampling techniques. The OA and kappa score (k_a) using (SRS)_proportional are OA = 90.25 and k_a = 0.874 by Random Forest, OA = 88.84 and k_a = 0.856 by Support Vector Machine and k Nearest Neighbours (kNN) obtained OA = 87.72 and k_a = 0.842, respectively. (ii) It was found that in the case of (SRS)_proportional, the majority classes like the deciduous forest, evergreen forest and cropland achieved higher recall and precision values in comparison to those obtained from the other two sampling strategies, i.e. (SRS)_equivalent and (SSS)_{D = 10 m}. (iii) The results showed that while switching from (SRS)_proportional to (SRS)_equivalent or from (SRS)_proportional to (SSS)_{D = 10 m}, there was a slight reduction in the precision and recall values for the majority classes and a slight increase for a few of the minority classes.

KEYWORDS:

• Remote sensing
• land use land cover
• stratified random sampling
• machine learning
• support vector machine
• random forest
• k nearest neighbours

Acknowledgements

We would like to thank the USGS Earth Explorer for free access to the Sentinel-2 data.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Code, data and materials availability

The data utilized in this study were obtained from the USGS Earth Explorer Website. Data are available from the authors upon request.