A comparison of the integrated fuzzy object-based deep learning approach and three machine learning techniques for land use/cover change monitoring and environmental impacts assessment

ABSTRACT

Recent improvements in the spatial, temporal, and spectral resolution of satellite images necessitate (semi-)automated classification and information extraction approaches. Therefore, we developed an integrated fuzzy object-based image analysis and deep learning (FOBIA-DL) approach for monitoring the land use/cover (LULC) and respective changes and compared it to three machine learning (ML) algorithms, namely the support vector machine (SVM), random forest (RF), and classification and regression tree (CART). We investigated LULC impacts on drought by analyzing Landsat satellite images from 1990 to 2020 for the Urmia Lake area in northern Iran. In the FOBIA-DL approach, following the initial segmentation steps, object features were identified for each LULC class. We then derived their respective attributes using fuzzy membership functions and deep convolutional neural networks (DCNNs), a deep learning method. The Fuzzy Synthetic Evaluation and Dempster-Shafer Theory (FSE-DST) also applied to validate and carryout the spatial uncertainties. Our results indicate that the FOBIA-DL, with an accuracy of 90.1% to 96.4% and a spatial certainty of 0.93 to 0.97, outperformed the other approaches, closely followed by the SVM. Our results also showed that the integration of Fuzzy-OBIA and DCNNs could improve the strength and robustness of the OBIA’s decision rules, while the FSE-DST approach notably improved the spatial accuracy of the object-based classification maps. While object-based image analysis (OBIA) is already considered a paradigm shift in GIScience, the integration of OBIA with fuzzy and deep learning creates more flexibility and robust OBIA decision rules for image analysis and classification. This research integrated popular data-driven approaches and developed a novel methodology for image classification and spatial accuracy assessment. From the environmental perspective, the results of this research support lake restoration initiatives by decision-makers and authorities in applications such as drought mitigation, land use management and precision agriculture programs.

KEYWORDS:

• Integrated approach
• fuzzy object-base
• deep learning CNN
• machine learning algorithms
• spatial uncertainty
• land use/cover
• change detection
• urmia lake

1. Introduction

Remote sensing methods have become indispensable for the environmental evaluation and monitoring of larger areas. The range of satellite sensors and the volume of remote sensing data has increased, as has the user base and the variety of methods and methodologies to process large amounts of data. The number of satellites and particular initiatives, such as the European Copernicus program with its Sentinel satellites, enable the acquisition of higher temporal resolution data (Cresson 2018). Some scientists have even identified revolutionary effects resulting from the transfer of technological developments from other areas to the remote sensing domain (Sudmanns et al. 2020). A brief review of research in this field reveals a wide range of existing satellite image analyzes and respective demands for applications. Particularly the more recent publications emphasize the need for cost-effective and fast data-driven methods (Chen et al. 2014; Zou et al. 2015; Chen, Zhao, and Jia 2015; Romero, Gatta, and Camps-Valls 2016; Cheng, Zhou, and Han 2016; Marmanis et al. 2016; Yu, Wu, and Luo 2017; Kussul et al. 2017; SharSharma et al. 2017; Vetrivel et al. 2018; Talukdar et al. 2020; Xia et al. 2020; Ghasemi, Karimzadeh, and Feizizadeh 2021). There is a growing demand for and interest in accurate and semi-automated data-driven approaches in the domain of remote sensing. The fields of remote sensing and earth observation science strive to propose, develop, and optimize new techniques and methods for data acquisition, information extraction and classification. The range of methods and techniques widens significantly by drawing from other fields, such as computer science or computer vision. More and more machine learning methods contribute to the field of remote sensing.

Machine learning (ML) methods are increasingly used for remote sensing image processing tasks (Cresson 2018; Abdi Geshlagi, and Feizizadeh 2021; Feizizadeh et al. 2021). According to Ma et al. (2019) and Talukdar et al. (2020), since around 2015, machine learning techniques and, in particular, DL algorithms have received significant attention in the remote sensing community. Technically speaking, the DL method represent a high potential for exploring the nonlinear patterns in dada. This technique has also the capability to be trained various image context features efficiently which leads to coincides it as it as effective data driven approach. Artificial neural networks are based on biological neural networks in several ways, with CNNs (a form of artificial neural network) being influenced by animal visual cortex (Chartrand et al. 2017; Yamashita et al. 2018). These methods have been efficiently used in many applications, such as land use and land cover (LULC) classification, change detection, agricultural crop monitoring, landslide detection and environmental degradation. In the remote sensing domain, ML techniques are widely used for LULC classifications. In brief, monitoring and prediction of LULC dynamics are used for evaluating and understanding landscape transformations (Mas et al. 2017), providing critical information for a wide range of activities and applications (e.g. land use planning, food security assessment, environmental degradation, global warming, natural hazards mitigation, climate change studies etc) (Hoan et al. 2018; Dutta et al. 2019). We, therefore, need efficient, fast, semi/automated and cost-effective data-driven approaches for monitoring and mapping LULC (Naboureh et al. 2017).

Object-Based Image Analysis (OBIA) has been considered an alternative approach to the traditional pixel-based image processing methods that constitute the range of remote sensing approaches. OBIA (also called Geographic Object-Based Image Analysis, or GEOBIA), which has been called a paradigm shift in remote sensing (Blaschke 2010), incorporates segmentation concepts in an essential initial step. It integrates spatial and spectral information and concepts through grouping and deriving object-based features, paving the way for spatial modeling (Blaschke 2010; Blaschke, Kelly, and Merschdorf 2015). Initially, OBIA methods aimed to precisely map the Earth’s multi-scale features (e.g. building, tree and the field) using high spatial resolution imagery through evolving image objects and combined radiometric and spatial analyses. The spatial dimensions of objects such as distances, neighborhoods, patterns, topologies, and geometric characteristics can then be used in a classification. More recently, semantics and ontologies have gained more prominence in the field of OBIA, partially triggered by interoperability requirements for the use of auxiliary data from geographic information systems (GIS). The OBIA methodology has also been characterized by its central role of almost natural language equivalent rule sets that also play an important role in fusing remote sensing and spatial assessment techniques (Blaschke, Kelly, and Merschdorf 2015). Early research indicated that OBIA, while being capable of achieving high precision, also requires maximum human supervision and that the best arrangement for an image usually cannot be extrapolated to another distinct image (Guirado et al. 2017).

OBIA is widely known as one of the most efficient methodologies that combines supervised learning, expert knowledge, and data-driven approaches (for an overview, see Kucharczyk et al. 2020). The efficiency of OBIA is based on the integration of spatial and spectral properties together with spatial modeling capabilities. However, in some applications, such as LULC mapping, uncertainty and ambiguous tends to be inevitable due to the large variety and similarity of the spatial and spectral characteristics when classifying LULC maps (Feizizadeh and Kienberger 2017). The vagueness, uncertainty and ambiguity of the input data are essentially intertwined with the problem and precision of the inferences, which depend on the quality of LULC maps (Maboudi et al. 2018). To overcome this issue, the integration of OBIA with fuzzy decision rules could provide an efficient image classification approach that has also been addressed by earlier research (e.g. Hofmann, Blaschke, and Strobl 2011; Aksoy and Ercanoglu 2012; Najafi et al. 2018; Feizizadeh 2018; Maboudi et al. 2018; Moradpour et al. 2020; Feizizadeh et al. 2020; Najafi, Feizizadeh, and Navid 2021). Fuzzy logic is an effective approach that interprets vague, ambiguous, and uncertain information (e.g. image objects with similar object values for describing the LULC classes) when implementing OBIA rule sets for the task of LULC classification. Fuzzy membership values of the image objects are heuristic information that enables us to classify objects efficiently (Maboudi et al. 2018; Feizizadeh et al. 2021).

The combination of fuzzy membership values and OBIA features (in short, FOBIA) facilitates the robust classification decision rules of OBIA. However, the FOBIA approach is still semi-automated, and the recent progress in deep learning methods enables efficient and automated feature analysis approaches. Deep learning and convolutional neural networks (CNN) in particular, allow us to capture the feature information using the network system and to improve the accuracy of the results. The CNN technique is based on the direct transfer of the data to the network while aiming to achieve an efficient performance by means of eliminating the large and challenging feature engineering stage of the whole process (Xia et al. 2020).

In terms of LULC mapping, the semantic segmentation of digital satellite images permits satellite image pixels to be classified into several LULC classes based on the similarity of their spectral properties. Thus, deep convolutional neural networks (DCNNs) methods have recently been implemented to solve some of the remaining issues that have been reported (Schmidhuber 2015; LeCun, Bengio, and Hinton 2015). DL-CNN have been successfully applied in computer vision tasks such as image processing (Alhassan et al. 2019). This algorithm’s capability to detect “low-level and high-level” features within the hierarchical approach makes this technique an efficient method for dealing with complex computer vision issues such as semantic segmentation of images, which is critical for LULC mapping (Henry et al. 2019; Alhassan et al. 2019). In this regard, central to the emerging paradigm shift toward data-intensive science, machine learning approaches are becoming increasingly significant in remote sensing. Particularly deep learning has proven to be both a major success and an exceptionally powerful tool in various domains (Zhu et al. 2017; Kazemi Garajeh et al. 2021). Likewise, since around 2017, deep learning algorithms and OBIA have been used in at least a loose combination, particularly for identifying objects in computer vision, with promising results (Guirado et al. 2017). However, only more recently have scientists coupled DCNN models and OBIA directly. Ghorbanzadeh et al. (2020) used DCNN models as input data for an OBIA knowledge-based semantic classification method. Unlike standard pixel-based classification methods that are usually applied for the DCNN model, their integrated approach aggregates DCNNs results on separately delineated objects as the basic units of a rule-based classification.

Considering the recent advances in remote sensing and Earth observation technologies, efficient and cost-effective data-driven machine-based approaches need to be proposed and applied. Thus, to analyze the capability of integrated fuzzy object-based image analysis with deep learning (FOBIA-DL) and machine learning algorithms for LULC monitoring, this research compares the efficiency of an integrated FOBIA-DL against the three well-known machine learning algorithms of support vector machine (SVM), random forest (RF), and classification and regression tree (CART). The LULC changes and associated impact on the severe and recurring droughts in the Urmia Lake region in northern Iran were also evaluated from the environmental perspective. The lake drought is a critical environmental problem in Iran because it causes extensive soil salinity and reduces the productivity of agricultural lands, giving rise to salt-dust storms and threatening food security and the lives and livelihoods of the 7.5 million inhabitants residing in the Urmia Lake Basin.

2. Study area

The study area is the Urmia Lake Basin (ULB) located in northwest Iran. Situated at an elevation of 1286 m above sea level, Urmia Lake is the 25^th largest lake and the second-largest hypersaline lake in the world (Figure 1), with an area of originally 5200 km². Urmia Lake is a shallow saline lake surrounded by the East and West Azerbaijan Provinces. According to the latest national census, about 7 million (10% of Iran’s total population) are residing in the two provinces adjacent to the lake. ULB is renowned for horticulture, agriculture, animal husbandry, manufacturing of handicrafts, mining, apiculture, industry, and business, for tourism and recreational activities, and regionally it is important for Artemia harvesting and salt extraction (Ghorbanzadeh et al. 2019a). The climate in the ULB is extreme and continental, primarily influenced by the mountains around the lake. Significant seasonal fluctuations in the air temperature occur in this semi-arid area. The mean annual precipitation is 341 mm. The region’s lowest and highest temperature varies from 0 to −23°C in winter and can reach up to 40°C during the summer. From this perspective, Urmia Lake is a vital resource for the region, as it helps to moderate extreme temperatures and makes the climate conducive to the growth of fungi and fauna. On the other hand, sea breeze circulation, penetrating inland and local heights, has a pronounced impact on the local climate and creates local clouds at low levels (Mardi et al. 2018; Kazemi Garajeh et al. 2021). Due to extensive land use/cover change, extending agricultural activities and population growth in the ULB, Urmia Lake is now facing several environmental problems. Historical data shows that the lake has been drying up since 2000, which has already caused several environmental problems. Extensive soil salinization, dust storms and health issues are some of the observed environmental threats in the ULB.

Figure 1. Location of the study area

3. Dataset

As indicated before, the main objective of this research is to apply and compare the efficiency of FOBIA-DL against the SVM, RF, and CART ML algorithms for a trend analysis of LULC changes and spatial modeling of the environmental issues associated with LULC changes. Figure 2 depicts the main aspects of the research methodology. To achieve the research objectives, we analyzed a time series of Landsat satellite images together with ground control points and a GIS dataset comprised of a digital elevation model (DEM) and its derivates (slope, aspect). The DEM data was developed from topography maps at a scale of 1/25,000 and a spatial resolution of 10 meters.

To capture and monitor the extensive horticultural and agricultural activities, we obtained satellite images for July of 1990, 1995, 2000, 2005, 2010, 2015 and 2020 (Table 1) . The month of July was selected because this is the time of the year when the pasture lands experience growth, which makes it easier to distinguish them from agricultural crops in spectral imagery. For each study year, we obtained 3 satellite images (from between the 5^th and the 30^th of July), and the most suitable images were selected based on the cloud coverage and image enmeshment properties. To use the satellite images as a time series, we applied pre-processing steps (e.g. layers stack, mosaic, atmospheric and radiometric correction). The LULC classes to be monitored from 1990 to 2020 were defined based on the characteristics of the study area, as bare lands, cultivated lands, dry and fallow lands, orchards, pasture and rangelands, rock outcrops, salty lands and settlement areas, water body and wet lands.

Table 1. List of satellite images and their details

Date	Satellite/sensor	Raw/Path		Date	Satellite/sensor	Raw/Path
1990/08/21	Landsat 05/TM		167/034	2005/08/05	Landsat 05/TM	168/035
1990/08/21			167/035	2005/07/27		169/033
1990/08/12			168/033	2005/07/27		169/034
1990/08/12			168/034	2010/08/12		167/034
1990/08/12			168/035	2010/08/28		167/035
1990/08/19			169/033	2010/08/19		168/033
1990/08/19			169/034	2010/08/19		168/034
1995/08/03	Landsat 05/TM		167/034	2010/08/19	Landsat08/OLI	168/035
1995/08/19			167/035	2010/08/10		169/033
1995/08/10			168/033	2010/08/10		169/034
1995/08/26			168/034	2015/08/10		167/034
1995/08/26			168/035	2015/08/10		167/035
1995/08/17			169/033	2015/08/17		168/033
1995/08/17			169/034	2015/08/17		168/034
2000/08/16	Landsat 05/TM		167/034	2015/08/17	Landsat08/OLI	168/035
2000/08/16			167/035	2015/08/08		169/033
2000/08/23			168/033	2015/08/08		169/034
2000/07/22			168/034	2020/08/05		167/034
2000/08/23			168/035	2020/08/05		167/035
2000/08/30			169/033	2020/08/12		168/033
2000/08/30			169/034	2020/08/12		168/034
2005/07/19	Landsat 05/TM		167/034	2020/08/28		168/035
2005/07/19			167/035	2020/08/19		169/033
2005/08/05			168/033	2020/08/19		169/034
2005/08/05			168/034

To train and analyze the accuracy of the intended data-driven approaches, a total of 35,000 sample points were taken into account for LULC mapping in the considered study period from 1990–2020. 70% of the dataset (24,500 points) was used as training data and 30% (9500 points) as ground control points for the accuracy assessment. This step was carried out separately for each study year (1990, 2000, 2005, 2010, 2015 and 2020) with 5000 training data points for each year. We obtained the training data and ground control points for the year 2020 during a field visit based on the random sampling, whereas for 2015, 2010, 2005, and 2000 we obtained historical GPS data from the Organization of Agriculture Jahad (OAJ) of West and East Azerbaijan provinces. For this goal, we employed analog aerial photography in the scale of 1/10,000-1/20,000 and the agricultural LULC and cadaster maps (with 1/2000 scale) and historical Google Earth images to collect the training and ground control points.

Figure 2. Methodological research scheme

4. Object-based image analysis

4.1. Image segmentation

Image segmentation is typically the initial step of OBIA workflows in which pixels are grouped into image objects. The generated objects contain information about their shape, geometric-, textural – and statistical properties, and about the relationships between the objects and the thematic layers (Baatz et al. 2004). The choice of segmentation parameters will directly affect the size and type of the image objects (Aksoy and Ercanoglu 2012). The multi-resolution segmentation algorithm is considered an efficient segmentation algorithm in OBIA and is very widely used, particularly through its integration in the prominent commercial software eCognition. This multi-resolution segmentation is a bottom-up region growing process that merges similar pixels and homogenous regions into objects. An important aspect is the so-called scale parameter, which specifies a heterogeneity threshold for the spectral information but also incorporates shape and compactness parameters that can be defined by the user (Eisank, Smith, and Hillier 2014). Determining the most efficient scale, shape, and compactness parameters has always been a serious challenge in the segmentation process for OBIA (Feizizadeh et al. 2021). The scale, shape and compactness are critical parameters that ultimately determine the size of an image object. How these parameters are defined depends on the purpose of the study, the data quality, and the characteristics of the case study area. To obtain efficient parameters, OBIA employs several user-defined segmentation scales for image analysis and classification. Such user-defined parameters might have a certain inherent uncertainty. To overcome this uncertainty, Drǎguţ, Tiede, and Levick (2010) proposed estimating the scale parameter (ESP) as a tool to compute segmentation scales and parameters statistically. The ESP tool allows for a fast estimation of scale parameters for a multi-resolution segmentation. The ESP aims to generate segments based on the user-defined values (e.g. shape, compactness, and scale parameter) and the computed local variance (LV) as the mean standard deviation of the objects (Drǎguţ, Tiede, and Levick 2010). Therefore, the graphics of LV can be applied to compute the suitable scale parameters in relation to the data properties of the scene. The rate of change (ROC) can be obtained from equation 1(1) $ROC=[\frac{L-\left(L-1\right)}{L-1}]\ast 100$(1) to assess the dynamics of LV from one object level to another:

(1) $ROC=[\frac{L-\left(L-1\right)}{L-1}]\ast 100$(1)

Where L = LV at the target level and L – 1 = LV at next lower level.

The ROC is a technical indicator employed in stock market analyses to compute ‘the value of stock’s price’. We assume that the peak point of the rate of change of the local variance graph shows the surface of the image objects that can be segmented in the most appropriate manner (Lees 2006). In this study, the ESP tool was applied to obtain the optimal scale parameter of the segmentation process. To obtain the final scale parameter, several scale parameters were examined and considered before selecting the most appropriate scale based on both spatial features and derivative values. The segmentation was repeated for three scales (15, 20 and 25) while using the same shape factor (0.6) and compactness factor (0.4).

4.2. Object-based features applied for LULC mapping

The object-based features were used to derive the LULC class information from the satellite images to develop the LULC maps. We applied spectral, spatial, geometric and contextual object features to detect the target LULC classes. We also used field operation data to examine the efficiency of each algorithm for deriving the LULC classes. The field operation data were collected by GPS to be used as training data for all classes. We used the existing LULC map of the study area along with historical aerial photography and time series cadaster maps of the agricultural lands to obtain historical training data. Accordingly, for each period of the study year, 3000 training sites (in sum 21,000 points) were used as sample data to obtain the thresholds of the object features and to train the deep learning algorithms. The object features also need to be identified and applied through a semi-automated rule set. Thus, it is critical to identify the context and efficiency of object-based features to map each LULC class. Therefore, we took the training data into account, along with the efficiency of related spatial and spectral object features for each LULC class obtained through fuzzy-based threshold values. Based on the spectral and spatial characteristics of each LULC class, the respective object features were identified, and their efficiency was evaluated by correlating the training data with the membership value of each object feature. Table 2 represents some of the selected object features and their respective computed values. We used 50 samples for each LULC class to obtain the threshold values. The samples were collocated in the field using GPS. Then, the object feature values for all samples of each LULC class were computed. The majority of feature values were computed, and the threshold values were determined as shown in Table 2. The equation and their components for the selected object features are also represented in Table 3. In addition to the object features, we also applied a number of spectral indices, namely the Green Normalized Difference Vegetation Index (GNDVI), Specific Leaf Area Vegetation Index (SLAVI), Modification of Normalized Difference Water Index (MNDWI), Normalized Built-up Index (NDBI), Relief and TRI indices (Table 3).

Table 2. LULC classes and their respective thresholds obtained for object features

LULC class	Threshold
Water body	−0.38<GNDVI<0.20133 –0.608<NDVI<-0.1443 MNDWI ≥ 0.172009276
Wet lands	MNDWI ≥0.141 NDVI<0.1 NDWI≥0.1 SWCI (IR) ≥ 0.4
Settlement	SBI≥0.32 –0.033<NDBI<0.26 0.068> < GLCM Correlation < 0.761 285.65> GLCM CONTRAST < 791.43 198<Brightness<318 0.125<Asymmetry<0.876 0.328< Rectangular Fit<1 169.45<Mean Red<327.32 0<NDBI<0.19
salty lands	NDSI≥0.257 110.58<Mean Green<346.75 1.1011<SI<2.3 Salinity Index (S5) ≥ 252
Bare Lands	50.806<Mean Blue<233.19 115.258 <GLCM Mean<147.4 81.321< TM4 over TM5 < 141.06 –0.05< Mean NDVI<0.035 Brightness≥105
Orchards	0.3<Mean NDVI<0.563 GNDVI≥0.3 0<Mean Slope<6 55.16 <SLAVI<148.7
Cultivated lands	0.0593 < GLCM Correlation <1 1.4652 <GLCM Entropy<8.654 GNDVI≥ 0.15 Mean Slope < 5 23.64<SLAVI<86.48 26.546<Mean NIR<189.65 1200<Mean DEM<1800 -0.123<Mean NDVI<0.365
Rock output	TRI≥5 0≤Relief≤110 NDVI<0.05
Dry and fallow lands	0<Mean NDVI<0.1 0≤SWCI (IR)≤0.4 1.02≤Shape Index≤1.36 Rectangular fit≥0.84 2.5531 <GLCM Contrast<9.642 SCI≤0.04
Pasture lands	Mean Slope ≥ 5 Mean NDVI≥0.18
Rangelands	Mean Slope ≥ 5 Mean NDVI≥0.08

Table 3. The implemented object features and their correlation between properties and quantifiable attributes

	Quantifiable attribute	Mathematical formulation
Spectral attributes	Brightness	$\mathrm{B}=\frac{1}{{\mathrm{n}}_{\mathrm{v}\mathrm{i}\mathrm{s}}}\hspace{0.17em}\sum _{\mathrm{i}=1}^{{\mathrm{n}}_{\mathrm{v}\mathrm{i}\mathrm{s}}}{\bar{\mathrm{c}}}_{\mathrm{i}\left(\mathrm{v}\mathrm{i}\mathrm{s}\right)}$B is the mean brightness of an object and${\bar{\mathrm{C}}}_{\mathrm{i}\left(\mathrm{v}\mathrm{i}\mathrm{s}\right)}$ is the sum of all the mean brightnesses in the visible bands divided by the corresponding number of bands ${\mathrm{n}}_{\mathrm{v}\mathrm{i}\mathrm{s}}$
	Normalized Difference Vegetation Index (NDVI) Specific leaf area vegetation index (SLAVI)	Tv = mean NDVI f (Object) = $\left\{\begin{array}{c}LCiff\left(\mathrm{O}\mathrm{b}\mathrm{j}\mathrm{e}\mathrm{c}\mathrm{t}\right)\le {\mathrm{T}}_{\mathrm{v}})\\ VAiff\left(Object\right)>{\mathrm{T}}_{\mathrm{v}})\end{array}\right.$ ${\mathrm{T}}_{\mathrm{v}}^{\prime }=\frac{\mathrm{m}\mathrm{e}\mathrm{a}\mathrm{n}\mathrm{N}\mathrm{D}\mathrm{V}{\mathrm{I}}_{\mathrm{L}\mathrm{C}}+\mathrm{m}\mathrm{e}\mathrm{a}\mathrm{n}\mathrm{N}\mathrm{D}\mathrm{V}{\mathrm{I}}_{\mathrm{V}\mathrm{A}}}{2}$ $T_v^{\prime }$ is an average of the mean NDVI values for “landslide candidates” (LC) and vegetated areas (VA). The NDVI, which has a value between −1.0 and +1.0 SLAVI = 100*[Mean band 4]/([Mean band 3]+[Mean band 5])
	Green Normalized Difference Vegetation Index (GNDVI)	GNDVI = 100*(1+(([Mean band 4]-[Mean band 2])/([Mean band 4]+[Mean Layer 2])))
	Modified Normalized Differenced Water Index (MNDWI)	MNDWI = 100*(1+(([Mean Band4]-[Mean Band5])/([Mean Band4] +[Mean Band5])))
	Normalized Built-up Index (NDBI)	NDBI = ([Mean band 5]-[Mean band 4])/([Mean band 5]+[Mean band 4])
	Soil Water Content Index (InfraRed Index) Soil Color Index	SWCI (IR) = (NIR – ETM7)/ (NIR+ETM7) SCI = R – G/R +G
	Normalized Built-up Index (NDBI)	NDBI = ([Mean band 5]-[Mean band 4])/([Mean band 5]+[Mean band 4])
	Salinity Index (SI) Normalized Difference Salinity Index	Salinity Index (SI) = ([Mean Layer 1]*[Mean Layer 3])^0.5 NDSI = ([Mean band 3]-[Mean band 4])/([Mean band 3]+[Mean band 4])
	mean band	${\bar{C}}_k\left(V\right)={\bar{C}}_k\left(P_v\right)=\frac{1}{\mathrm{\#}{P}_v}\sum _{\left(x,y,z,t\right)\in P_v}{\bar{C}}_k\left(x,y,z,t\right)$[$C_k^{min}$, $C_k^{max}$]
	Standard Deviation	${\sigma }_k\left(v\right)={\sigma }_k\left(P_v\right)$$\sqrt{\frac{1}{\mathrm{\#}{P}_v}\left(\underset{\left(x,y,z,t\right)\epsilon P_v}{\mathrm{\Sigma }}{C}_k^2\left(x,y,z,t\right)-\frac{1}{\mathrm{\#}{P}_v}(\underset{\left(x,y,z,t\right)\epsilon P_v}{\mathrm{\Sigma }}{C}_k^2{\left(x,y,z,t\right)}^2\right)}$[0,$\frac{1}{2}{C}_k^{range}$] • ${\sigma }_k\left(v\right)$ is the standard deviation of intensity values of image layer k of all pixel/voxels forming an image object v • Pv is the set of pixel/voxels of an image object v • #Pv is the total number of pixel/voxels contained in Pv • (x,y, z, t) are the pixel/voxel co-ordinates • ck(x,y, z, t) is the image layer intensity value at pixel/voxel (x,y, z, t) • $C_k^{range}$ is the data range of image layer k with $C_k^{range}=C_k^{max}$]-$C_k^{min}$
Geometric attributes	Shape length/width	$\frac{\mathrm{L}\mathrm{e}\mathrm{n}\mathrm{g}\mathrm{t}\mathrm{h}}{\mathrm{W}\mathrm{i}\mathrm{d}\mathrm{t}\mathrm{h}}$Length of the image object Width of the image object
	Shape asymmetry Shape Rectangular	$1-\sqrt{\frac{{\lambda }_{\mathrm{m}\mathrm{i}\mathrm{n}}}{{\lambda }_{\mathrm{m}\mathrm{a}\mathrm{x}}}}$${\lambda }_{\mathrm{m}\mathrm{i}\mathrm{n}}$ is the minimal eigenvalue ${\lambda }_{\mathrm{m}\mathrm{a}\mathrm{x}}$ is the maximal eigenvalue$\frac{\left\{\mathrm{\#}\left(x,y\right)\epsilon P_v:\rho v\left(x,y\rho \right)\le 1\right\}}{\mathrm{\#}{P}_v}$$P_v=$(x,y) is the elliptic distance at a pixel (x,y) [0,1]; where 1 is a perfect rectangle.
	Shape Index	$\frac{{\mathrm{B}}_{\mathrm{v}}}{4{\sqrt{{\mathrm{\#}}_{\mathrm{V}}}}_{\mathrm{v}}}$${\mathrm{B}}_{\mathrm{v}}$ is the image object border length $4{\sqrt{{\mathrm{\#}}_{\mathrm{p}}}}_{\mathrm{v}}$ is the border of square with area of $\mathrm{\#}{P}_v$
Textural attributes Texture	Contrast (Con)	$\sum _{\mathrm{i},\mathrm{j}=0}^{\mathrm{N}-1}{\mathrm{P}}_{\mathrm{i},\mathrm{j}}{\left(\mathrm{i}-\mathrm{j}\right)}^2\hspace{0.17em}$i is the row number j is the column number N is the number of rows or columns Pi,j is the probability value from the GLCM
		$\sum _{i,j=0}^{N-1}{P}_{i,j}\left[\frac{\left(i-{\mu }_i\right)\left(j-{\mu }_i\right)}{\sqrt{\left({\sigma }_i^2\right)\left({\sigma }_j^2\right)}}\right]$${\sigma }_i^{}$is the GLCM standard deviation
	Correlation (Cor)	μi is the GLCM mean
	Entropy (Ent)	$\sum _{\mathrm{i},\mathrm{j}=0}^{\mathrm{N}-1}{\mathrm{P}}_{\mathrm{i},\mathrm{j}}\left(-1\mathrm{n}{\mathrm{P}}_{\mathrm{i},\mathrm{j}}\right)$
	Standard deviation (StdDev)	${\sigma }_{\mathrm{i}}^2=\sum _{\mathrm{i},\mathrm{j}=0}^{\mathrm{N}-1}{\mathrm{P}}_{\mathrm{i},\mathrm{j}}{\left(\mathrm{i}-{\mu }_{\mathrm{i}}\right)}^2{\sigma }_{\mathrm{j}}^2={\left(\mathrm{j}-{\mu }_{\mathrm{i}}\right)}^2\hspace{0.17em}$
	Mean	${\mu }_{\mathrm{i}}=\sum _{\mathrm{i},\mathrm{j}=0}^{\mathrm{N}-1}\mathrm{i}\left(\mathrm{P}\mathrm{i},\mathrm{j}\right)\hspace{0.17em}{,}^{}{\mu }_{\mathrm{j}}=\sum _{\mathrm{i},\mathrm{j}=0}^{\mathrm{N}-1}\mathrm{j}{\left(\mathrm{P}\mathrm{i},\mathrm{j}\right)}^{}$Relief = ([Max. pixel value DEM]-[Min. pixel value DEM])

4.3. Fuzzy-OBIA based deep learning implementation

Deep learning (DL), as discussed in the introduction section, is a paradigm shift in remote sensing. Ma et al. (2019), in their comprehensive meta-analysis and review of deep learning applications in remote sensing, pointed out that since 2015 the remote sensing community has shifted its attention to DL algorithms. DL techniques have been successfully applied in several application domains, such as LULC classification (Liu et al. 2018; Tsagkatakis et al. 2019; Ma et al. 2019), change detection and time series analysis (Liu, Pan, and Dezert 2016; Das and Ghosh 2016; Cai et al. 2018), landslide mapping (Feizizadeh et al. 2017; Ghorbanzadeh et al. 2019b) and the retrieval of precipitation data.

In the context of remote sensing image processing, we consider an integration of OBIA and DL algorithms as a promising solution for LULC classification problems. Such an integrated approach will allow us to employ OBIA’s class modeling capabilities, together with the power of DL. Several studies (Zhao et al. 2017; Zhu et al. 2017; Tong et al. 2018; Huang, Zhao, and Song 2018; Ma et al. 2019) confirmed that CNNs are effective DL algorithms that can, in principle, be integrated with OBIA. According to Zhu et al. (2017), CNNs can be used to extract mid – and high-level abstract features from raw object images based on convolutional and assembling layers (i.e. feature maps produced by object features in OBIA, such as mean, standard deviation, shape index, etc.). Earlier studies that integrated CNNs and OBIA for image classification have already confirmed that the particular combination of object feature representations trained by CNNs is an efficient approach for large-scale semantic image segmentation, recognition, classification and change detection analysis (Zhu et al. 2017).

The CNNs can be used for a variety of applications, particularly in the domain of remote sensing time series analysis (e.g. change detection mapping); the time index t can be used as an integer value. The convolution can be determined as follows:

(2) $s\left(t\right)=\left(x\ast w\right)\left(t\right)=\odot h\left(x\right)=\sum _{-\mathrm{\infty }}^{\mathrm{\infty }}x\left(a\right)w\left(t-a\right)$(2)

Equation 2(2) $s\left(t\right)=\left(x\ast w\right)\left(t\right)=\odot h\left(x\right)=\sum _{-\mathrm{\infty }}^{\mathrm{\infty }}x\left(a\right)w\left(t-a\right)$(2) can be applied for time series image analysis in image processing.

In this research, we considered the DCNN framework proposed by Namatēvs (2017), whereby 5 layers in 5 steps were adapted based on the approach implementation setting in Python environment to carry out the Fuzzy-OBIA-DL-CNN. The 5 steps are as follow:

A) Input layer: every DCNN input layer is a multidimensional array of data. In the context of image processing, the input layer can be considered as image pixels or their transformation, such as object features values. In this research, we employed the outcome of Fuzzy-OBIA as input layers for our DCNN implementation. Therefore, after identifying the relevant object features for the LULC classes, the fuzzy membership values for whole image objects were computed and exported to Python as input for the DCNN.

B) Convolutional layers or convolutional stage: this layer is the main building block of DCNNs that extracts distinct features from the input layer by employing the training data. In our case, we employed the computed FOBIA values to compute the object feature spaces for each of the considered LULC classes. In this step, we also introduced 5000 training data for each study year.

C) Non-linear layer or detection stage: this is a critical stage that detects each linear activation through the non-linear activation function by introducing the non-linearity into neural networks and training models that are more complex (e.g. FOBIA-based LULC classification).

D) The pooling (also known as down sampling or subsampling) layer aims to reduce the size of the feature maps (e.g. object based features) through compressing features and the computational complexity of the network. Technically, this step produces down-sampled versions of the input map in order to reduce the dimensionality of the object-based feature maps (e.g. the relevant features for the LULC classification were identified and are listed in Table 2). In the case of our DCNN implementation, this step leads to a reduced dimensionality of the initial input layers of the fuzzy-based object feature maps for classifying themes into LULC classes.

E) Fully connected layer: this is the last step of a CNN implementation when developing a generic multi-layer network. In this step, the outcomes of all four previous layers are connected to all activations (the LULC classes in our case) to extract features to train another classifier using the training data. To implement the FOBIA-DL, this step employs the computed fuzzy membership values for LULC classes and performs the classification using the DCNN. Thus, in this step, we reduce the dimensionality of the fuzzy-based feature maps (the fourth layer is the output of step D) being classified to obtain the LULC map based on the obtained training dataset.

4.4. Implementation of machine learning algorithms

To compare the efficiency of Fuzzy-OBIA-DL with machine learning algorithms, we also tested the following three machine learning (ML) algorithms: support vector machine (SVM), random forest (RF), and classification and regression tree (CART). Due to the popularity of the ML algorithms for satellite image classification, several ML algorithms have been proposed and applied in earlier research (Talukdar et al. 2020; Ebrahimy et al. 2021; Ghorbanzadeh et al. 2020; Omarzadeh et al. 2021; Valizadeh Kamran et al. 2021). The ML algorithms allow analyzing complex data spaces (e.g. satellite images), which makes them effective data-driven approaches. In addition, the ML techniques make use of ensembles of classification techniques, which leads to further flexibility and efficiency for the classification of remote sensing imagery (Doyle et al, 2021). We compared the efficiency of three ML algorithms against the integrated FOBIA-DL. To apply the ML algorithms, we used the satellite images (from the years 1990, 1995, 2000, 2005, 2010, 2015, and 2020) and training data to generate the LULC maps based on each ML algorithm. We used Python to classify and create the LULC maps based on 35,000 training data – and control points for the FOBIA-DL (5000 training data – and control points for each study year), and the outcome was 7 LULC maps (1 for each study year) for each of the ML algorithms.

5. Accuracy assessment

The accuracy assessment is a critical step in image classification, which is not complete without the accuracy assessment and validation of results (Lillesand and Kiefer 2001; Blaschke et al., 2014). In the context of the accuracy assessment, the error matrix, which is assigned based on ground control data, is the most common and widely applied method for the validation of classified maps in remote sensing (Zhou and Troy 2008). This approach is based on the computation of several measures, including the “overall accuracy,” the producer’s accuracy’ as well as the “user’s accuracy” (Janssen and Vanderwel 1994; Comber et al. 2012; Kassouk et al. 2014). The error matrix is a well-known and frequently applied accuracy assessment in the field of remote sensing. Despite its popularity, the error matrix has been criticized for its inherent uncertainty (Aronoff 1985; Foody 2002, 2006; Feizizadeh et al. 2017). Feizizadeh (2018) pointed out that a confusion matrix might even face more uncertainty when it is used for accuracy assessment of the object-based classification maps. In this context, it must be emphasized that OBIA makes use of fuzzy decision rules and membership functions to classify objects. It can, therefore, be considered as a “soft classifier” technique. Thus, due to the segmentation process and the scale regulation, as well as the fuzzy decision rules, the results of an object-based classification are challenging to validate using a binary mode and be assigned to “true” and “false” categories (Albrecht, Lang, and Hölbling 2010; Feizizadeh 2018). Feizizadeh (2018) proposed a novel approach based on Fuzzy Dempster-Shafer Theory for spatial uncertainty analysis and accuracy assessment of object-based image classification to overcome these issues. This approach uses Fuzzy Synthetic Evaluation (FSE) as the fuzzy decision rule and Dempster-Shafer Theory (DST) for validation and spatial uncertainty assessment. In this approach, the FSE technique calculates the matches and non-matches between a classified map and reference data (e.g. ground truth data).

5.1. Fuzzy synthetic evaluation

The FSE approach makes use of two groups of data (ground truth data and the respective rate computed for each data point to calibrating the overall and per-class accuracy of classified maps) in the two-step process. The first step employs the Difference fuzzy function to calculate the classification confidence and the observed level of error for each class. The second step weights the Difference function categories to compute the accuracy value. The confidence of the classified maps can then be obtained based on the results from these steps and the observed ratio of match between the control data and reference data. To compute the ratio of match between the control data and reference data, we employed the interpretation of confidence ratings (ICR), which was initially proposed by Sarmento, Carrao, and Caetano (2008) and adapted for OBIA by Feizizadeh (2018). Table 4 represents the Difference function and its respective default ICR values for each category. Based on this approach, the overall accuracy and the confidence of each class of object-based classified image can be computed by applying the weighted sum of the sample observation ratios in each class derived using ICR values (Feizizadeh 2018). For further details regarding the FSE and its implementation, we refer to Feizizadeh et al. (2017) and Feizizadeh (2018). To apply the FSE approach in this research, we employed 1500 ground control points for each study year. These validation data were used together with the ICR values to compute the final accuracy values, which are shown in Table 5. The 1500 ground control points constitute 30% of the 5000 training data points, as discussed in the dataset section.

Table 4. Difference function and its respective default values for each category (Feizizadeh 2018)

Accuracy	Level of Confidence	Class map		ICR %
		Demand class	Alternative class
Confidence in classification	Very High Confidence in Classification (VHCC)	*		≥ 90
	High Confidence in Classification (HCC)	*		≥ 85
	Acceptable Confidence in Classification (ACC)		*	≥ 80
	Reduced Confidence in Classification (RCC)	*		80 ≤
	Very Reduced Confidence in Classification (VRCC)		*	50
Magnitude errors	Acceptable Error (AE)			50 ≤
	High Error (HE)			≥ 85
	Very High Error (VHE)			≥ 90

Table 5. Results of the accuracy assessment and ICR values for the FOBIA-DL based classified LULC maps

ICRs	Bare lands							Cultivated area							Dry and fallow							Orchards
Year	1990	1995	2000	2005	2010	2015	2020	1990	1995	2000	2005	2010	2015	2020	1990	1995	2000	2005	2010	2015	2020	1990	1995	2000	2005	2010	2015	2020
VHCC	0.91	0.89	0.91	0.96	0.90	0.91	0.92	0.81	0.94	0.87	0.95	0.91	0.88	0.90	0.92	0.94	0.93	0.89	0.90	0.89	0.91	0.93	0.96	0.91	0.94	0.93	0.95	0.93
HCC	0.23	0.34	0.36	0.22	0.13	0.14	0.12	0.14	0.25	0.23	0.15	0.23	0.25	0.14	0.17	0.14	0.12	0.15	0.14	0.12	0.15	0.16	0.14	0.16	0.14	0.13	0.15	0.16
ACC	0.09	0.11	0.08	0.92	0.10	0.11	0.07	0.13	0.11	0.09	0.05	0.06	0.16	0.12	0.13	0.07	0.03	0.92	0.19	0.12	0.06	0.03	0.14	0.06	0.17	013	0.07	0.06
RCC	0.09	0.15	0.17	0.18	0.10	0.09	0.08	0.19	0.16	0.13	0.05	0.09	0.18	0.14	0.12	0.06	0.07	0.11	0.10	0.13	0.09	0.07	0.14	0.11	0.13	0.17	0.15	0.17
VRCC	0.02	0.03	0.06	0.07	0.03	0.08	0.02	0.09	0.08	0.06	0.05	0.04	0.06	0.05	0.07	0.04	0.03	0.02	0.04	0.09	0.02	0.03	0.08	0.06	0.06	0.05	0.05	0.03
AE	0.03	0.02	0.01	0.02	0.03	0.01	00.2	0.03	0.01	0.02	0.03	0.02	0.02	0.02	0.03	0.02	0.01	0.02	0.03	0.02	0.01	0.01	0.02	0.03	0.01	0.02	0.02	0.01
HE	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
CHE	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
FSE %	91.2	90.1	91.4	90.8	91.6	93.1	92.0	94.4	92.1	94.1	93.1	93.2	94.1	95.6	94.1	93.3	92.8	91.9	93.2	91.6	95.2	93.9	94.3	94.2	92.1	94.5	95.4	94.9

5.2. Dempster-shafer theory for spatial accuracy assessment

The Dempster-Shafer Theory (DST) is a mathematical function based on the Bayesian probability theory that can be applied for spatial accuracy assessment and spatial uncertainty mapping. The DST makes use of the concepts of three functions, namely the basic belief assignments (BBA), the belief function, and the plausibility function. The capabilities of the DST makes it an efficient technique for modeling imprecision and for spatial uncertainty analysis (Liu, Pan, and Dezert 2015). The applicability of DST for spatial uncertainty mapping has been reported by a number of researchers (Feizizadeh and Blaschke 2014b; Feizizadeh et al. 2017; Shokati and Feizizadeh 2018; Feizizadeh 2018). As early researchers pointed out, the DST technique allows us to determine the spatial uncertainty of the classified maps. Thus, we can conclude that the DST approach offers an extension of the probability framework for computing the uncertainty of classified images in the domain of remote sensing (Liu, Pan, and Dezert 2016). Technically, the DST can compute the epistemic uncertainty that affects the expert knowledge of the probability P($M_l$) within the alternative model $M_l$, $l$ = $1$, …, n. According to this approach, lower and upper bounds shows the inherent uncertainty associated with P($M_l$). In this context, the lower bound determines the belief Bel($M_l$), which represents the degree of belief that supports the $M_l$. The upper band, which is also known as the plausibility, $P_l$($M_l$), allows computing $M_l$, which is supposed to be the correct model “up to that value.” Thus, contrary to the probability processes, the probability mass is used to obtain the individual elementary events. Therefore, the DST technique or theory of evidence is used to carry out the BBA m (A) on sets A (the focal sets) of the power set P(Z) of the event space Z, i.e. on sets of outcomes rather than on single elementary events. In more detail, m(A) indicates the degree of belief that a specific element x belongs to set A only, and not to any subset of A. The bpa satisfies the following requirements (Baraldi and Zio 2010; Liu, Pan, and Dezert 2016):

(3) $m:P{\left(Z\right)}^{}\to \left[0,1\right],m\left(0\right)=0;\sum _{A\epsilon \left(\mathrm{Z}\right)}^m\left(A\right)=1$(3)

To carry out the spatial uncertainty analysis, the belief function computes the lower bound value for a (known) probability. The plausibility function also obtains the upper bound value for an (unknown) probability. The difference between the plausibility (Pl) and the belief (Bel) function can be computed to determine the spatial uncertainty (Feizizadeh 2018). Figure 3 depicts a schematic representation of the DST formwork. In our research, after developing the LULC maps using different data-driven approaches, the DST was applied to determine the spatial uncertainty of the classifications.

Figure 3. DST Framework of the spatial uncertainty modeling

6. Results

In the previous sections, we described how we applied the integrated approach of FOBIA and CNNs and compared the resulting LULC maps against those created with the ML algorithms (SVM, RF and CART). The LULC maps were developed for the years 1990, 1995, 2000, 2005, 2010, 2015, and 2020 (Figures 4–7). We used the FSE-DST approach to validate the results, which are given in Table 5. Since it was not feasible to present the ICR values of all LULC classes in the final 28 maps due to the sheer numbers (8 IRC values for each class in the LULC map), we only presented the computed ICR values for the LULC classes of the 7 maps produced using the FOBIA-DL approach as an example. The IRC values for the LULC classes of the other 21 maps are available in the online version as an appendix. Table 6 shows the obtained overall FSE values from the IRCs given in Table 4 for all classes in the 28 produced maps. As the computed FSEs show, the FOBIA-DL outperformed all other classifications with minimum and maximum achieved accuracies of 90.1% and 96.4%, respectively. The second-best performing classification technique was the SVM, which achieved minimum and maximum accuracies of 88.1% and 91.9%, respectively. The RF method achieved minimum and maximum accuracies of 80.2% and 89.3%, respectively. The CART method was the technique with the lowest classification performance, achieving minimum and maximum accuracies of 74.1% and 86.7%, respectively.

Figure 4. Results of the LULC classification Fuzzy-OBIA-DL, a) 2020, b) 2015, c) 2010, d) 2005, e) 2000, f) 1995 and g) 1990

Figure 5. Results of the LULC classification based on the SVM, a) 2020, b) 2015, c) 2010, d) 2005, e) 2000, f) 1995 and g) 1990

Figure 6. Results of the LULC classification Based on the RF, a) 2020, b) 2015, c) 2010, d) 2005, e) 2000, f) 1995 and g) 1990

Figure 7. Results of the LULC classification based on the CART, a) 2020, b) 2015, c) 2010, d) 2005, e) 2000, f) 1995 and g) 1990

Table 6. Results of the accuracy assessment and ICR values for the FOBIA-DL based classified LULC maps

ICRs	Settlement							Water body							Wet Lands
Year	1990	1995	2000	2005	2010	2015	2020	1990	1995	2000	2005	2010	2015	2020	1990	1995	2000	2005	2010	2015	2020
FOBIA-DL	92.2	90.1	91.4	94.6	92.1	91.4	93.6	92.4	93.1	95.1	96.2	94.2	96.1	95.6	94.1	93.3	92.8	91.9	93.2	91.6	95.2
SVM	90.1	89.2	88.1	90. 2	89.1	86.5	83.4	89.1	91. 3	91. 2	91. 5	91.2	92.0	91.1	90.9	92. 5	91. 1	89.1	91. 3	91.1	90. 3
RF	82. 4	80.2	83. 4	80.9	82.1	87. 8	87.2	88. 2	87.1	88.4	86.7	87. 6	88.1	89. 2	89. 3	87. 7	88. 3	88.1	89. 8	89.7	89.1
CART	76. 8	74.1	75.1	76. 2	77.4	74. 9	75. 2	84.1	81. 2	80.1	81.4	79. 8	80. 8	82.1	83. 2	86. 2	84. 5	85.1	85.1	84. 9	86. 7
	Bare lands							Cultivated area							Dry and fallow
FOBIA-DL	91.2	90.1	91.4	90.8	91.6	93.1	92.0	94.4	92.1	94.1	93.1	93.2	94.1	95.6	94.1	93.3	92.8	91.9	93.2	91.6	95.2
SVM	89.1	88.4	90.3	90.2	89.9	91.4	91.9	91.0	90.4	93.2	91.2	90.9	91.2	94.3	93.2	91.9	90.3	90.2	91.1	91.7	90.8
RF	87.2	86.2	88.1	89.6	88.6	87.3	89.5	88.3	88.0	89.2	87.2	88.6	89.3	88.7	87.1	84.6	82.6	81.8	84.8	82.1	80.1
CART	81.4	80.3	82.5	79.9	81.6	80.2	82.1	83.8	79.2	80.1	85.7	83.9	82.6	80.2	88.9	81.0	79.9	80.4	79.3	79.2	80.1
	Orchards							Pasture lands							Rangelands
FOBIA-DL	93.9	94.3	94.2	92.1	94.5	95.4	94.9	94.1	91.3	92.3	90.8	92.7	92.3	92.0	90.9	93.1	90.1	92.9	91.7	94.5	95.6
SVM	91.1	90.2	91.0	91.8	91.7	90.6	90.7	93.1	90.2	90.2	90.1	91.6	92.0	90.1	89.8	92.1	88.2	90.2	89.3	90.3	91.6
RF	88.1	88.2	87.8	87.9	87.6	88.2	89.4	86.5	88.4	87.8	87.2	89.3	84.3	85.2	88.1	89.1	87.1	86.2	85.9	87.8	89.2
CART	83.1	84.2	80.7	83.2	84.2	82.8	85.1	84.7	84.2	80.0	81.8	83.4	83.2	81.6	82.4	84.7	83.5	84.9	81.3	85.3	81.5
	Rock outcrops							Salty lands
FOBIA-DL	94.1	93.3	91.6	91.9	93.2	90.6	93.2	94.2	95.3	94.2	92.1	94.5	92.3	94.9
SVM	91.3	90.2	89.5	88.3	89.2	87.5	87.2	91.3	91.6	91.4	89.2	91.4	88.4	90.6
RF	88.2	89.1	87.2	86.2	88.4	85.4	87.6	89.3	87.2	89.1	84.1	87.1	84.9	87.4
CART	83.1	82.2	82.8	79.8	84.1	80.2	81.9	84.8	86.8	84.3	83.2	84.2	82.7	83.1

As a second validation approach, we determined the spatial uncertainty based on the DST method. The spatial uncertainties of the LULC maps are illustrated in Figures 8–11. Table 7 also shows the DST results for each data-driven method. The FOBIA-DL method achieved the highest reliability with a minimum and maximum value of 0.93 and 0.97, respectively. The SVM’s reliability was also computed to be in the range of 0.81 to 0.89, which makes it the second most efficient data-driven approach. The RF method achieved reliability values between 0.72 and 0.79, and the CART technique yielded the lowest reliability with values between 0.62 and 0.68. As the results of both validation approaches (FSE and DST) indicated, the FOBIA-DL was more efficient than the machine learning data-driven techniques of SVM, RF and CART.

Figure 8. Fuzzy-OBIA-DL Spatial uncertainty assessment results for LULC maps based on the DST method: a) 2020, b) 2015, c) 2010, d) 2005, e) 2000, f) 1995 and g) 1990

Figure 9. SVM Spatial uncertainty assessment results for LULC maps based on the DST method: a) 2020, b) 2015, c) 2010, d) 2005, e) 2000, f) 1995 and g) 1990

Figure 10. RF Spatial uncertainty assessment results for LULC maps based on the DST method: a) 2020, b) 2015, c) 2010, d) 2005, e) 2000, f) 1995 and g) 1990

Figure 11. CART Spatial uncertainty assessment results for LULC maps based on the DST method: a) 2020, b) 2015, c) 2010, d) 2005, e) 2000, f) 1995 and g) 1990

Table 7. Kappa coefficient for each algorithm in each study year

Method		1990	1995	2000	2005	2010	2015	2020
	FOBIA-DL	0.94	0.93	0.98	0.95	0.96	0.93	0.97
	SVM	0.83	0.80	0.89	0.84	0.82	0.81	0.87
	RF	0.72	0.73	0.75	0.78	0.79	0.77	0.76
	CART	0.62	0.63	0.64	0.65	0.68	0.67	0.66

After the validation of the results, we evaluated the LULC classes to determine the trend of LULC changes and their respective impacts on the Urmia Lake drought. Table 8 shows their respective changes for each study period based on the FOBIA-DL data-driven approach. The LULC analysis results indicate that cultivated areas, orchards, and dry and fallow lands have increased significantly since 1990. Significant changes were also observed in areas of bare land and rangelands. Monitoring LULC also revealed a significant expansion of settlement areas due to the increasing population between 1990 and 2020. On the other hand, our results indicate that wetlands and water bodies faced significant reductions in size, while salt lands increased substantially. The changes in rangelands and pasture land were related to respective changes in the class of rock outcrops.

Table 8. LULC area changes (km²)

LULC	1990–1995	1995–2000	2000–2005	2005–2010	2010–2015	2015–2020
Bare land	−1019.81	+1651.19	−54.39	+104.32	+308.46	+587.80
Cultivated area	+134.92	+206.518	+51.52	+331.09	+290.05	+81.73
Dry and fallow lands	+1000.07	+1126.53	−354.11	+190.46	+524.13	+198.04
Orchards	+201.71	+177.04	+212.22	+143.87	+210.84	+135.47
Pasture lands	+761.74	+349.66	+123.48	+130.77	+177.32	+47.55
Rangelands	−1245.28	+966.93	−472.31	−84.41	−930.02	−789.13
Rock outcrops	−672.10	−143.13	−95.34	−461.51	+891.69	−76.46
Salty lands	−20.23	+236.69	−47.30	+996.72	+924.44	−1314.94
Settlement	+10.39	+62.67	+69.55	+8.92	+99.8435	+10.96
Water body	+895.52	−1320.08	−358.96	−1263.78	−2161.22	+2471.79
Wet lands	−47.059	−11.65	−18.97	−3.750	+19.88	−13.695

According to these results, the surface area of UL was 5114 km² in 1990 and this surface area increased to a maximum of 6005 km² in 1995^. (991 Km² = 19% increase from 1990 to 1995). After this first extension period, the UL surface decreased to 4697 km² in 2000 (1308 km² = 21.78% reduction from 1995 to 2000). This reduction continued, and the surface area of UL decreased substantially to reach 4307 km² in 2005 (390 km² = 8.31% reduction from 2000–2005). Between 2005 and 2010, the UL surface area decreased again to 3026 km² (1011 km² = 25% reduction from 2005–2010). However, the most significant reduction in the surface area of UL occurred from 2010 to 2015 when the area of the lake shrunk to 890 km², and UL lost about 2136 km² of its surface body (70.58%), exposing extensive salt lands. Fortunately, the surface area of UL increased to 3348 km² in 2020 (2458 km ² = 48.06% extension). As we will discuss in the next section, this recent expansion was predominantly caused by raising awareness for reducing water consumption in agricultural activities and as a result of higher precipitation in 2018–2019. However, our investigation showed that despite the recent positive trend in the UL surface area, the lake has still lost 1766 km² (34.54%) of its surface area between 1990 and 2020.

Table

ICRs	Pasture lands							Rangelands							Rock output							Salty lands
Year	1990	1995	2000	2005	2010	2015	2020	1990	1995	2000	2005	2010	2015	2020	1990	1995	2000	2005	2010	2015	2020	1990	1995	2000	2005	2010	2015	2020
VHCC	0.95	0.92	0.93	0.92	0.91	0.93	0.92	0.86	0.93	0.89	0.91	0.89	0.93	0.91	0.94	0.92	0.95	0.94	0.92	0.89	0.92	0.91	0.95	0.94	0.93	0.94	0.95	0.97
HCC	0.02	0.21	0.09	0.13	0.13	0.12	0.12	0.14	0.25	0.23	0.15	0.23	0.25	0.14	0.17	0.14	0.12	0.15	0.14	0.12	0.15	0.16	0.14	0.16	0.14	0.13	0.15	0.16
ACC	0.01	0.14	0.12	0.92	0.10	0.10	0.07	0.13	0.11	0.09	0.05	0.06	0.12	0.11	0.13	0.07	0.03	0.92	0.19	0.12	0.06	0.03	0.14	0.06	0.17	013	0.07	0.06
RCC	0.05	0.14	0.11	0.18	0.10	0.09	0.04	0.14	0.16	0.12	0.05	0.09	0.12	0.14	0.12	0.06	0.06	0.11	0.10	0.13	0.09	0.07	0.14	0.11	0.13	0.17	0.15	0.12
VRCC	0.01	0.02	0.02	0.07	0.03	0.08	0.02	0.09	0.08	0.06	0.05	0.04	0.06	0.05	0.02	0.04	0.03	0.02	0.04	0.09	0.02	0.03	0.08	0.06	0.06	0.05	0.05	0.03
AE	0.02	0.0	0.03	0.02	0.03	0.01	00.2	0.03	0.01	0.02	0.02	0.01	0.03	0.02	0.01	0.02	0.01	0.02	0.03	0.02	0.01	0.01	0.02	0.03	0.01	0.02	0.02	0.01
HE	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
CHE	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
FSE %	94.1	91.3	92.3	90.8	92.7	92.1	92.0	90.9	93.1	90.1	92.9	91.7	94.5	95.6	94.1	93.3	91.6	91.9	93.2	90.6	93.2	94.2	95.3	94.2	92.1	94.5	92.3	94.9

Table

ICRs	Settlement							Water body							Wet Lands
Year	1990	1995	2000	2005	2010	2015	2020	1990	1995	2000	2005	2010	2015	2020	1990	1995	2000	2005	2010	2015	2020
VHCC	0.94	0.92	0.93	0.95	0.92	0.93	0.94	0.92	0.94	0.95	0.95	0.95	0.97	0.96	0.95	0.94	0.93	0.89	0.94	0.96	0.96
HCC	0.13	0.21	0.12	0.05	0.13	0.16	0.10	0.11	0.13	0.02	0.15	0.13	0.02	0.11	0.12	0.15	0.11	0.10	0.13	0.11	0.13
ACC	0.04	0.09	0.04	0.92	0.12	0.08	0.04	0.02	0.10	0.11	0.05	0.06	0.16	0.12	0.13	0.07	0.03	0.92	0.19	0.12	0.06
RCC	0.08	0.12	0.11	0.12	0.10	0.09	0.02	0.11	0.12	0.11	0.02	0.08	0.18	0.14	0.12	0.06	0.06	0.11	0.10	0.09	0.07
VRCC	0.03	0.10	0.02	0.05	0.03	0.08	0.02	0.09	0.08	0.06	0.05	0.04	0.06	0.05	0.07	0.04	0.03	0.02	0.04	0.09	0.02
AE	0.01	0.02	0.03	0.02	0.03	0.01	00.2	0.03	0.01	0.02	0.03	0.02	0.02	0.02	0.03	0.02	0.01	0.02	0.03	0.02	0.01
HE	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
CHE	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
FSE %	92.2	90.1	91.4	94.6	92.1	91.4	93.6	92.4	93.1	95.1	96.2	94.2	96.1	95.6	94.1	93.3	92.8	91.9	93.2	91.6	95.2

7. Discussion

The main objective of this research was to apply different data-driven approaches and compare their efficiency for LULC mapping and trend analysis. According to our results, the integrated FOBIA-DL achieved the most accurate results. OBIA offers various spatial and spectral object features, of which the most appropriate ones need to be selected for analysis. Applying the most appropriate features may significantly impact the accuracy and robustness of the results. Earlier studies showed that any improvement in the accuracy of the object-based classification mainly depends on the selection of appropriate features and parameters for classification (Ma et al. 2017; Maboudi et al. 2018; Kucharczyk et al. 2020; Najafi, Feizizadeh, and Navid 2021). In this study, we employed different spectral, spatial, and geometric object features as the basis of the LULC classification through DCNN. We applied object-based features, such as vegetation and brightness indices, color insensitivities, shape indicators, compactness, asymmetry, homogeneity, contrast information, and texture analysis using the gray-level co-occurrence matrixes (GLCM). We employed training data to compute a fuzzy membership degree of each object features to classify the LULC classes. Our results confirm the results of earlier studies (e.g. Hofmann, Blaschke, and Strobl 2011; Feizizadeh et al. 2020; Najafi, Feizizadeh, and Navid 2021) and highlight the significance of fuzzy rules integrated with OBIA. The obtained results confirm that the integration of Fuzzy-OBIA can deal with uncertain, incomplete and vague information for steering or controlling processes as well as assigning image objects to LULC classes using the membership functions. In this context, we emphasize that one of the most important factors in achieving a high accuracy with this method is analyzing the effective membership degrees in the classification for each parameter and using the parameters with the highest membership degree. The results confirm that the combination of fuzzy methods and OBIA allows computing the efficiency of each object feature for LULC classification.

OBIA has been proven to be a methodology that can improve the accuracy of remote sensing classifications by employing homogeneous objects resulting from image segmentation and applying spatial and spectral object features to classify these objects (Blaschke 2010; Blaschke et al., 2014; Kucharczyk et al. 2020). Such capability reduces classification errors, particularly the errors between individual pixels that belong to a different LULC class but have a spectral similarity. Thus, classifying objects – rather than pixels – has again proven beneficial. Still, in combination with the power of DL algorithms (i.e. CNNs), further improvements can be achieved. Our results proved that an integration of FOBIA and DL-CNN could make use of the huge range of object features in OBIA and the power of machine-based decision rules in DCNN to classify homogenous objects generated by image segmentation. The obtained results confirm that an integrated FOBIA method improves the robustness of OBIA as a decision-making system and also improves the accuracy of the results. The integration of FOBIA with DCNN supports remote sensing users in developing an efficient automated image classification approach that can be used for a wide range of applications such as LULC, change detection analysis, and environmental mapping. Along with recent improvements in remote sensing and the availability of satellite images with improved spatial, spectral, and temporal resolutions, such an automated approach can be efficient and cost-effective.

In the context of integrating FOBIA with DL methods, the results of this research prove that the integrated approach of FOBIA-DCNNs could deliver the highest accuracy for image classification. The CNNs are deep learning models that have been used in remote sensing since 2014 (Ma et al. 2017). Their combination with OBIA and FOBIA created new possibilities and efficient data-driven approaches to satellite image classification (Kucharczyk et al. 2020). Deep learning is a form of machine learning that uses multiple layers of artificial neural networks. Artificial neural networks are based on biological neural networks in several ways, whereby CNNs (a form of artificial neural network) are influenced by the animal visual cortex (Chartrand et al. 2017; Yamashita et al. 2018).

It can automatically extract high-level features from low-level features of the input image. Significant progress is currently being made in remote sensing research devoted to investigating techniques that allow relevant information to be effectively interpreted, analyzed, and extracted (Singh, Verma, and Chaudhari 2016). Therefore, strong, robust, and automated decision rules are being supplied by DCNNs. On the other hand, FOBIA can also be used to efficiently derive the spatial and spectral characteristics of the LULC classes. For classifications, the conventional OBIA makes use of general classification techniques, such as the nearest neighbor classifier. On the other hand, the integrated FOBIA and DCNNs method has the advantage of fuzzy membership values, the spatio-spectrally explicit image analysis provided by the FOBIA, and robust decision rule fulfilled by the DCNNs.

Machine learning algorithms have become very popular since the impressive return of neural networks in 2006 (Hinton, and Salakhutdinov 2006). When comparing their efficiency, our results showed that the SVM achieved better results than the RF and CART. Machine learning methods have emerged over the past few years as one of the most successful techniques for a wide variety of remote sensing applications. In the scientific community, these approaches have gained substantial interest due to their ability to break records, for example, in the fields of speech – and visual recognition (Vieira, Pinaya, and Mechelli 2017). The standard SVM model could be a two-class or multi-class model (an amalgamation of a two-class SVMs chain). The most widely used form of ML approach is the two-class SVM (Tien Bui et al. 2020). The SVM contains several kernels for the task of image registration and classification. However, the polynomial and radial basis function (RFB) kernel has been frequently applied in remote sensing applications (Maxwell, Warner, and Fang 2018). This kernel is also recognized as the most common technique for LULC classification and achieves better accuracy than conventional methods (Talukdar et al. 2020). The RF approach is also one of the most used machine learning techniques (Feizizadeh et al. 2021). This algorithm is popular because it can be used for classification and regression as well as for categorical and continuous variables (Woznicki et al. 2019). Because of this versatility, RF has been used in a broad range of earth science applications, such as LULC mapping (Betts et al. 2017), land use (Araki, Shima, and Yamamoto 2018), and land cover and object-oriented mapping (Nitze, Barrett, and Cawkwell 2015). RF is an ensemble learning algorithm based on the premise that aggregated bootstrap classifiers perform better than a single classifier in a combination (Feizizadeh et al. 2021).

The accuracy assessment has been one of the main challenges in remote sensing for many years (Foody 2002, 2006; Feizizadeh et al. 2017; Feizizadeh 2018). To overcome potential flaws, we proposed and applied a new FSE-DST approach for the accuracy assessment. Our results confirmed the efficiency of this approach for the accuracy assessment of remote sensing-based classified maps. The main advantage of the FSE-DST approach is that it utilizes the fuzzy decision rules while subsequently minimizing the uncertainty of a conventional error matrix. Using fuzzy decision rules and membership values to express the confidence of each LULC class creates a mutual relationship between references data and classified images that can be used to go back and forth between analysis and interpretation. In addition, the DST technique allows depicting the spatial accuracy by representing the certainty of every single unit (e.g. object or pixel) in the classified map. Contrarily, a conventional error matrix provides only single accuracy values per class.

From the environmental perspective, various environmental problems within the study area were already known, but this study provides a systematic investigation of the extent and the severity of some major environmental issues. We investigated the shrinkage of the Urmia Lake (UL) and the related extensive LULC changes. The results reveal an increase in the settlement area of 262 km², which is a very high value. In addition, 933 km² of land were cultivated, 2685 km² were turned into dry farming and fallow field systems and another 1081 km² into orchards. In total, this represents a substantial extension in agricultural activities and residential areas. Due to the impact of the intensive anthropogenic pressure on natural resources, the UL and the surrounding area are now facing a critical situation. Over the course of the 30-year study period (1990–2020), the lake drought resulted in a loss of 34.54% of the lake surface area and exposed a 1766 km² area of salt flats. The lake’s surface area varies naturally to some extent between the wet and dry seasons, depending on seasonal precipitation. However, the major trend and timeline of the lake drought are unquestionably beyond the seasonal changes, and the rapid LULC changes have already approached a “tipping point” that could lead to sudden lake death (Agha Kouchak et al., 2015). It is well understood that the lake drought causes several environmental problems, such as soil and water salinization, that reduce the productivity of agricultural lands and lead to food insecurity for the population of the area. The health of the population is also directly affected by the impacts of dust storms. Thus, by highlighting the environmental impacts of the Urmia Lake drought, government awareness has been raised by social and environmental communities in recent years, and the matter is now receiving significant attention. As the results show, the countermeasures have recently had an effect and, in combination with two years of higher precipitation, reversed the trend of the decreasing lake surface area somewhat.

Awareness and mitigation campaigns mainly focused on water scarcity and drought. The main goal of awareness and mitigation campaigns was to reduce the water consumption in agricultural, residential, and industrial sectors by gaining support for the lake restoration initiative. In 2018, the UL surface increased by about 48%. This is accredited to these awareness and mitigation measures, but also to relatively high rainfall values. According to the 2019 report of the Iranian Meteorological Organization, the average annual precipitation at Urmia Lake Basin (ULB) in 2018 was 328 mm. This is 11.3% more than in 2017 (291 mm). The long-term (30 years) average precipitation in ULB is 238.8 mm and, accordingly, the precipitation in 2018 was about 89.7 mm or 27.35% higher than usual. Despite these relatively humid conditions in 2018, salty land areas still existed in 2020, which potentially lead to dust storms. Due to the geographical characteristics (e.g. topography and semi-arid climate as well as being impacted by the western winds), it is expected that the UL drought will result in dust storms that carry salt over the exposed lakebed and impact the productivity of agricultural lands through soil salinization, thus forcing farmers to give up and move away. Our analysis results are in line with earlier studies (e.g. Delju et al. 2012; Agha Kouchak et al., 2015; Hamidi-Razi et al. 2018; Rahimi Balkanlou et al. 2020; Najafi et al. 2019) that reported similar environmental problems caused by the UL drought.

8. Conclusion and future works

Intensive progress in Earth observation technologies yielded significant improvements in the spatial, spectral, and temporal resolutions of satellite images, which is increasingly taking remote sensing into the arena of big data technology. Applying conventional image processing techniques to analyze and classify enormous data demands fast, effective, semi-automated and cost-effective data-driven methods (see also Sudmanns et al. 2020). In this research, we integrated FOBIA with DCNNs techniques and compared its efficiency to that of other well-known machine learning techniques (SVM, RF and CART). We also compared the accuracies of the different methods using the new spatial accuracy assessment approach of FSE-DST. Our results proved that this integrated approach was robust and effective for image classification. We conclude that an integrated FOBIA – DCNNs methodology allows improving the decision rules of OBIA for classifying the objects into LULC classes. Conventional classification techniques in OBIA (e.g. nearest neighbor, support vector machine, etc.) can be prone to errors or ambiguities when classifying objects with similar attributes. Thus, DCNNs with strong decision rules and the ability to simulate the LULC class based on intensive training data enable a minimization of the ambiguity and uncertainty of object-based classifications even compared to the effective machine learning techniques of SVM, RF and CART.

The proposed FSE-DST approach also supported us in computing the validity of the results and deriving the spatial accuracy more efficiently. We conclude that future research in OBIA applications can benefit from both aspects of our approach, namely a) FOBIA and its integration with DCNNs, and b) accuracy assessment using FSE-DST. The recent progress in remote sensing and the availability of a wide range of satellite images demands novel, cost-effective and efficient data-driven methods for remote sensing analysis.

OBIA turned out to be a paradigm shift in remote sensing (Blaschke, 20,210). Thus, this research is advancing the fields of remote sensing and GIScience by proposing and applying a new integrated approach for satellite image classification and comparing its efficiency against the best-known machine learning techniques. Our approach integrates FOBIA with DL-CNN while also implementing the FSE-DST accuracy assessment method. Therefore, the results of this study support future research that aims to identify the most effective data-driven approach in the remote sensing domain. It may thus help to improve the classification accuracy of LULC mapping and change detection analysis through fuzzy operators and membership functions in combination with an automated DCNN data-driven approach. From the environmental perspective, our results indicate that Urmia Lake has been in a fragile state under the impacts of intensive LULC changes, and that agricultural and residential activities in the ULB have increased drastically. This seems to be a significant driver of the problems – aside from the drought. Based on the results of this research, our future research will focus on monitoring the soil salinity extension and how this affects the productivity of agricultural lands and food security in the ULB. In addition, part of our future research will focus on applying different DL algorithms such as deep multi-layer perceptron and AlexNet and comparing their efficiency when integrating them with FOBIA-DL. In light of the Urmia Lake crises and various national and international lake restoration attempts, our results shall support decision-makers and authorities in a range of applications, such as mitigation, land use allocation and precision agriculture activities. We are currently witnessing the beginning of an integration of OBIA and state-of-the-art machine learning approaches.

Acknowledgements

This research was jointly funded by a research grant from the University of Tabriz (s818) and the Alexander von Humboldt Foundation.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This research was jointly funded by a research grant from the University of Tabriz (s818) and the Alexander von Humboldt Foundation.