DSFA: cross-scene domain style and feature adaptation for landslide detection from high spatial resolution images

ABSTRACT

Rapid and accurate landslide inventory mapping is significant for emergency rescue and post-disaster reconstruction. Nowadays, deep learning methods exhibit excellent performance in supervised landslide detection. However, due to differences between cross-scene images, the performance of existing methods is significantly degraded when directly applied to another scene, which limits the application of rapid landslide inventory mapping. In this study, we propose a novel Domain Style and Feature Adaptation (DSFA) method for cross-scene landslide detection from high spatial resolution images, which can leverage labeled source domain images and unlabeled target domain images to mine robust landslide representations for different scenes. Specifically, we mitigate the large discrepancy between domains at the dataset level and feature level. At the dataset level, we introduce a domain style adaptation strategy to shift landslide styles, which not only bridges the domain gap, but also increases the diversity of landslide samples. At the feature level, adversarial learning and domain distance minimization are integrated to narrow large feature distribution discrepancies for learning domain-invariant information. In addition, to avoid information omission, we improve the U-Net3+ model. Extensive experimental results demonstrate that DSFA has superior detection capability and outperforms other methods, showing its great application potential in unsupervised landslide domain detection.

KEYWORDS:

• Landslide detection
• deep learning
• remote sensing
• domain adaptation
• high spatial resolution

1. Introduction

Landslides are a catastrophic natural disaster that causes significant damage to human safety, property, and the natural environment (Azarafza et al. 2021; Nikoobakht et al. 2022; Taalab, Cheng, and Zhang 2018; Trinh et al. 2022). Effective and accurate landslide detection is critical to support post-disaster rescue and reconstruction efforts (Fang et al. 2020, 2021; Lissak et al. 2020; Lu, Bai, et al. 2019; Lu, Qin, et al. 2019; Wang, Fang, and Hong 2019). Initial landslide identification methods mainly depend on field surveys, which are not only time-consuming and laborious, but also do not adequately satisfy the requirements of large-scale landslide mapping. Remote sensing technology offers significant advantages in terms of efficiency, objectivity, and timeliness (Scaioni et al. 2014). Therefore, landslide detection based on remote sensing images has been investigated extensively (Chen et al. 2007; Yang et al. 2023).

Many landslide detection methods are proposed to complete the target identification task. In general, they can be classified into two categories: traditional and deep learning methods. To distinguish landslides using traditional methods, researchers propose a pixel-based strategy. However, this strategy does not consider adjacent region connections and results in many isolated landslide points (Ghorbanzadeh et al. 2019; Vohora and Donoghue 2004; Yi and Zhang 2020). Hence, object-based methods are devised to construct subregion pieces as clauses to detect landslides, which reduces isolated points and eliminates spectral variations within the objects (Amatya et al. 2021; Ghorbanzadeh, Gholamnia, and Ghamisi 2022; Hölbling et al. 2012). However, object-based methods require extensive experiments to define the input parameters as well as sophisticated expertise.

Recently, deep learning methods allow feature information to be analyzed automatically without human–computer interaction and thus receive widespread attention from the computer vision community (Ding et al. 2016; Ghorbanzadeh et al. 2022; Si et al. 2023; Si, He, Wu, et al. 2022; Si, He, Zhang, et al. 2022; Zhang, Wang, et al. 2022). Deep learning has been applied for the landslide detection field, and the results demonstrate the superiority of deep learning techniques over traditional methods (Ghorbanzadeh et al. 2019; Prakash, Manconi, and Loew 2020; Yi and Zhang 2020). Effective network frameworks such as matrix semantic segmentation network (SegNet), pyramid scene parseing network (PSPNet), visual geometry group (VGG), Residual network (ResNet), dense network (DenseNet), and U-Net are utilized to detect landslides (Hacıefendioğlu, Demir, and Başağa 2021; Li et al. 2021; Li and Guo 2020; Yu et al. 2021). Among them, the U-Net model is a well-known encoder-decoder architecture that yields superior results (Ghorbanzadeh et al. 2021; Meena et al. 2022; Shahabi and Ghorbanzadeh 2022; Soares, Dias, and Grohmann 2020). Researchers develop U-Net variants to learn deep semantic information and capture multi-scale features (Ghorbanzadeh, Gholamnia, and Ghamisi 2022; Li et al. 2023; Liu et al. 2020; Qi et al. 2020). For example, U-Net3+ is a typical upgraded version of U-Net that can better fit fine-granularity information and exhibits significant potential for landslide detection (Huang et al. 2020).

However, the superior performance of deep learning techniques is driven by a large number of annotations, where the training and test datasets have the same feature distribution (Ghorbanzadeh et al. 2022; Zou et al. 2018). These models do not perform well for cases involving new landslide detection scenes because the information from new scenes is inconsistent with the knowledge learned using previous methods (Zou et al. 2018). Therefore, they often suffer significant performance degradation when faced with new landslide detection scenes (Qin et al. 2021; Xu et al. 2022). Labeling significant amounts of data for each scene is time-consuming and impractical for real-world applications. Hence, transferring landslide detection models from one scene to another is essential and has become a widely investigated research topic.

Unsupervised domain adaptation (UDA) focuses on addressing domain shifts and avoiding expensive laborious annotations (Wang et al. 2021). UDA utilizes labeled source data and unknown target data to complete the knowledge transfer process such that advanced results in target domains that do not participate in the training process can be generated (Song et al. 2019; Toldo et al. 2020). The existing UDA methods can be classified into three categories: consistency regularization, adversarial learning, and self-learning. Consistency regularization aims to learn an appropriate feature transformation that projects the source and target domains into similar feature representation spaces (Long et al. 2015; Sun and Saenko 2016). For example, in Tzeng et al. (2014), the authors construct a maximum average discrepancy constraint to address the domain-shift problem by projecting the source and target domains into a similar feature space. Adversarial learning employs adversarial loss driven by an additional discriminator to minimize domain shifts (Dong et al. 2017; Zhang, Lemoine, and Mitchell 2018). Typically, the process contains an adversarial network G and a discriminative model D. G attempts to extract various target information, whereas D attempts to distinguish different types of features. After iterative games between G and D, G is stimulated to mine meaningful domain-invariant representations (Shamsolmoali et al. 2021). In addition, self-learning typically involves the generation of pseudo-label for the target domain. Subsequently, the segmentation network is fine-tuned based on the generated pseudo-label and labeled source domain data (Zou et al. 2018). Hence, the performance of self-learning methods primarily depends on the quality of the generated pseudo-label.

Although domain adaptation strategies receive considerable attention in other fields, only a few related studies propose suitable technologies for cross-scene landslide detection. Hence, unsupervised deep learning models for landslide detection are still in the early development stages. Cross-scene landslide detection is affected by many factors, such as significant differences in the environment, shape, and complex spatial structure. Thus far, researchers consider feature distributions and ignore dataset level landslide domain discrepancies; as such, cross-scene information cannot be fully utilized to mine landslide features. Therefore, we propose the domain style and feature adaptation (DSFA) method for cross-scene landslide detection to transfer landslide features from the source domain to the target domain for alleviating the significant domain bias from different scenarios at the dataset and feature levels. As shown in Figure 1, style discrepancies and visual differences result in significant domain bias between the source and target domain images. We propose a domain style adaptation strategy that shifts the landslide style from the source domain to the target domain at the dataset level. Concretely, we randomly select a source domain sample and a target domain sample as the input sample pair to optimize the style-transferred model, which generates the corresponding style-transferred landslide samples. These style-transferred landslide samples preserve content information and alter the landslide style, which not only enhances the diversity of landslide images, but also reduces the distance between domains. Owing to the promotion of style-transfer samples, deep models can perceive different types of landslide information. Subsequently, we combine actual and style-transferred landslide samples to optimize our model for analyzing domain-invariant features.

Figure 1. Cross-scene domain adaptation from two different levels. (a) Domain style adaptation method bridges the gap between source and target domain images at the dataset level, (b) adversarial learning and domain distance minimization are integrated to explore domain-invariant information by narrowing the domain discrepancy at the feature level.

Considering the complex spatial structure of different domains, we propose a response feature alignment scheme to further mine domain knowledge based on the feature structure similarity at the feature level. Specifically, we leverage domain distance minimization and adversarial loss to guide the feature extractor in learning global domain-invariant feature representations. For adversarial learning, a general adversarial model comprising a segmentation network and a domain discriminator is established to reduce the data distribution discrepancy between different domains. The segmentation network focuses on extracting beneficial and invariant landslide representations to confuse the discriminators. Considering that U-Net3+ may cause information omission in the max-pooling operation, we use the tokenized multilayer perceptron (MLP) instead of the max-pooling operation in the original U-Net3+ to learn better representations because the tokenized MLP can use appropriate parameters and computations to encode meaningful feature information (Valanarasu and Patel 2022). The improved U-Net3+ is regarded as a segmentation network. The discriminator aims to distinguish the domains from their source or target domains. By performing consecutive games, the segmentation model can exploit domain-invariant feature presentations. Additionally, we introduce a domain distance minimization method to project different domain features into a close feature space to alleviate the domain gap. Finally, adversarial learning, domain distance minimization, and domain style adaptation strategy are seamlessly integrated to mine the domain invariance features for cross-scene landslide identification. We evaluate the effectiveness of the proposed method in three independent landslide areas to demonstrate that the DSFA achieves state-of-the-art performance and outperforms other competitive approaches.

The main contribution of this study can be summarized as follows:

• At the dataset level, we propose a domain style adaptation strategy by transferring landslide styles from one domain to another. These style-transferred landslide images preserve the content information and alter the landslide style, which not only enhances the diversity of landslide images, but also reduces the distance between domains.

• At the feature level, owing to the complexity of landslide shapes, spatial structures, and environments, we propose a corresponding feature alignment scheme for mining different domain knowledge based on domain distance minimization and adversarial constraints.

• As for the segmentation network, to avoid information omission during down sampling, we improve U-Net 3+ by embedding a tokenized MLP module into U-Net3+ to explore detailed landslide representations.

• Finally, we perform a comprehensive analysis of the cross-scene landslide detection and demonstrate the effectiveness of each component in DSFA. Results of extensive experiments show that DSFA performs better than other state-of-the-art methods, thus demonstrating its potential for future cross-scene landslide detection.

2. Study area

Numerous experiments are conducted in three typical independent areas with different environmental contexts to validate the performance of the proposed method. In Figure 2, some landslide samples are presented, and a significant domain shift caused by different landslide surfaces, illumination conditions, vegetation recoveries, background context, spatial resolution, etc. is indicated among the three experimental sites. As the three study areas belong to different scenes and possess completely different properties, they indicate significant feature distribution discrepancies. Hence, the three study areas offer adequate complementary advantages for verifying the cross-scene landslide identification effect of our unsupervised method.

Figure 2. Optical images of landslide samples from three study areas. The first column indicates landslides in Experimental site 1 (Earthquake-induced in Japan). The second column denotes landslides in Experimental site 2 (Rainfall-induced in China). The third column represents landslides in Experimental site 3 (Rainfall-induced in Brazil).

2.1. Experimental site 1: earthquake-induced in Japan

Japan is located on the border of the Eurasian Plate, which is susceptible to natural disasters. Hokkaido belongs to the temperate monsoon climate, with annual precipitation ranging from 800 to 1200 mm. In September 2018, a magnitude Mw 6.6 earthquake hit the Iburi-Tobu area of Hokkaido, Japan, with the epicenter located at 42.72° North and 142.0° East at a depth of approximately 37.0 km. This event resulted in numerous landslides over an area measuring 700 km², as well as resulted in more than 40 fatalities and 600 injuries, most of which are caused by landslides. In this study, we select a severely affected location in this region, as indicated by the red rectangle in Figure 3. The ground truth of the experimental site has been obtained by other researchers and utilized to complete some study works (Shahabi and Ghorbanzadeh 2022; Xu et al. 2022). The earthquake caused 6287 landslides that affected areas ranging from 81.64 m² to 1798397.26 m². We download remote sensing images with a spatial resolution of 3 m from PlanetScope.

Figure 3. Geographic location of three typical experimental sites. The red dot area represents the location of experimental site 1, the green dot area represents the location of experimental site 2, and the blue dot area represents the location of experimental site 3.

2.2. Experimental site 2: rainfall-induced in China

According to the Carpens Climate Classification Standard, the region belongs to the subtropical monsoon climate. The average annual precipitation is 1585.62 mm, and most of the precipitation occurs in June, July, and August. The unique geography of Taiwan renders it extremely susceptible to natural disasters such as earthquakes, floods, and landslides. The Forestry Bureau of Taiwan launched a project to generate an island-wide landslide inventory map for the period 2004–2018. This landslide inventory has been used extensively for various applications (Chen et al. 2019; Lin, Ke, and Lo 2017). Because planet satellites have been launched in recent years, we download PlanetScope satellite images with a spatial resolution of 3 m in 2017. We randomly select a region in Taiwan as our experimental site, as indicated by the green rectangle in Figure 3. It has 3021 landslides, which encompass an area measuring 61.89–665124.79 m².

2.3. Experimental site 3: rainfall-induced in Brazil

Experimental site 3 locates in the Nova Friburgo Mountainous Area of Rio de Janeiro, Brazil. Based on the criterion of the Carpens climate classification, the region belongs to the subtropical plateau climate. The average annual precipitation is 1585.62 mm, and most of the rainfall occurs in November, December, and January. In January 2011, Rio de Janeiro, Brazil, experienced its most treacherous rainfall, which featured a precipitation rate of 350 mm/ 48 h and resulted in more than 3500 translational landslides. The incident killed at least 1500 people and damaged most city facilities. We select one of the most severely affected regions as our third experimental site (blue rectangle in Figure 3). The landslide inventory mapping of this region has been manually generated using high-resolution satellite images by other scholars (Netto et al. 2013). The study area measured 625 km², and heavy rainfall resulted in 832 landslides encompassing an area measuring 200.32–78117.35 m². Remote sensing images are downloaded from RapidEye with a spatial resolution of 5 m.

3. Methodology

For a labeled source domain image S and an unlabeled target domain image T, the proposed DSFA method aims to analyze domain-invariant representations of landslides by implementing dataset and feature level constraints for unsupervised landslide domain adaptation. The overall framework of the DSFA is shown in Figure 4, and comprises three components: landslide domain style adaptation, segmentation network, and feature alignment.

Figure 4. Domain Style and Feature Adaptation (DSFA) framework for cross-scene landslide detection. (a) Style-transferred samples are generated for domain style adaptation to bridge domain discrepancy at the dataset level, (b) a tokenized MLP is embedded into U-Net3+ to avoid information damage in the segmentation network, and (c) adversarial learning and domain distance minimization are integrated to mitigate the large feature discrepancy between source and target domains at the feature level.

3.1. Landslide domain style adaptation

Because of the different types of landslide surfaces, illumination conditions, vegetation recoveries, background contexts, and spatial resolutions, the source and target domain images differ significantly, as shown in Figure 2, which further complicates unsupervised landslide domain adaptation. Reducing the significant discrepancy between different domains is an effective method for solving problems associated with unsupervised domain adaptations. Thus far, researchers merely implement feature constraints to minimize the feature distribution discrepancy at the feature level, which renders it difficult to eliminate the considerable visual effects between the source and target domain images. Researchers of computer vision propose image-generation methods for generating new images to solve specific unsupervised adaptation tasks. Early conventional methods focus on texture synthesis but require complex formulation computations (Bruckner and Gröller 2007). The rapid development of convolutional neural networks has enabled the adoption of convolutional operations to construct generative adversarial networks for transferring image styles, thereby resulting in better performance (Wang, Li, et al. 2020; Zhu et al. 2017).

However, the application of deep learning operations to cross-scene landslide detection has not been discussed. In this study, our goal is to generate new landslide samples that preserve the full content of the source domain images and possess the landslide style of the target domain images to reduce the domain distance at the dataset level. Hence, we propose a novel landslide domain style adaptation scheme that not only smooths image-style disparities, but also enhances the diversity of landslide images.

Because of the limited receptive field of the convolution operation, long-distance dependency cannot be captured; hence, semantic information is omitted. Recently, transformers demonstrate significant potential in capturing long-range dependencies and achieving superior performance (Paul and Chen 2022). Therefore, we extend a state-of-the-art transformer-based network, StyTr², using a domain style adaptation strategy to reduce the distance between the two domains (Deng et al. 2022). Leveraging the ability of the transformer to capture long-term dependencies, StyTr² constructs two different transformer-based encoders to generate feature sequences for modeling the content and style features, separately. Additionally, the model uses a transformer-based decoder to stylize the content sequence based on the style sequence. Therefore, StyTr² can simultaneously maintain fine content structures and rich style patterns. To complete model optimization, we randomly select a source domain sample and a target domain sample as the input sample pair to train the deep model. After training, a content image from the source domain and a style image from the target domain is selected to generate new style-transfer landslide samples, which not only store innate contextual information, but also exhibit styles similar to other scenes. Finally, we combine the actual and style-transferred images to optimize the segmentation network, which can improve the generalization performance of cross-scene landslide detection.

3.2. Segmentation network

As U-Net3+ combines low-level details and high-level representations from multiple scales, we employ U-Net3+ as the baseline network to detect landslides. Four max-pooling operations in the U-Net3+ encoder block cause information omission during the down sampling process. Tokenized MLP modules show excellent capabilities for building models (Valanarasu and Patel 2022). To reduce feature omission, we replace max pooling with an MLP-based tokenization module in each encoder block of U-Net3+ and named it the ‘improved U-Net3+’.

Tokenized MLPs aim to project convolutional landslide features into abstract objects and then employ the MLP to learn valuable information, thus requiring fewer computations and enabling the modeling of high-level representations (Valanarasu and Patel 2022). Specifically, the feature maps from each encoder block are projected onto the tokens via convolutional operations. These generated tokens are transmitted to a shifted MLP (across the width), which is suitable for analyzing local features corresponding to specified axial shifts. Subsequently, the output is fed in a depth-wise manner to promote encoder features without requiring excessive computation time. Subsequently, these features are fed into another shifted MLP (across the height) to obtain the feature maps. Finally, residual connections are employed to add the original tokens as residuals. Therefore, the tokenized MLP blocks perform better than standard positional encoding techniques. Thus, the improved U-Net3+ can exploit more detailed landslide representations.

3.3. Feature alignment

On the other side of the feature level, we design a feature alignment process to constrain the feature distribution and eliminate significant feature discrepancies. Adversarial learning can be used to learn domain-invariant feature representations, and domain distance minimization can be directly used to reduce feature distances across different domains. Hence, we seamlessly integrate adversarial learning and perform distance minimization to complete feature alignment, which results in an effective mapping of different features to a similar distribution space.

Adversarial learning is performed to search for domain-invariant features between source and target domains that are competent in domain adaptation tasks (Tsai et al. 2018). To reduce the significant feature distribution discrepancy, we implement an adversarial learning scheme to learn robust landslide features and improve the generalization ability of the segmentation model. In particular, we design an adversarial model comprising a segmentation network G and a domain discriminator D. In this study, the improved U-Net3+ is termed the segmentation network G. D is composed of five convolutional layers with 4 × 4 kernels, where the numbers of channels are e {64, 128, 256, 512, 2}. G explores domain-invariant landslide representations to confuse D, whereas D distinguishes whether the information originated from the source or target domain. After a continuous game between G and D, landslide features from different domains are projected onto a similar spatial distribution.

The decoder of our segmentation model contains five blocks. To fully analyze the domain-invariant information, we impose adversarial constraints on two types of predicted probability from different blocks, i.e. the fourth and fifth blocks. For the constraint of the fifth block, we extract the predicted probabilities of the source and target domain images. Subsequently, we feed them into the discriminator to ascertain the domain to which they belong. In the adversarial optimization phase, we utilize the binary cross-entropy loss to optimize the domain discriminator. (1) $L_{D_1}=-{\mathbb{E}}_{p_{s_1}\sim P\left(p_{s_1}\right)}\log D\left(p_{s_1}\right)-{\mathbb{E}}_{p_{t_1}\sim P\left(p_{t_1}\right)}\log \left(1-D\left(p_{t_1}\right)\right)$(1) where $p_{s_1}$ and $p_{t_1}$ denote the predicted landslide probabilities of the source and target domains, respectively; $P\left(p_{s_1}\right)$ and $P\left(p_{t_1}\right)$ indicate the distributions of predicted probability from the source and the target domains, respectively; $D\left(p_{s_1}\right)$ and $D\left(p_{t_1}\right)$ represent the probabilities that the discriminator distinguishes $p_{s_1}$ and $p_{t_1}$ as the source and target domains, respectively.

For the target domain images, we feed the predicted landslide probabilities into the discriminator to discriminate the inputs belonging to the source or target domains. To mine the domain-invariant features and complete the adversarial process, we force the discriminator to identify the target domain information as the source domain by optimizing the segmentation model to project different domains into similar feature spaces. Therefore, the adversarial loss for the segmentation network can be written as: (2) $L_{G_1}=-{\mathbb{E}}_{D\left(p_{t_1}\right)\sim p\left(D\left(p_{t_1}\right)\right)}\log \left(D\left(p_{t_1}\right)\right)$(2) where $D\left(p_{t_1}\right)$ denotes the probability of the discriminator distinguishing $p_{t_1}$ from the source domain.

For adversarial learning, the optimization process of the fourth block is the same as that of the fifth block. Therefore, we extract the predictions from the fourth block and fed the results to the discriminator. The constraint function for optimizing the discriminator and segmentation network is expressed as: (3) $L_{D_2}=-{\mathbb{E}}_{p_{s_2}\sim P\left(p_{s_2}\right)}\log D\left(p_{s_2}\right)-{\mathbb{E}}_{p_{t_2}\sim P\left(p_{t_2}\right)}\log \left(1-D\left(p_{t_2}\right)\right)$(3) (4) $L_{G_2}=-{\mathbb{E}}_{D\left(p_{t_2}\right)\sim p\left(D\left(p_{t_2}\right)\right)}\log \left(D\left(p_{t_2}\right)\right)$(4) We combine these two scales to implement the adversarial process and calculate the overall adversarial loss as follows: (5) $L_{adv}=L_{D_1}+L_{D_2}+{\lambda }_1{L}_{G_1}+{\lambda }_2{L}_{G_2}$(5) where ${\lambda }_1$ and ${\lambda }_2$ control the importance of different scales to optimize the segmentation model. We empirically set ${\lambda }_1$ and ${\lambda }_2$ to 0.01 and 0.001, respectively.

Domain distance minimization directly reduces the feature distribution discrepancy between the source and target domains. As a practical constraint function, the maximum mean discrepancy can be used to learn feature representations by minimizing the feature distribution discrepancy (Tzeng et al. 2014). Therefore, at the feature level, we employ a maximum mean discrepancy constraint function to reduce the distance between the landslide representation and the first block of the decoder. The discrepancy between the two domains is formulated as: (6) $L_{mmd}=\frac{1}{m}\sum _{i=1}^{m}k(a{)}^i-\frac{1}{n}\sum _{j=1}^{n}k(b{)}^{j2}$(6) where k indicates the Gaussian radial basis function kernel for computing the feature distribution; and a and b denote the source and target features, respectively.

3.4. Final optimization objective

Because the source domain possesses label information, we use labeled source data to optimize the segmentation model. Considering the imbalance between landslide positive and negative samples, we exploit the Dice function to analyze the landslide features of the source domain at two scales from the fourth and fifth decoder blocks (Milletari, Navab, and Ahmadi 2016). The constraint function is expressed as: (7) $L_{seg}=\left(1-{\displaystyle \frac{2\cdot d\cdot p_{s_1}}{d+p_{s_1}}}\right)+\ell \cdot \left(1-{\displaystyle \frac{2\cdot d\cdot p_{s_2}}{d+p_{s_2}}}\right)$(7) where d denotes the ground truth; and $\ell$ represents the weight of the dice loss in the fourth block, which is set to 0.1.

We employ source domain label information to learn the landslide features. Meanwhile, we reduce the domain discrepancy at both the dataset and feature levels. Therefore, the overall objective of the proposed method can be expressed as follows: (8) $L_{total}=L_{seg}+L_{adv}+\gamma L_{mmd}$(8) where γ indicates the importance of L_mmd, and its value is set to 0.01 in this study.

During the training operation, we alternately optimize the segmentation and discriminator networks. When the parameters of one component are updated, those of the other components are fixed. After training, we use only the optimized segmentation model to predict landslides in the target domain images during testing.

3.5. Data preprocessing

Understanding the environmental conditions in which landslides occur is essential for landslide detection. Researchers discover that adopting a combination of optical and digital elevation model (DEM)-related factors can yield better results compared with using a single optical image, thus demonstrating the effectiveness of multiple-source data (Liu et al. 2020). Therefore, we employ the NDVI, DEM, and six related topographic influencing factors, slope, aspect, hill shade, curvature, profile curvature, plan curvature, and combine them with optical images to identify landslides.

In terms of data preprocessing, to match the spatial resolution of the optical image to that of the DEM of the Alaska Satellite Facility (i.e. 12.5 m), we use the QGIS to resample the DEM data through bilinear interpolation. To facilitate model optimization, we conduct experiments using only four typical spectral channels (blue, green, red, and near-infrared) of the RapidEye and PlanetScope data, excluding the red edge band in the RapidEye data. Subsequently, we incorporate these spectral channels with the aforementioned predisposing factors into a single merged file and utilize the Z-scores for image normalization. Finally, we partition the remote sensing image into 128 × 128 patches with a 20% overlap to preserve contextual information. The proposed model is comprehensively validated in three independent test areas, as shown in Figure 3.

3.6. Experimental settings

All experiments are implemented on the PyTorch framework on a Linux platform equipped with NVIDIA RTX 3090 GPUs with 24 G memory. During the training phase, we employ the Glorot normal initializer to initialize the network weights. We utilize stochastic gradient descent to optimize the model with a momentum of 0.9 and a weight decay of 10⁻⁴. The batch size, initial learning rate, and number of iterations are set to 8, 0.01, and 20k, respectively. During the testing phase, we utilize only the trained segmentation network to detect landslides. In our experiments, we use six standard evaluation metrics to demonstrate the effectiveness of the proposed DSFA, i.e. accuracy, precision, recall, F1-score, kappa, and mean intersection over union (mIoU).

We refer to experimental sites 1, 2, and 3 as target domains. For the source domain, six ranking combinations are established, among which three (i.e. Groups A, B, and C) are randomly selected for cross-scene landslide detection experiments. The three experimental groups are described in Table 1.

Table 1. Description of three groups.

Group name	Source domain	Target domain
Group A	Experimental site 2	Experimental site 1
Group B	Experimental site 1	Experimental site 2
Group C	Experimental site 1	Experimental site 3

4. Experimental results

4.1. Comparison with state-of-the-art methods

To demonstrate the superiority of the DSFA method, we compare it with five other competitive unsupervised methods: MCD (Tzeng et al. 2014), FADA (Wang, Shen, et al. 2020), PyCDA (Lian et al. 2019), DAFormer (Hoyer, Dai, and Van Gool 2022b), and HRDA (Hoyer, Dai, and Van Gool 2022a). The predicted results and landslide inventory mappings are qualitatively shown in Figures 5, 6, and 7. The orange regions indicate the predicted results derived from the deep learning models, and the blue regions indicate landslide inventory mapping. Therefore, the overlapping area between the reference and automatic inventories is the intersection of both sets, which indicates the correctly predicted area.

Figure 5. Qualitative comparison of methods on Experimental site 1. (a) MCD, (b) FADA, (c) PyCDA, (d) DAFormer, (e) HRDA, and (f) DSFA.

Figure 6. Qualitative comparison of different methods on Experimental site 2. (a) MCD, (b) FADA, (c) PyCDA, (d) DAFormer,(e) HRDA, and (f) DSFA.

4.1.1. Results of Group A

For a qualitative comparison, we randomly select a sub-region for better visualization. As shown in Figure 5, most large landslides are correctly identified. Unlike other UDA models, the DSFA can yield correct and fine-grained detailed segmentation results. Among the compared approaches, PyCDA yields the worst results for cross-scene landslide detection. During the training process, PyCDA defines high-confidence prediction regions as pseudo-label to optimize the deep model. However, the generated pseudo-label inevitably contains considerable noise, which can easily mislead model optimization (Zhang, Jing, et al. 2021).

For a quantitative comparison, the evaluation results are listed in Table 2. DAFormer achieves a maximum precision of 88.35%, which is higher than that of the proposed DSFA. However, the values of the other metrics derived from DAFormer are much lower than those derived from the DSFA, which demonstrates the weak landslide detection ability of DAFormer. As for the recall value, PyCDA achieves a maximum value of 80.24% but indicates the lowest precision, which implies that most of the predicted landslides are incorrect. In terms of the other metrics, the DSFA indicates values exceeding 70%, which implies that is more reliable than the other UDA methods. Additionally, the DSFA results in more detailed boundaries and a label that is highly similar to the actual landslide label. In conclusion, the DSFA can be successfully applied to cross-scene landslide detection at experimental site 1 and exhibits accurate and reliable capabilities for landslide identification.

Table 2. Evaluation indices (%) of the six UDA methods in Group A. The highest accuracy for each index is highlighted in bold.

Methods	Accuracy	Precision	Recall	F1-score	Kappa	mloU
MCD	94.67	86.85	57.60	69.26	66.47	73.65
FADA	94.24	83.98	55.33	66.71	63.70	71.97
PyCDA	83.52	36.72	80.24	50.39	42.10	57.85
DAFormer	93.22	88.35	40.33	55.38	52.26	65.61
HRDA	91.85	76.21	31.71	44.79	41.18	60.21
DSFA	97.51	85.43	71.00	77.55	75.21	79.35

4.1.2. Results of Group B

Qualitative detection results are shown in Figure 6. Similarly, PyCDA yields numerous over detections. MCD, FADA, DAFormer, and HRDA exhibit better detection capabilities than PyCDA. However, many regions are not successfully recognized by these methods, unlike our method. In addition, coarse landslide segmentation boundaries are generated. The proposed DSFA generates fine-grained landslide location boundaries with less over-detection compared with the other methods.

The results are summarized in Table 3. As shown, the DSFA yields the highest value among the methods compared, and its evaluation metrics show values exceeding 80%, except for precision, which confirms the stable and superior performance of the DSFA. PyCDA yields the highest recall of 87.75% and the lowest precision of 9.35%, thus indicating that a significant percentage of regions are incorrectly classified as landslides. Therefore, PyCDA is unsuitable for UDA landslide detection.

Table 3. Evaluation indices (%) of the six UDA methods in Group B. The highest accuracy for each index is highlighted in bold.

Methods	Accuracy	Precision	Recall	F1-score	Kappa	mloU
MCD	96.64	72.35	78.20	75.16	73.37	78.34
FADA	97.47	79.48	82.26	80.85	79.50	82.59
PyCDA	44.30	9.35	87.75	16.89	5.85	25.09
DAFormer	96.31	70.44	74.41	72.36	70.39	76.41
HRDA	96.73	79.21	67.23	72.23	71.00	76.86
DSFA	97.85	83.69	83.16	83.42	82.27	84.64

4.1.3. Results of Group C

For a qualitative comparison, the detection results by different methods are shown in Figure 7. The source and target domain images have different spatial resolutions of 3 and 5 m, respectively. Furthermore, they are subjected to different landslide types, such as earthquakes and rainfall events. As shown in Figure 7, the DSFA indicates better accuracy than the other methods. In terms of quantitative comparison, Table 4 shows that the DSFA offers the highest accuracy, precision, F1-socre, kappa, and mIoU, thus further corroborating the superiority of the DSFA.

Figure 7. Qualitative comparison of different methods on Experimental site 3. (a) MCD, (b) FADA, (c) PyCDA, (d) DAFormer, (e) HRDA, and (f) DSFA.

Table 4. Evaluation indices (%) of the six UDA methods in Group C. The highest accuracy for each index is highlighted in bold.

Methods	Accuracy	Precision	Recall	Fl-score	Kappa	mloU
MCD	89.73	47.43	79.14	59.31	53.86	65.53
FADA	87.50	42.17	86.78	56.76	50.45	63.00
PyCDA	64.22	19.26	87.22	31.56	19.01	39.87
DAFormer	86.51	38.89	74.73	51.16	42.22	59.93
HRDA	91.30	54.21	51.36	52.75	47.96	63.34
DSFA	90.76	50.67	88.13	64.34	59.48	68.68

4.2. Ablation experiments

To comprehensively evaluate the advantages of each component in the proposed DSFA method, we randomly select three groups (i.e. Groups A, B, and C) for cross-scene landslide detection experiments to conduct a series of ablation experiments. The evaluation results for the three databases are listed in Tables 5, 6, and 7, respectively.

Table 5. Evaluation indices on Group A (%). The highest accuracy of each index is highlighted in bold.

Es1→Es2	Accuracy	Precision	Recall	F1-score	Kappa	mIOU
U-Net3+	93.18	82.55	43.92	57.33	54.00	66.52
Lseg	93.57	84.89	46.60	60.17	57.00	68.14
Lseg + Ladv	95.18	85.63	64.61	73.65	71.05	76.56
Lseg + Ladv + Lmmd	95.34	86.38	65.68	74.62	72.11	77.26
Lseg + Ladv + Lmmd + DSA	95.71	85.43	71.00	77.55	75.21	79.35

Table 6. Evaluation indices on Group B (%). The highest accuracy of each index is highlighted in bold.

Es2→Es1	Accuracy	Precision	Recall	F1-score	Kappa	mIOU
U-Net3+	89.69	35.09	69.13	46.55	41.51	59.77
Lseg	89.39	35.65	78.82	49.09	44.09	60.68
Lseg + Ladv	97.74	87.18	76.40	81.44	80.24	83.15
Lseg + Ladv + Lmmd	97.50	77.64	86.28	81.73	80.39	83.23
Lseg + Ladv + Lmmd + DSA	97.85	83.69	83.16	83.42	82.27	84.64

Table 7. Evaluation indices on Group C (%). The highest accuracy of each index is highlighted in bold.

Es1→Es3	Accuracy	Precision	Recall	F1-score	Kappa	mIOU
U-Net3+	88.29	43.49	79.59	56.24	50.15	63.23
Lseg	88.68	44.52	79.67	57.12	51.20	63.87
Lseg + Ladv	89.33	46.56	86.78	60.61	55.08	65.93
Lseg + Ladv + Lmmd	90.16	48.83	84.31	61.85	56.65	67.04
Lseg + Ladv + Lmmd + DSA	90.76	50.67	88.13	64.34	59.48	68.78

U-Net3+ is a baseline model trained using only source domain data. We test the model performance on the target domain and obtain unsatisfactory performance. L_seg indicates that the improved U-Net3+ is employed to optimize the segmentation network, which is better than the original U-Net3+. The experimental results confirm the effectiveness of employing a tokenized MLP instead of max pooling to avoid information omission.

We incorporate adversarial learning into unsupervised domain adaptation, which can be expressed as L_seg + L_adv. Based on Tables 5, 6, and 7, L_seg + L_adv performs better than L_seg, which proves that adversarial learning allows domain-invariant features to be analyzed such that cross-scene landslide detection can be performed more effectively.

In addition, we further integrate domain distance minimization into the optimization process, which can be expressed as L_seg + L_adv + L_mmd. We discover that L_seg + L_adv + L_mmd results in better performance than L_seg + L_adv, which implies that L_mmd successfully reduces the domain distance between the source and target domains, thereby improving the generalization performance. As shown in Table 7, the accuracy and precision values of L_seg + L_adv exceed those of L_seg + L_adv + L_mmd, whereas its recall, F1-score, kappa, and mIoU values are much lower than those of L_seg + L_adv + L_mmd. This indicates that L_seg + L_adv only detects regions with significant landslides, whereas L_seg + L_adv + L_mmd detects more regions with landslides. The results presented in the three tables indicate above show that we successfully integrate adversarial learning and domain distance minimization to learn domain-invariant landslide information at the feature level.

Finally, we add style-transferred images to train the proposed model, which we named L_seg + L_adv + L_mmd +DSA. Compared with the results of L_seg + L_adv + L_mmd, the recall, F1-score, kappa, and mIoU of L_seg + L_adv + L_mmd +DSA are 0.37%, 5.32%, 2.93%, 3.10%, and 2.09% higher, respectively, whereas its precision decreases by 0.95% in Group A. Although L_seg + L_adv + L_mmd +DSA indicates a precision value lower than those of the other methods, its values for the other metrics are significantly higher. In Group B, although L_seg + L_adv + L_mmd indicates a recall value higher than that of L_seg + L_adv + L_mmd +DSA, its values for the other evaluation metrics are lower. L_seg + L_adv + L_mmd + DSA demonstrates stable and advanced performance, with precision, recall, F1-score, kappa, and mIoU values of 97.85%, 83.69%, 83.16%, 83.42%, 82.27%, and 84.64%, respectively. In Group C, L_seg + L_adv + L_mmd + DSA outperforms L_seg + L_adv + L_mmd in all metrics, thus indicating the extraordinary ability of the DSFA in solving problems related to landslide domain adaptation. These results verify the effectiveness and practicability of the domain style adaptation strategy in the UDA for landslide detection.

5. Discussion

5.1. Effectiveness of domain style adaptation

We propose a domain style adaptation strategy to bridge the gap between source and target domains. Figure 8 shows the visual effects of domain style adaptation. We refer to experimental sites 1 and 2 as source domain and the target domain, respectively. As shown in Figure 8(a,b), the source and target domain images have different image styles and visual effects, which is the main challenge that hinders the identification of cross-scene landslide detection.

Figure 8. Visualization of different types of landslide samples. (a) Source domain images, (b) target domain images, (c) style-transferred images, and (d) frequency histograms.

After domain style adaption, we convert source domain landslides to have a similar style to the target domain, as shown in Figure 8(c). The style transfer images possess both the original content of the source domain and the style of the target domain, which not only bridges the domain gap, but also increases the diversity of landslide images. It is well-known that domain style adaptation methods can decrease the brightness of source domain images and generate similar style features to target domain images.

Furthermore, we compute the frequency histogram in Figure 8(d) to further analyze the effect of domain style adaptation. We can observe that the statistical curve of the source domain image tends to the left with low values, while the statistical curve of the target domain image tends to the right side with higher values. After domain style adaptation, the statistical curves of style-transferred images are shifted to the middle, thus bridging the modality gap. We combine style-transferred images with real images to optimize the segmentation model, which helps the model effectively explore robust landslide representations. From Tables 5, 6, and 7, it can be concluded that using style-transferred images produces better accuracy than using only real images, which proves that the domain style adaptation strategies are meaningful for landslide detection.

5.2. Advantages of the proposed DSFA

Deep learning has been applied to landslide detection and achieves promising performance (Song et al. 2021). However, the superior performance is driven by the large amounts of labeled data in specific scenes (Hacıefendioğlu, Demir, and Başağa 2021; Li et al. 2021; Li and Guo 2020). When transferred to a different landslide scene, they tend to generate poor identification results, greatly hindering the development of the landslide detection community. In practice, it is difficult to label a large number of landslide samples for each specific scene (Madadi et al. 2020; Pinheiro 2018). To address this problem, Xu et al. (2022) propose an adversarial learning method to explore shared domain landslide features and demonstrated the effectiveness of adversarial learning. However, they ignore the domain discrepancy at the dataset level, which makes it difficult to mine robust landslide features.

Considering the large discrepancy between the source and target domains, we propose DSFA to solve the domain shift problem from the dataset level and feature level. At the dataset level, we introduce a domain style adaption strategy to bridge the domain gap by converting the style of the source domain image to a style similar to the target domain. With the domain style adaption scheme, the landslide segmentation model has made great progress, as listed in Tables 5, 6, and 7.

At the feature level, we improve the original U-Net3+ to avoid representation omissions. Moreover, we integrate adversarial learning and domain distance minimization to align feature distributions from different domains. The effectiveness of each component has been verified in ablation experiments. Extensive experiments demonstrate that DSFA yields fine-grained landslide segmentation boundaries and better results than other UDA methods. DSFA can successfully integrate dataset level and feature level constraints to solve the domain shift problem, showing the potential for future cross-scene landslide detection.

5.3. Observations from comparison models

Based on these sufficient experiments and comparison results, we have the following new observations:

1. Among the comparison methods, PyCDA aims to generate pseudo labels for model optimization. Due to the complexity and diversity of landslides, the pseudo-label generated by PyCDA is of poor quality, affecting the model optimization process. DAFormer and HRDA also achieve better performance than PyCDA by integrating multiple information from different levels to produce pseudo labels. However, their performance is still poor compared to our method, which illustrates that the proposed DSFA is valuable for landslide identification and also proves that the pseudo-label has a vital impact on the final result.

2. As for the comparison methods, MCD and FADA focus on feature alignment, and the proposed DSFA achieves the highest performance. These comparative experiments demonstrate that feature alignment alone is not sufficient for the landslide domain adaption task. In addition, the domain style adaptation strategy significantly bridges the domain gap between source and target domain images. Experiment results demonstrate that it is valuable for narrowing large domain differences from the dataset level.

3. For comparison, the original structures of MCD, FADA, and PyCDA are all deeplabv2, which is not suitable for landslide detection. Therefore, we set MCD, FADA, and PyCDA, to have the same modified structure in the proposed DSFA. As for DAFormer and HRDA, they utilize different model structures to learn effective information. Extensive experiments demonstrate that MCD, FADA, and DSFA have better accuracy than the DAFormer and HRDA, which shows that our baseline is important for landslide identification.

4. Previous segmentation models usually simply fuse contextual information from adjacent scales, ignoring the problem of dense connections from different network layers. U-Net3+ merges feature maps of different scales through full-scale skip connections that promote the landslide detection community (Huang et al. 2020). However, there are four down-sampling operations from max-pooling in U-Net3+, which may destroy some landslide information. To avoid information loss in the down-sampling step and learn detailed landslide representations, we replace the max-pooling operation with a tokenized MLP module in each encoder block of U-Net3+. By integrating full landslide features of different scales, the improved U-Net3+ shows excellent performance in landslide detection. The ablation results show that the modified U-Net3+ can avoid information corruption in the down sampling operation.

5.4. Limitations and future work

Although extensive experimental results show that DSFA has a great ability in solving the problem of cross-scene landslide detection, there is still a large improvement room. To address landslide identification in a wider range of real-world scenes, more variable datasets are required. For Group C, we combine source domain images with a spatial resolution of 3 m and target domain images with a spatial resolution of 5 m to implement cross-scene landslide detection. They are characterized by different landslide types and resolutions. As a result, the identification accuracy is relatively lower than that of Group A and Group B. This means that the source domain is important and affects the whole optimization process, which is a universal consensus problem throughout the field of unsupervised domain adaption. To this end, we further consider valuable solutions. We believe that the synthesis, generation, and realism evaluation of landslide datasets will be meaningful for cross-scene landslide detection. In the future, we will further explore multi-scene landslide detection techniques by acquiring new large-scale or creating meaningful synthesis landslide samples.

6. Conclusion

In this study, a novel DSFA method for unsupervised landslide domain adaptation is proposed. DSFA innovatively resolves the large domain differences at the dataset level and the feature level, thereby greatly improving the transferable accuracy of cross-scene landslide detection. At the dataset level, a domain style adaptation scheme is introduced to transfer the style of source domain images to similar target domain images, bridging domain gaps and increasing the diversity of datasets. Meanwhile, adversarial learning and domain distance minimization are integrated to exploit domain-invariant landslide information at the feature level. To this end, we develop an adversarial model consisting of a segmentation network and a discriminator network. In addition, to avoid representation omission, we improve the original U-Net3+ by importing a tokenized MLP module, and call it a segmentation network. Finally, these modules are seamlessly concatenated to enable the landslide domain adaption task. We conduct extensive experiments on three different scene datasets and compare them with several popular methods, fully demonstrating the superior performance of DSFA. Also, ablation experiments demonstrate that each component improves detection accuracy and they complement each other. This indicates that DSFA is an effective procedure that can facilitate the development of large-scale landslide inventory mapping.

Acknowledgment

The authors would like to thank the handling editors and four anonymous reviewers for their valuable comments and suggestions, which significantly improved the quality of this paper.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

The data that support the findings of this study are available from the corresponding author upon reasonable request.

Additional information

Funding

This work was supported by the National Natural Science Foundation of China under grants 42311530065, in part by the Joint Funds of the National Natural Science Foundation of China under grant U21A2013.