Self-supervised change detection of heterogeneous images based on difference algorithms

ABSTRACT

The presence of heterogeneous image disparities often leads to inferior quality in the generated difference images during change detection. This paper proposes a self-supervised change detection of heterogeneous images based on a difference algorithm. Firstly, a combination of phase consistency and a simplified pulse-coupled neural network (PC-SPCNN) is used to fuse the heterogeneous images, and the result is used to compute the difference image (DI). The new DI generation method can generate the standard and exponential difference images. Secondly, the hierarchical FCM clustering algorithm is improved to extract stable and correct self-supervised samples by difference images so that the clustering process is not overly dependent on thresholds. Then, the support vector machine classifier is trained based on the heterogeneous images, the fused images, and self-supervised sample sets, and the information from the fused images is utilized to increase the feature dimension for better detection of changes. Finally, the support vector machine classifier automatically detects whether the intermediate pixels are changed and produces the change detection results. The experimental results confirm the improvements made by the proposed method in difference image extraction, training sample selection, and clustering algorithm, and the stability of the method exceeds that of the state-of-the-art change detection methods.

KEYWORDS:

• Heterogeneous images
• change detection
• self-supervised learning
• difference image
• hierarchical fuzzy c-means clustering

Introduction

Heterogenous remote sensing image change detection (CD) can relieve the limitations of traditional CD, hampered by adverse lighting conditions, imaging period, and modality. It can meet the technical requirements of land-based applications such as emergency response. However, heterogeneous images with different spectral, textural, and geometric information characteristics pose challenges to high-performance CD (Lv et al., 2022; Shi et al., 2023; Yin et al., 2023).

Recent research methods have been used to find changed regions by clustering the DI to obtain sample data and using trained classification models. However, poor-quality difference images often result in significant overlapping of changed and unchanged information, making it difficult to classify and analyze difference images accurately (Dong et al., 2020).

For example, Xuan et al. (2022) constructed a new DI generation method ratio – mean ratio (RMR) by fusing multiplication-based ratio (R) DI and mean ratio (MR) DI, which preserves image details and reduces noise. However, the method must be extended to multiple scales and rely on suitable fusion weights. Based on the up-to-date literature, it is expected to fuse multiple DIs linearly, often without considering the weights associated with each DI (Shi et al., 2021; Xu et al., 2023). Jiang et al. proposed a local energy weighting (LEW) method to fuse individual DIs instead of the commonly used equal weights, and the method is susceptible to the quality of individual DIs (Jiang et al., 2022). The quality of the DI significantly impacts clustering performance and self-supervised sample extraction. In the self-supervised sample extraction stage, the quality of the DI significantly impacts the sample’s accuracy. Low-quality difference images may contain noise, reducing the accuracy of sample set construction.

Gao et al. (2016) utilized the hierarchical FCM clustering algorithm to use the first cluster of the second-level clustering results as changed pixels and compare the intermediate pixels with the threshold number to complete the classification of intermediate and unchanged pixels. However, the method relies on the threshold number, which is prone to the problem of the insufficient number of changed samples and the low quality of unchanged samples. Geng et al.(2019) utilized the hierarchical FCM clustering algorithm to use the first and last clusters of the second level of clustering as the changed and unchanged sample data, respectively, which leads to a decrease in the number of changed and unchanged samples and an excessive number of intermediate pixels, which increases the difficulty of subsequent classification and also relies on the threshold number (Gao et al., 2016). Difference images affect the clustering accuracy of the changed and unchanged pixels, so obtaining high-quality difference images and using more accurate clustering algorithms and classification models is necessary to improve the CD performance(Jiang et al., 2022). Sun et al. performed CD through a Markov co-segmentation model. The model can fuse the pre- and post-difference images during the segmentation process and enhance the robustness of the images through an iterative framework, which improves the final detection performance. However, the operation efficiency of the method is low (Sun et al., 2021).

Compared with the above methods, deep learning-based methods are characterized by incredible computational difficulty, high hardware requirements, and more complex model design (Li et al., 2022). Deep learning-based classification methods perform better. However, the detector’s performance needs to be improved by increasing the training dataset and changing the training hyperparameters (Davari et al., 2021). The supervised deep learning-based synthetic aperture radar image CD method uses the Deep Belief Network (DBN) as the deep architecture. This network’s training process includes unsupervised feature learning and supervised network fine-tuning. By training the DBN using input images and images generated by morphological operators, a dataset with an appropriate amount and diversity of data can be provided, and a method to significantly reduce the amount of computation without compromising the training performance is introduced. The method exhibits better performance and higher accuracy while achieving shorter processing time. However, its drawback is that a large amount of data is required, and the lack of data affects the accuracy of the neural network (Samadi et al., 2019). Li et al. (2021) proposed a change detection network for optical and SAR remote sensing images based on depth panning, which utilizes image panning to reduce the differences of heterogeneous remote sensing images and an improved depth network to detect the changes. Depth-divisible convolution exhibits satisfactory performance, which reduces the number of model parameters and the time. However, too few parameters lead to the instability of the network, and the computational cost is very high. Du et al. (2023) proposed a multitask change detection framework of optical and SAR images, which avoids complex operations and many parameters and reduces the computational cost. However, it is still a simple framework with room for improvement and needs to be generalized to different types of sensors.

The performance of deep learning-based methods may depend on the quality of large datasets. Once the dataset changes, the parameters and network structure must be manually adjusted, which consumes a lot of labor and time (Davari et al., 2021; H. Li et al., 2022). DI-based CD methods are relatively easy to implement and do not require training data, which can reduce the accumulation of human error to complete the self-supervised extraction of sample data (Ye et al., 2016). Therefore, we consider self-supervised learning, DI, and clustering algorithms to reduce sample extraction’s labor and material costs to achieve CD. However, the existing methods have the following problems that need to be solved:

1. Significant differences and image noise exist between heterogeneous images, which leads to lower CD accuracy.

2. The quality of difference images is easily affected by the differences between heterogeneous images and noise. Improving the quality of difference images is a research direction to improve the model’s performance (Jiang et al., 2020; Liu et al., 2016).

3. Clustering methods involve the setting of a threshold, the size of which directly affects the number and quality of self-supervised samples extracted(Niu et al., 2018; Ou et al., 2022; Pan et al., 2022).

Therefore, existing heterogeneous CD methods regarding high-quality DI generation, CD accuracy, and clustering of changed and unchanged pixels must be improved. To this end, this paper proposes a self-supervised CD of heterogeneous images based on a difference algorithm. First, the PC-SPCNN (Fu et al., 2023) fusion method is used to fuse the heterogeneous images to generate the transition image so that it is comparable to the original image and can provide complementary information. It is similar to the original image regarding radiance spectrum and channel number and is highly sensitive to changed pixels. Second, standard DIs and exponential DIs are generated based on heterogeneous and fused images to mitigate the unfavorable effects of heterogeneous image differences and noise, improve the quality of DIs, and preserve detailed information. Then, a bi-hierarchical FCM clustering method extracts high-quality changed and unchanged samples on standard DIs and exponential DIs. Finally, the heterogeneous and fused images are utilized for training, and a support vector machine classifier is used for prediction to obtain more detailed and accurate CD results. Experimental results from heterogenous image datasets of three scenes confirm that the proposed method does not require auxiliary information from other hyperspectral or high spatial resolution image data. At the same time, it can obtain satisfactory CD results without the need for large datasets. Unlike previous combined DI methods, the proposed method uses standard and exponential difference images to extract changed and unchanged samples separately instead of combining difference images to generate final CD results. This makes the detection accuracy independent of the weights of individual difference images and does not overly rely on the threshold setting in the clustering process (Ferraris et al., 2020).

The remaining sections of this paper are organized as follows: Section 2 describes the proposed method NR-CFI-BI-FCM-SVM in detail, Section 3 introduces the experimental dataset as well as the evaluation metrics, Section 4 demonstrates the performance of our method through comparative experiments and proves the effectiveness of the improved DI and classifier modules through ablation experiments. Finally, Section 5 summarizes the paper and provides an outlook for future research.

Method

The self-supervised CD method for heterogeneous images proposed in this paper adds the fused image into the neighborhood ratio computation to generate high-quality difference images and reduce the influence of the feature differences of the heterogeneous images and noises on the extraction of the correct samples. It is not a direct comparison of the heterogeneous images to extract the samples, the proposed method utilizes the support vector machine (SVM) to classify heterogeneous and fused images with enriched information to obtain highly accurate CD results. As shown in Figure 1, the proposed method comprises three stages: high-quality DI generation, automatic sample sets extraction, and CD.

Figure 1. Technology Roadmap.

1. The PC-SPCNN method fuses heterogeneous images to generate spatiotemporally fused images with detailed spectral, textural, and spatial information. High-quality standard and exponential DIs are generated based on heterogeneous images and their fused images.

2. Based on the standard and exponential DIs, stable and correct self-supervised samples are automatically extracted using the bi-hierarchical FCM clustering method.

3. Train the SVM classifier based on the heterogeneous images, fused images, and automatically extracted sample sets. Use the trained SVM classifier to automatically determine whether the intermediate pixels are changed and produce the final CD results.

Difference image extraction based on neighborhood ratio combined with fused images(NR-CFI)

DI-based methods are used more in homologous image CD, such as difference, ratio (R), improved ratio (IR), logarithmic ratio (LR), mean value ratio (MR), and their different combinations, to obtain DI. However, these methods do not apply to heterogeneous images because the heterogeneous images are so different that it does not make sense to compare the pixel values directly. It is prone to generate much noise. IR differs from R in that it does not require a dual-thresholding algorithm, reducing the difficulty of CD. Suppose$I_i(\mathit{m},\mathit{n})$ denotes the pixel intensity values in the image at the position of $(\mathit{m},\mathit{n})$, m$\in [1,\mathit{M}]$, n$\in [1,\mathit{N}]$, where M and N indicate the height and width of the image$I_i$, respectively. Then IR is expressed as equation(1)(Zhuang et al., 2018).

(1) $D_{\mathit{IR}}(\mathit{m},\mathit{n})=\frac{min\left\{I_1(\mathit{m},\mathit{n}),I_2(\mathit{m},\mathit{n})\right\}}{\mathit{max}\left\{I_1(\mathit{m},\mathit{n}),I_2(\mathit{m},\mathit{n})\right\}}$(1)

Where, $D_{\mathit{IR}}$ denotes the DI obtained via the IR method, while$D_{\mathit{IR}}(\mathit{m},\mathit{n})$represents the intensity value of the pixel DI at position $(\mathit{m},\mathit{n})$ in $D_{\mathit{IR}}$. A high value indicates that the pixel remains unchanged, and vice versa. MR differs from IR in that it uses local area averaging to reduce the effect of noise on CD (Seo et al., 2018). The MR method is expressed as equation(2) (Zhuang et al., 2018)

(2) $D_{\mathit{MR}}(\mathit{m},\mathit{n})=\frac{min\left\{u_1(\mathit{x}),u_2(\mathit{x})\right\}}{\mathit{max}\left\{u_1(\mathit{x}),u_2(\mathit{x})\right\}}$(2)

Where,$u_1(\mathit{x})$ and $u_2(\mathit{x})$ represent the mean intensity values of the neighborhood space centered on the location $(\mathit{m},\mathit{n})$for images $I_1$and$I_2$, respectively, with a size of R×R. A high value of$D_{\mathit{MR}}(\mathit{m},\mathit{n})$ indicates that the corresponding pixel remains unchanged, and vice versa. The MR method can detect various changes and suppress the noises to a certain extent, improving CD accuracy (Zhuang et al., 2018). The NR method effectively combines the advantages of IR and MR methods. It uses the heterogeneity measure $\mathit{\theta }$ as the weight ratio, where a low $\mathit{\theta }$ value indicates local homogeneity and a high $\mathit{\theta }$ value indicates local heterogeneity, which can reduce the difficulty of detecting heterogeneous image changes. It is expressed as equation(3)(Zhuang et al., 2018).

(3) $\begin{array}{rl}& D_{\mathit{NR}(\mathit{m},\mathit{n})}=\mathit{\theta }\frac{min\left\{I_1(\mathit{m},\mathit{n}),I_2(\mathit{m},\mathit{n})\right\}}{\mathit{max}\left\{I_1(\mathit{m},\mathit{n}),I_2(\mathit{m},\mathit{n})\right\}}+\\ & \left(1-\mathit{\theta }\right)\frac{{\sum }_{\mathrm{x}\in \mathrm{\Omega }}\mathit{min}\left\{I_1(\mathrm{x}),I_2(\mathit{x})\right\}}{{\sum }_{\mathit{x}\in \mathrm{\Omega }}\mathit{max}\left\{I_1(\mathit{x}),I_2(\mathit{x})\right\}}\end{array}$(3)

(4) $\mathit{\theta }=\frac{\mathit{\sigma }(\mathit{x})}{\mathit{\mu }(\mathit{x})}$(4)

Where, $D_{\mathit{NR}(\mathit{m},\mathit{n})}$ denotes the neighborhood ratio at the image location $(\mathit{m},\mathit{n})$ and describes the local similarity between images $I_1$ and $I_2$ (Fan et al., 2023). A higher similarity corresponds to a higher probability of the pixel remaining unchanged, $\mathrm{\Omega }$ denotes the set of neighborhood space centered at the location $(\mathit{m},\mathit{n})$, while $I_i(\mathit{x})$ represents the pixel intensity value of the x-block matrix within the neighborhood space centered at the location $(\mathit{m},\mathit{n})$, $\mathit{\sigma }(\mathit{x})$denotes the grayscale variance of the neighborhood, and $\mathit{\mu }(\mathit{x})$ denotes the grayscale mean value of the neighborhood (Gao et al., 2016).

However, when$D_{\mathit{NR}}$ used for heterogeneous image CD, the performance is unsatisfactory and has four drawbacks (Seo et al., 2018). First, only the heterogeneity measure values were calculated at the exact locations of the multi-temporal images (Zhuang et al., 2018). Second, the heterogeneity measure value is a weight that maintains a balance between suppressing noise and preserving detailed information (Zhuang et al., 2018). Thus, its desired dynamic range is [0,1]. However, when the standard deviation of the neighborhood area is higher than its mean, the heterogeneity measure is greater than 1. Third, the intensity of the changed pixels in the obtained DIs should be small, and the intensity of the unchanged pixels should be significant (Seo et al., 2018). However, the difference images computed by Seo et al. show greater intensity in the changing pixels. Finally, $D_{\mathit{NR}}$ is more suitable for SAR image CD. Based on this, this paper proposes a DI generation method based on neighborhood ratio combined with fused images, which improves the accuracy of self-supervised sample extraction and effectively suppresses the noise compared with the traditional DI algorithm. The flow of the $D_{\mathit{NR}-\mathit{CFI}}$ generation is shown in the Figure 2 as follows:

Figure 2. The process of difference image generation.

The improved $D_{\mathit{NR}-\mathit{CFI}}$ method can balance noise suppression and detailed information preservation by heterogeneity measures. Moreover, it can use fused images to retain detailed spectral, textural, and spatial information, which can mitigate the adverse influence of heterogeneous differences and accompanied noises and retain the detailed changed and unchanged information. Further, the proposed method generates two types of DI, the standard and exponential DIs. The improved $D_{\mathit{NR}-\mathit{CFI}}$ one can be expressed as equation (5).

(5) $\begin{array}{rl}& D_{\mathit{NR}-\mathit{CFI}(\mathit{m},\mathit{n})}=\mathit{\theta }\frac{min\left\{I_1(\mathit{m},\mathit{n})+F_3(\mathit{m},\mathit{n}),I_2{(\mathit{m},\mathit{n})}^{\mathrm{q}}\right\}}{\mathit{max}\left\{I_1(\mathit{m},\mathit{n})+F_3(\mathit{m},\mathit{n}),I_2{(\mathit{m},\mathit{n})}^{\mathrm{q}}\right\}}\\ +& \left(1-\mathit{\theta }\right)\frac{{\sum }_{\mathrm{x}\in \mathrm{\Omega }\wedge x\ne m,n}\mathit{min}\left\{I_1(\mathit{x})+F_3(\mathit{x}),I_2{(\mathit{x})}^{\mathrm{q}}\right\}}{{\sum }_{\mathit{x}\in \mathrm{\Omega }\wedge \mathit{x}\ne \mathit{m},\mathit{n}}\mathit{max}\left\{I_1(\mathit{x})+F_3(\mathit{x}),I_2{(\mathit{x})}^{\mathrm{q}}\right\}}\end{array}$(5)

Where, $F_3(\mathit{m},\mathit{n})$ indicates the fused image. When $\mathit{q}=1$ is used, a high neighborhood ratio value indicates that the pixel has changed and can be used to generate the standard DI. Conversely, When $\mathit{q}>1$ is employed, a high neighborhood ratio value indicates that the pixel is unchanged and can be utilized in the exponential DI. $I_1$, $I_2$, and $F_3$ are summed along the third dimension to extract data features, respectively, ensuring that the computation is in the same dimension. By adding the fused image, the standard DI can reduce the noise caused by the difference of the heterogeneous images and improve the extraction accuracy of the changed pixels. Exponential difference images reduce noise by exponentially amplifying the feature differences between heterogeneous images while ensuring the acquired unchanged pixels are correct. By extracting pixel-level features, not only the spectral values of each pixel are taken into account, but also the rich texture information and spatial context information (Seo et al., 2018).

The imaging mechanisms of heterogeneous images are different, and the physical meanings of the image element values and the number of channels are not the same, resulting in the spectral information and texture information are not comparable, so it is necessary to construct transitional images that are comparable to the original image. Therefore, this paper generates an excessive image by fusing the front and back time-phase images. The transitional images are spectrally similar to the spectral information of the original image, which is spectrally comparable. At the same time, the transitional images contain rich texture information, and the spectral values of the changed pixels change significantly due to the injection of texture information. Therefore, the transitional images constructed by fusing the before and after temporal phases can provide complementary information, which is not only comparable to the original image in terms of radiance spectrum, channel number, etc., but also can have a higher sensitivity to the change image elements, which can effectively assist in generating self-supervised samples and can perform more detailed and accurate CD.

The fusion method used in this paper is PC-SPCNN (Fu et al., 2023), which solves the severe spectral distortion and incomplete and inconspicuous feature injection in RS image CD methods. It exhibits exceptional resilience against nonlinear radiometric disparities and noise while preserving spectral and spatial information (Wu et al., 2021). The processing results of Figure 3(a) and (b) are shown in Figure 3(c). The generated spatiotemporal fused images have both detailed spectral and textural information. Thus, the proposed method that generates DIs based on heterogeneous images and fused images can reduce the influences of image differences and noise and improve the separability of changed and unchanged regions.

Figure 3. PC-SPCNN method fusion.

Bi-hierarchical FCM (BI-FCM)clustering algorithm

Based on the difference images, CD has high detection accuracy and fast computational speed, but the thresholding problem affects the CD accuracy. The number of change samples is small when extracting changed samples and unchanged samples using the hierarchical FCM clustering algorithm, and the extraction of unchanged samples is overly dependent on the threshold value (Chen et al., 2020; Gao et al., 2016; Geng et al., 2019; Hou et al., 2021; Zhan et al., 2018; Zheng et al., 2021). Therefore, in this paper, in order to select credible samples to train the support vector machine binary classification model, we use the Bi-hierarchical FCM clustering algorithm to complete the automatic extraction of changed and unchanged samples, and the steps of the algorithm are as follows:

1. The FCM algorithm is executed on the DI to divide the pixels into three clusters: ${\mathrm{\Omega }}_{\mathrm{c}}^1$,${\mathrm{\Omega }}_{\mathrm{i}}^1$ and ${\mathrm{\Omega }}_{\mathrm{u}}^1$, which represent changed pixels, intermediate pixels, and unchanged pixels, respectively. The number of pixels in ${\mathrm{\Omega }}_{\mathrm{c}}^1$ is denoted by $T^1$. The upper bound on the number of actual changed pixels is defined as TT =$\mathit{\sigma }\cdot T^1$, where the $\mathit{\sigma }$ is experimentally determined to be 1.2 (Gao et al., 2016).

2. Execute the FCM algorithm on the DI to divide the pixels into five clusters: ${\mathrm{\Omega }}_1^2$, ${\mathrm{\Omega }}_2^2$, ${\mathrm{\Omega }}_3^2$, ${\mathrm{\Omega }}_4^2$and ${\mathrm{\Omega }}_5^2$, which are arranged in descending order based on their mean value. Among them, the mean value of pixels ${\mathrm{\Omega }}_1^2$ is the largest, and the mean value ${\mathrm{\Omega }}_5^2$ is the smallest. A cluster with a higher mean value indicates a higher change probability. The number of pixels in the five clusters is $T_1^2$, $T_2^2$, $T_3^2$, $T_4^2$ and $T_5^2$. Set t = 1, c=$T_1^2$, and assign the pixels ${\mathrm{\Omega }}_1^2$ to the class ${\mathrm{\Omega }}_c$ (Gao et al., 2016).

3. Set t = t + 1 and c = c +$T_{\mathrm{t}}^2$(Gao et al., 2016).

4. If c < TT, assign the pixels in ${\mathrm{\Omega }}_{\mathrm{t}}^2$ to ${\mathrm{\Omega }}_i$. Otherwise, assign the pixels in ${\mathrm{\Omega }}_{\mathrm{t}}^2$ to ${\mathrm{\Omega }}_u$. Execute step 3 until t = 5(Gao et al., 2016).

5. Output pre-classification results are composed of labels {${\mathrm{\Omega }}_c$, ${\mathrm{\Omega }}_i$, ${\mathrm{\Omega }}_u$} (Gao et al., 2016).

6. The non-zero element of the classification result is set to 1 to obtain the binarization matrix, where 1 denotes the clustered region, and 0 denotes the other regions.

7. The binarized matrix is labeled to obtain the label matrix and the number of labels.

8. Remove small areas of clustering results by traversing each label.

9. An image showing the changed pixels is obtained by labeling the regions with a clustering result value of 1. Repeat the above steps to obtain an image showing unchanged pixels.

10. 10) Combine the changed pixel and unchanged pixel regions, where the unchanged pixel is labeled 0, the change pixel is labeled 1, and the intermediate pixel is labeled 0.5.

In the standard DI, the intensity value of the changed pixels is significant, and the intensity value of the unchanged pixels is small. In the exponential DI, the intensity value of the unchanged pixels is significant, and the intensity value of the changed pixels is small. The accurately changed pixels and unchanged pixels are obtained by the DI and exponential DI, respectively. This avoids the problem of a small number of change samples and poor accuracy of unchanged samples. Changed and unchanged sample extraction are independent and have no requirement for the magnitude of the pixel values, and it does not involve problems related to determining the double threshold value.

Unlike the hierarchical FCM clustering algorithm, step (6) increases the number of changed and unchanged samples, and step (8) improves the accuracy of the extraction of changed and unchanged samples. The extracted changed pixels and unchanged pixels using the bi-hierarchical FCM clustering algorithm are completely correct, and the results are shown in Figure 4 (a) and (b). The obtained sample set is shown in Figure 4 (c), with yellow pixels referring to changed pixels (${\mathrm{\Omega }}_c$) and blue pixels representing unchanged pixels (${\mathrm{\Omega }}_u$). These changed and unchanged pixels are selected as training samples. The cyan pixels in (${\mathrm{\Omega }}_i$) represent the intermediate pixels, and their changed and unchanged states are determined by the SVM classifier. The acquired samples were divided into training and test sets, comprising 65% and 35% total samples, respectively.

Figure 4. Sample data.

Change detection

CD can be defined as a binary classification problem involving categorizing pixels into changed and unchanged pixels. Compared to state-of-the-art neural network methods, choosing SVM as a classifier has low requirements for training samples, hardware and software. In this process, the appropriate feature selection plays a pivotal role (Asokan & Anitha, 2019; Lantzanakis et al., 2020; Shirani et al., 2023). This paper uses information from fused images in the SVM model to increase the feature dimensionality and detect the correct change information that cannot be detected from original heterogeneous images. Cross-validation techniques are employed to select optimal hyperparameters to maximize the performance of SVM models.

Experimental datasets and evaluation metrics

Experimental datasets

This paper uses three sets of heterogeneous images to validate fused images’ impact on DI generation and sample extraction accuracy, as shown in Figure 5. The detailed information on datasets is shown in Table 1 (Longbotham et al., 2012; Mignotte, 2020). Dataset 1 is from Google Earth and Landsat-5 sensors (Sun et al., 2021). Dataset 2 is from GF1 and GF2 sensors. Dataset 3 is from TerraSAR-X and QuickBird2 sensors (Geng et al., 2019). All datasets are resampled, registered, and cropped to cover the same geographical area (Geng et al., 2019).

Figure 5. Heterogeneous image dataset.

Table 1. Data set information.

Data	Time	Spatial resolution	Sensors	Location	Size(pixels)	Event
1	Sept.1995 July.1996	≈30 m 30 m	Landsat-5 Google Earth	Sardinia，Italy	412 × 300	Lake expansion
2	July.2016 Apr.2018	2 m 1 m	GF1 GF2	Lanzhou City，China	412 × 300	construction
3	July.2006 July.2007	0.61 m 1 m	QuickBird2 TerraSAR- X	UK	802 × 500	Flooding

Evaluation metrics

This paper adopts the F1-measure (Fm), Kappa coefficient (Kappa coefficient, Kc), and PCC index to quantitatively evaluate the performance of the method (Agapiou, 2020). The corresponding measurement equations are expressed as equation (6), (7), and (8) (Gao et al., 2016; Xuan et al., 2022).

(6) $\mathit{Fm}=(2\times \mathit{precision}\times \mathit{recall})/(\mathit{precision}+\mathit{recall})$(6)

(7) $\mathit{Kc}=(\mathit{OA}-\mathit{PRE})/(1-\mathit{PRE})$(7)

(8) $\mathit{PCC}=(\mathit{N}\mathrm{u}+\mathit{N}\mathrm{c}-\mathit{FP}-\mathit{FN})/(\mathit{N}\mathrm{u}+\mathit{N}\mathrm{c})\times 100\mathrm{\%}$(8)

Fm represents the comprehensive evaluation metric, simultaneously integrating precision and recall. Meanwhile, the Kappa coefficient is utilized to assess consistency (Pan et al., 2022). PCC denotes the classification correctness rate, while Nu and Nc denote the number of unchanged and changed actual pixels. The formulas for each metric are as follows: (9), (10), (11), (12), where TP, FP, TN, and FN stand for True Positives, False Positives, True Negatives, and False Negatives, respectively (Sun, Lei, Guan, Wu, et al., 2022; Xuan et al., 2022).

(9) $\mathit{recall}=\mathit{TP}/(\mathit{TP}+\mathit{FN})$(9)

(10) $\mathit{precision}=\mathit{TP}/(\mathit{TP}+\mathit{FP})$(10)

(11) $\mathit{OA}=(\mathit{TP}+\mathit{TN})/(\mathit{TP}+\mathit{TN}+\mathit{FP}+\mathit{FN})$(11)

(12) $\begin{array}{rl}& \mathit{PRE}=(\mathit{TP}+\mathit{FN})(\mathit{TP}+\mathit{FP})+\\ & (\mathit{TN}+\mathit{FP})(\mathit{TN}+\mathit{FN})(\mathit{TP}+\mathit{TN}+\mathit{FP}+\mathit{FN})\end{array}$(12)

Experimental results and evaluation

Change detection based on different difference images

The DIs of datasets I, II, and III are depicted in Figures 7, 8 , and 9, respectively, where the value of changed pixels is higher than that of unchanged pixels in the standard difference images (DI₁-T₁/T₂, DI₂-F₃/T₂, DI₃-(T₁+F₃)/T₂). In contrast, the value of unchanged pixels is higher than that of changed pixels in the exponential DI (DI₄-T₁/T₂^1.5, DI₅-F₃/T₂^1.5, DI₆-(T₁+F₃)/T₂^1.5, DI₇-T₁/T₂², DI₈-F₃/T₂², DI₉-(T₁+F₃)/T₂²). The CD results vary with different DIs. The CD accuracies for datasets I, II, and III are presented in Tables 2, 3, and 4, respectively, and the ROC curves are shown in Figure 6(a), (b), and (c), respectively.

Figure 6. ROC curves for CD in different datasets.

Figure 7. Different DI for experiment 1.

Figure 8. Different DI for experiment 2.

Figure 9. Different DI for experiment 3.

Table 2. Data 1 Change detection accuracy of different difference images.

Experiment	Sample changes	Unchanged sample	FA rate	MD rate	PCC	Kappa
1	($T_1$,$T_2$)	($T_1$+$F_3$,$T_2^{1.5}$)	0.0261	0.0091	0.9648	0.7308
2	($F_3$,$T_2$)	($T_1$+$F_3$,$T_2^{1.5}$)	0.0212	0.0106	0.9682	0.7457
3	($T_1+F_3$,$T_2$)	($T_1$+$F_3$,$T_2^{1.5}$)	0.0085	0.0135	0.9779	0.8020
4	($T_1+F_3$,$T_2$)	($T_1$,$T_2^{1.5}$)	0.0099	0.0127	0.9774	0.8006
5	($T_1+F_3$,$T_2$)	($F_3$,$T_2^{1.5}$)	0.0319	0.0079	0.9602	0.7088
6	($T_1+F_3$,$T_2$)	($T_1+F_3$,$T_2^2$)	0.0083	0.0142	0.9776	0.7972
7	($T_1+F_3$,$T_2$)	($T_1$,$T_2^2$)	0.0099	0.0128	0.9773	0.7992
8	($T_1+F_3$,$T_2$)	($F_3$,$T_2^2$)	0.0220	0.0093	0.9688	0.7541

Table 3. Data 2 Change detection accuracy of different difference images.

Experiment	Sample changes	Unchanged sample	FA rate	MD rate	PCC	Kappa
1	($T_1$,$T_2$)	($T_1$+$F_3$,$T_2^{1.5}$)	0.0768	0.0895	0.8337	0.6411
2	($F_3$,$T_2$)	($T_1$+$F_3$,$T_2^{1.5}$)	0.0499	0.0785	0.8716	0.7204
3	($T_1+F_3$,$T_2$)	($T_1$+$F_3$,$T_2^{1.5}$)	0.0671	0.0458	0.8871	0.7608
4	($T_1+F_3$,$T_2$)	($T_1$,$T_2^{1.5}$)	0.0187	0.1650	0.8163	0.5717
5	($T_1+F_3$,$T_2$)	($F_3$,$T_2^{1.5}$)	0.1598	0.0207	0.8195	0.6408
6	($T_1+F_3$,$T_2$)	($T_1+F_3$,$T_2^2$)	0.1222	0.0607	0.8170	0.6208
7	($T_1+F_3$,$T_2$)	($T_1$,$T_2^2$)	0.0323	0.2025	0.7653	0.4447
8	($T_1+F_3$,$T_2$)	($F_3$,$T_2^2$)	0.1039	0.0747	0.8214	0.6234

Table 4. Data 3 different difference image change detection accuracy.

Experiment	Sample changes	Unchanged sample	FA rate	MD rate	PCC	Kappa
1	($T_1$,$T_2$)	($T_1$+$F_3$,$T_2^{1.5}$)	0.0077	0.0698	0.9225	0.7913
2	($F_3$,$T_2$)	($T_1$+$F_3$,$T_2^{1.5}$)	0.0135	0.0663	0.9202	0.7875
3	($T_1+F_3$,$T_2$)	($T_1$+$F_3$,$T_2^{1.5}$)	0.0125	0.0606	0.9269	0.8063
4	($T_1+F_3$,$T_2$)	($T_1$,$T_2^{1.5}$)	0.0291	0.0542	0.9167	0.7853
5	($T_1+F_3$,$T_2$)	($F_3$,$T_2^{1.5}$)	0.0168	0.0610	0.9222	0.7949
6	($T_1+F_3$,$T_2$)	($T_1+F_3$,$T_2^2$)	0.0061	0.0733	0.9206	0.7848
7	($T_1+F_3$,$T_2$)	($T_1$,$T_2^2$)	0.0057	0.0746	0.9196	0.7818
8	($T_1+F_3$,$T_2$)	($F_3$,$T_2^2$)	0.0060	0.0746	0.9194	0.7811

Compared to experiments 1 and 2 of dataset 1, the Kappa coefficient of experiment 3 is 5.63 and 7.12% higher. The remote sensing images of dataset 1 have high spectral variability, which leads to many errors in changed sample extraction based on the DI₁ and DI₂. Comparatively, the fusion of the heterogeneous images significantly reduces the spectral disparities. The accuracy of actual changed pixel detection is highest based on the DI₃-(T₁+F₃)/T₂, and thus the corresponding Kappa coefficient reached 0.8020. Compared with experiments 1 and 2 of dataset 2, the Kappa coefficient of experiment 3 is improved by 4.04% and 11.97%, respectively. The remote sensing images in dataset 2 have significant differences in texture information, resulting in elevated errors in changed sample extraction based on the DI₁ and DI₂. By adding spectral and textural information, the fused images exhibit high homogeneity so that the adverse effect of texture differences on CD is weakened based on the DI₃-(T₁+F₃)/T₂. Thus, the actual changed pixels are detected with the highest accuracy. Compared to experiments 1 and 2 of dataset 3, the Kappa coefficient of experiment 3 is 1.5 and 1.88% higher. The remote sensing images of dataset 3 have high spectral variability, which leads to many errors in changed sample extraction based on the DI₁ and DI₂. Comparatively, the fusion of the heterogeneous images significantly reduces the spectral disparities.

Compared with experiments 4, 5, 6, 7, 8, and 9 on Dataset 1, the Kappa coefficients of experiment 3 are improved by 0.14–9.32%. Dataset 1 achieves good CD on DI₄, DI₆, DI₇, and DI₈, which can accurately identify the actual unchanged pixels. Particularly, the Kappa coefficient on DI₆ is as high as 0.8020, which retains the completely changed and unchanged detail information. Compared with experiments 4, 5, 6, 7, 8, and 9 of dataset 2, the Kappa coefficients of experiment 3 are improved by 12–31.61%. At the same time, the range of pixel values of the DI₄, DI₅, DI₇, DI₈, and DI₉ is small, which can easily cause confusion between the actual changed and the unchanged pixels. In contrast, the DI₆-(T₁+F₃)/T₂^1.5 value range is extensive, leading to good CD accuracy. Experiments on Dataset 3 achieve good CD accuracy on six exponential DIs, and compared to other experiments, the Kappa coefficients of experiment 3 are improved by 0.74–2.26%. In particular, the range of values in the DI₆-(T₁+F₃)/T₂^1.5 is the most extensive and facilitates the distinction between actually changed and unchanged pixels.

Therefore, the standard DI₃-(T₁+F₃)/T₂ can obtain high-quality DI and improve the accuracy of changed sample extraction, and the exponential DI₆-(T₁+F₃)/T₂^1.5, which ensures the accuracy of unchanged sample extraction.

To quantitatively analyze the quality of DIs, we use the Davies-Bouldin index based on intra-cluster distance and inter-cluster distance, which can measure the tightness and separation of clusters(Wu & Chow, 2004). A smaller intra-cluster distance for DIs indicates better tightness, and a large inter-cluster distance between changed and unchanged pixels indicates better separation. Thus, a small Davies-Bouldin index indicates the preferred clustering result with good tightness and separation. According to the Davies-Bouldin validity index, optimal clustering minimizes (Roy et al., 2023; Wu & Chow, 2004)

(13) $\frac{1}{C}\sum _{\mathrm{k}=1}^C\underset{\mathit{l}\ne \mathit{k}}{max}\{\frac{S_c\left(Q_k\right)+S_c\left(Q_l\right)}{d_{\mathit{ce}}\left(Q_k,Q_l\right)}\}$(13)

Where C is the number of clusters, Sc is the average of the distances between all samples in cluster k and its cluster center, which measures the clusters’ closeness, and $d_{\mathit{ce}}$ is the distance between cluster k and cluster j centers, which measures the degree of separation between the clusters (Baik et al., 2001).

Figure 10 represents the mean and Davies-Bouldin index for different DI changed pixels and unchanged pixels, and the outcomes are depicted using error bars, with the results for datasets I, II, and III displayed in the first, second, and third columns, respectively. The central point of the error bars denotes the pixel’s mean value, while the length of the bars represents the Davies-Bouldin index. Shorter error bars indicate better tightness and separation of changed and unchanged pixels, and vice versa.

Figure 10. Error bars for different DIs.

Comparing the different DIs in Experiment 1, in Figure 10 (a), the mean value of DI₃ is more significant than that of DI₁ and DI₂, which is favorable for extracting changed pixels. The length of its error bars is also shorter, indicating that the changed pixels are more tightly clustered and better separated from the unchanged pixels, which is favorable for the dividend of the changed and unchanged pixels. The mean value of DI₇ and DI₈ is smaller, which is favorable for extracting changed pixels. However, the length of the error bars is longer, which is unfavorable for the distinction between changed and unchanged pixels. In Figure 10 (d), the mean values of DI₁, DI₂, and DI₃ are all larger, which is unfavorable for the extraction of unchanged pixels, comparing the mean values of DI₄, DI₅, DI₆, DI₇, DI₈, and DI₉, the values of DI₄ and DI₆ are more significant, which is favorable for the extraction of unchanged pixels. The length of the error bars of DI₆ is shorter than that of DI₄, indicating that the tightness of unchanged pixels in DI₆ is better. The separation of unchanged and changed pixels is better, which is favorable for the difference between unchanged pixels and other pixels’ distinction.

Comparing the different DIs in Experiment 2, in Figure 10 (b), the mean value of DI₃ is more significant than that of DI₁ and DI₂, which is favorable for the extraction of changed pixels, and the length of the error bars is shorter, indicating that the changed pixels are more tightly clustered and better separated from the unchanged pixels. The mean values of DI₇, DI₈, and DI₉ are smaller, which is favorable for extracting changed pixels. However, the length of the error bars is longer, which is unfavorable in distinguishing changed pixels from unchanged pixels. In Figure 10 (e), the mean value of DI₂ is smaller than that of DI₁ and DI₃, which is favorable for extracting the unchanged pixels. However, the length of the error bars is longer, which is unfavorable for the distinction between unchanged and changed pixels. Among DI₄, DI₅, DI₆, DI₇, DI₈, and DI₉, the mean value of DI₆ is the largest, and its error bar is the shortest, which is favorable for the distinction between changed and unchanged pixels.

Comparing the different DIs of Experiment 3, in Figure 10 (c), the situation of DI values is similar to Figure 10 (a), in Figure 10 (f), the situation of DI values is similar to Figure 10 (d). In summary, through the quantitative analysis of DI plots, it is further verified that the standard DI₃-(T₁+F₃)/T₂ is favorable for the extraction of changed samples, and the exponential DI₆-(T₁+F₃)/T₂^1.5 is favorable for the extraction of unchanged samples. Therefore, this paper uses DI₃-(T₁+F₃)/T₂ for changed sample extraction, and DI₆-(T₁+F₃)/T₂^1.5 for unchanged sample extraction.

CD under different fusion methods

This subsection compares three methods based on different image fusion algorithms to analyze the effect of fusion methods on the CD results, including HCS(Hao et al., 2019), PCA (Wang et al., 2005), and PC-SPCNN (Fu et al., 2023). The CD results of the three datasets are shown in Figure 11, where red represents the (miss detection) MD pixels, orange represents the (false alarm) FA pixels, and blue represents the correct CD pixels. The accuracy results are shown in Tables 5, 6, and 7, respectively. The PC-SPCNN-based method maintains a more stable CD accuracy. Therefore, the PC-SPCNN algorithm is used for image fusion in this paper.

Figure 11. Detection results of changes in different fusion methods.

Table 5. Change detection accuracy of different fusion methods for Data 1.

Method	FA rate	MD rate	PCC	Kappa	F1
HCS	0.1322	0.0067	0.8611	0.3848	0.4419
PCA	0.1006	0.0069	0.8925	0.4570	0.5050
PC-SPCNN	0.0085	0.0135	0.9779	0.8020	0.8137

Table 6. Data 2 different fusion methods change detection accuracy.

Method	FA rate	MD rate	PCC	Kappa	F1
HCS	0.0333	0.1087	0.8580	0.6824	0.7869
PCA	0.0702	0.0543	0.8756	0.7356	0.8357
PC-SPCNN	0.0671	0.0458	0.8871	0.7608	0.8520

Table 7. Data 3 different fusion methods change detection accuracy.

Method	FA rate	MD rate	PCC	Kappa	F1
HCS	0.0153	0.0606	0.9241	0.7997	0.8500
PCA	0.0050	0.0807	0.9144	0.7655	0.8198
PC-SPCNN	0.0125	0.0606	0.9269	0.8063	0.8546

Comparison of different CD methods

To validate the efficacy of SVM in a self-supervised CD method based on a difference algorithm, five competing classifiers, including k-Nearest Neighbor (KNN)(Liu & Chen, 2022), random forest (RF) (Mastro et al., 2022), fuzzy C means-sensitive Bayesian network (FCM-SBN) (Li et al., 2023), FITENSEMBLE (Matikainen et al., 2016), and extreme learning machine (ELM) (Gao et al., 2016), are employed for comparison. The method was validated on three heterogeneous image datasets, where the same training samples were fed into the five classifiers for comparison. The results are shown in Figure 12. In addition, the method proposed in this paper is compared with other existing CD methods, including neighborhood-based ratio and extreme learning machine (NR_ELM) (Gao et al., 2016), Sparse-constrained adaptive structure consistency (SCASC) (Sun et al., 2021), spectral domain analysis (SDA) (Sun et al., 2022), iterative robust graph and Markovian co-segmentation (IRG-McS) (Sunet al., 2021), Structural regression fusion (SRF) (Sun et al., 2023), and deep learning-based graph convolutional network and metric learning (GMCD) (Tang et al., 2021), structural relationship graph convolutional autoencoder (SRGCAE) (H. Chen et al., 2022) and convolutional-wavelet neural networks (CWNN)(Gao et al., 2019) method, and Figure 13 shows the results of CD.

Figure 12. Change detection results of different classifier methods for three heterogeneous image datasets.

Figure 13. Change detection results of different method changes of three groups of heterogeneous images.

The KNN method updates the changed and unchanged pixel labels based on contextual information (Liu & Chen, 2022). The RF method is based on the classification and regression tree (generalized likelihood ratio test, CART) algorithm without any assumptions of the data frequency distribution (Mastro et al., 2022). The FCM-SBN uses the FCM algorithm to decompose the pixel into several signal classes with different fuzzy memberships and utilize SBN to estimate the pixel-wise posterior probability vector for CD (Li et al., 2023). FITENSEMBLE accomplishes change and un-change classification by constructing a classification tree set (Matikainen et al., 2016). As a CD classifier, ELM can adapt to different conditions and capture the nonlinear relationships of multi-temporal images (Gao et al., 2016). NR-ELM uses NR to obtain the changed or unchanged pixels with high probabilities and classify the original image by the ELM model (Gao et al., 2016). SCASC method performs CD based on structural consistency by comparing the structure of two images rather than their pixel values (Sun et al., 2021). SDA decomposes the source signal into a spectrally constrained regression signal and a change signal to fulfill the CD tasks (Sun et al., 2022). GMCD is an unsupervised CD method leveraging graph convolutional networks (GCN) and metric learning (Tang et al., 2021). The Structural Relational Graph Convolutional Autoencoder (SR-GCAE) learns robust and representative features from graphs for multimodal remote sensing image CD (Chen et al., 2022). In CWNN, the dual-tree complex wavelet transform is incorporated into a convolutional neural network to classify changed and unchanged pixels (Gao et al., 2019).

Examples of change detection

Figure 12, 13 shows the detection results for Data 1: NR-CFI-BI-FCM-FCM-SBN and NR-ELM significantly overestimate the change region since the FCM-SBN and ELM classifiers have significant detection errors for changed and unchanged pixels. The SCASC method utilizes graphical extraction to obtain the structural information of the image. However, this method underestimates the change region and needs to improve the DI obtained from the regression process. On the other hand, the SDA method decomposes the source signal into the regression signal and the change signal, which restricts the spectral characteristics of the regression signal, underestimates the region of change, and needs to improve its DI. Similarly, IRG-McS underestimates changed regions, and SRF overestimates changed regions. GMCD is a multiscale decision fusion used to integrate DIs, which overestimates the change region. SRGCAE is a DI fusion strategy based on change intensity variance, which overestimates the change region. Similarly, CWNN overestimates the changed area. By training high-quality DIs, NR-CFI-BI-FCM-KNN, NR-CFI-BI-FCM-RF, NR-CFI-BI-FCM-FITENSEMBLE, NR-CFI-BI-FCM-ELM, and NR-CFI-BI-FCM-SVM obtain more accurate CD results with relatively few overestimated change regions. In contrast, NR-CFI-BI-FCM-SVM has the smallest overestimated change regions with clear boundaries under the SVM classification model due to the combination of detailed information from the fused images.

Due to many shadows, the alignment accuracy of Data II is poor, resulting in many CD errors. Figure 12、13 shows the detection results of Data II: NR-CFI-BI-FCM-FCM-SBN, NR-CFI-BI-FCM-FITENSEMBLE, and NR-CFI-BI-FCM-ELM severely overestimated the change regions, indicating that the FCM-SBN, FITENSEMBLE, and ELM classifiers detected a large number of incorrect changed and unchanged pixels. NR-ELM, SCASC, SDA, IRG-McS, SRF, GMCD, and SRGCAE severely underestimated the change region, indicating that the DIs obtained by these methods need to be further improved. The CWNN has more areas for false detection and miss detection. NR-CFI-BI-FCM-KNN, NR-CFI-BI-FCM-RF, and NR-CFI-BI-FCM-SVM obtain more accurate CD results by training high-quality DIs, but the region of change is slightly underestimated. In contrast, NR-CFI-BI-FCM-SVM committed the slightest underestimation of the change region.

Figure 12, 13 shows the detection results for Data III. Compared with NR-CFI-BI-FCM-ELM, the detection error of NR-ELM is evident, which verifies the reliability of high-quality DI in heterogeneous image CD. In addition, the classification results based on NR-CFI-BI-FCM-KNN, NR-CFI-BI-FCM-RF, NR-CFI-BI-FCM-FCM-SBN, NR-CFI-BI-FCM-FITENSEMBLE, and NR-CFI-BI-FCM-SVM are accurate. In contrast, SCASC, SDA, IRG-McS, SRF, GMCD, SRGCAE, and CWNN methods have significant errors in detecting changed and unchanged pixels.

Quantitative analysis of change

The CD accuracy for Data I is presented in Tables 8 and 9. Compared with the NR_ELM method, the FA rate of the NR-CFI-BI-FCM-ELM method is reduced by 0.0232, and the Kappa coefficient is improved by 0.1391, which verifies the superior performance of $D_{\mathit{NR}-\mathit{CFI}}$ in heterogeneous image CD. Similarly, $D_{\mathit{NR}-\mathit{CFI}}$ achieves promising results based on KNN, RF, FITENSEMBLE, and SVM classifiers. In particular, the SVM-based method obtained the highest Kappa coefficient of 0.8020 with a decreased FA rate compared to the other classifiers. The MD rate of NR-CFI-BI-FCM-FCM-SBN is lower than that of NR-CFI-BI-FCM-SVM, while the FA rate is improved by 0.0440 by establishing a many-to-many relationship between the signal class and the ground cover types. Compared to NR-CFI-BI-FCM-SVM, SCASC, SDA, and IRG-McS have lower FA rates. However, these methods confuse many changed pixels with unchanged pixels, which leads to higher MD rates. Further, the SRF, GMCD, and SRGCAE methods have higher FA rates and lower MD rates. In contrast, the method proposed in this paper has a low FA rate and an MD rate of 0.0135 and obtains better Kappa, F1, and PCC values than all other competing methods.

Table 8. Change detection accuracy of different classifier methods for Data 1.

Method	FA rate	MD rate	PCC	Kappa	F1
NR-CFI-Bi-FCM-KNN	0.0098	0.0130	0.9771	0.7975	0.8097
NR-CFI-Bi-FCM -RF	0.0085	0.0162	0.9752	0.7728	0.7859
NR-CFI-Bi-FCM -FCM-SBN	0.0525	0.0064	0.9411	0.6229	0.6525
NR-CFI-Bi-FCM -FITENSEMBLE	0.0119	0.0125	0.9757	0.7886	0.8020
NR-CFI-Bi-FCM -ELM	0.0115	0.0129	0.9756	0.7871	0.8000
NR-CFI-Bi-FCM -SVM	0.0085	0.0135	0.9779	0.8020	0.8137

Table 9. Change detection accuracy of different methods for Data 1.

Method	FA rate	MD rate	PCC	Kappa	F1
NR_ELM	0.0347	0.0128	0.9525	0.6480	0.6730
SCASC	0.0074	0.0236	0.9690	0.6952	0.7111
SDA-HCD	0.0072	0.0230	0.9698	0.7035	0.7191
IRG-McS	0.0069	0.0218	0.9714	0.7212	0.7361
SRF	0.0172	0.0106	0.9722	0.7711	0.7860
GMCD	0.0470	0.0187	0.9343	0.5325	0.5668
SRGCAE	0.1029	0.0079	0.8892	0.4435	0.4928
CWNN	0.0338	0.0127	0.9536	0.6541	0.6786
NR-CFI-Bi-FCM -SVM	0.0085	0.0135	0.9779	0.8020	0.8137

The CD accuracy for Data II is presented in Tables 10 and 11. Compared with the NR_ELM method, the FA rate of the NR-CFI-BI-FCM-ELM method is increased by 0.0467, the MD rate is reduced by 0.2325, and the Kappa coefficient is improved by 0.5210, respectively, which verified the reliability of the improved $D_{\mathit{NR}-\mathit{CFI}}$ in heterogeneous image CD. Similarly, high-quality DIs obtained accurate CD results on KNN, RF, FITENSEMBLE, and SVM classifiers. At the same time, the Kappa coefficient ranked first, and the MD rate was the lowest at 0.0458 when using SVM classifiers for CD. Compared with the NR-CFI-BI-FCM-SVM method, the FA and MD rates of the NR-CFI-BI-FCM-FCM-SBN are increased by 0.1180 and 0.0573, respectively. Compared with the NR-CFI-BI-FCM-SVM method, the SCASC, SDA, IRG-McS, SRF, GMCD, and SRGCAE methods decreased the FA rate by 0.0525, 0.0490, 0.0167, 0.0512, 0.0465, and 0.0118, respectively, but increased the MD rate by 0.2688, 0.1642, 0.2468, 0.2486, 0.2013, and 0.2473, respectively. In contrast, the CWNN method increased the FA and the MD rates by 0.0893 and 0.1529, respectively. From Tables 10 and 11, it can be observed that the CD FA rates for Data II are relatively high due to the shadows, which result in poor alignment accuracy. However, the proposed method’s final Kappa, F1, and PCC values are the highest.

Table 10. Change detection accuracy of different classifier methods for Data 2.

Method	FA rate	MD rate	PCC	Kappa	F1
NR-CFI-Bi-FCM -KNN	0.0640	0.0520	0.8840	0.7531	0.8462
NR-CFI-Bi-FCM -RF	0.0617	0.0711	0.8672	0.7140	0.8187
NR-CFI-Bi-FCM -FCM-SBN	0.1851	0.1031	0.7118	0.4092	0.6501
NR-CFI-BI-FCM-FITENSEMBLE	0.0986	0.0526	0.8488	0.6841	0.8081
NR-CFI-Bi-FCM -ELM	0.0569	0.0643	0.8788	0.7392	0.8349
NR-CFI-Bi-FCM -SVM	0.0671	0.0458	0.8871	0.7608	0.8520

Table 11. Change detection accuracy of different methods for Data 2.

Method	FA rate	MD rate	PCC	Kappa	F1
NR_ELM	0.0102	0.2968	0.6930	0.2182	0.3256
SCASC	0.0146	0.3146	0.6708	0.1542	0.2549
SDA-HCD	0.0181	0.2110	0.7709	0.4505	0.5827
IRG-McS	0.0504	0.2926	0.6570	0.1512	0.3134
SRF	0.0159	0.2944	0.6896	0.2137	0.3300
GMCD	0.0206	0.2471	0.7323	0.3441	0.4805
SRGCAE	0.0553	0.2931	0.6516	0.1402	0.3087
CWNN	0.1564	0.1987	0.6448	0.2207	0.4922
NR-CFI-Bi-FCM -SVM	0.0671	0.0458	0.8871	0.7608	0.8520

The CD accuracy for Data III is presented in Tables 12 and 13. Compared with the NR_ELM method, the MD rate of the NR-CFI-BI-FCM-ELM method is increased by 0.0589, and the FA rate is reduced by 0.2714, which evidenced the reliability of the improved $D_{\mathit{NR}-\mathit{CFI}}$ in the heterogeneous images CD. Similarly, the KNN, RF, FCM-SBN, FITENSEMBLE, and SVM classifiers also achieved satisfactory results. Among them, the SVM-based CD method obtained a Kappa coefficient of 0.8063 and a low MD rate of 0.0606. Compared with the NR-CFI-BI-FCM-SVM method, the SCASC, IRG-McS, SRF, GMCD, SRGCAE, and CWNN methods increased the FA rate by 0.0497, 0.0896, 0.0064, 0.0575, 0.1230 and 0.0576, and the MD rate by 0.0286, 0.1461, 0.0529, 0.1484, 0.0741 and 0.0317, respectively, and compared with the NR-CFI-BI-FCM-SVM method, the SDA method decreased the MD rate by 0.0111 but increased the FA rate by 0.0774. In contrast, the proposed method demonstrated superiority over other competitors in terms of Kappa value, F1 value, and PCC value.

Table 12. Change detection accuracy of different classifier methods for Data 3.

Method	FA rate	MD rate	PCC	Kappa	F1
NR-CFI-Bi-FCM -KNN	0.0101	0.0636	0.9263	0.8037	0.8520
NR-CFI-Bi-FCM -RF	0.0075	0.0674	0.9251	0.7987	0.8470
NR-CFI-Bi-FCM -FCM-SBN	0.0035	0.0925	0.9040	0.6984	0.7920
NR-CFI-Bi-FCM -FITENSEMBLE	0.0099	0.0638	0.9264	0.8037	0.8519
NR-CFI-Bi-FCM -ELM	0.0041	0.0885	0.9075	0.7438	0.8020
NR-CFI-Bi-FCM -SVM	0.0125	0.0606	0.9269	0.8063	0.8546

Table 13. Change detection accuracy of different methods for Data 3.

Method	FA rate	MD rate	PCC	Kappa	F1
NR_ELM	0.2755	0.0296	0.6949	0.5775	0.5022
SCASC	0.0622	0.0892	0.8485	0.6087	0.7109
SDA-HCD	0.0899	0.0495	0.8606	0.6660	0.7643
IRG-McS	0.1021	0.2067	0.6912	0.1165	0.2933
SRF	0.0189	0.1135	0.8676	0.6287	0.7098
GMCD	0.0700	0.2090	0.7210	0.1716	0.3228
SRGCAE	0.1355	0.1347	0.7299	0.3239	0.5104
CWNN	0.0701	0.0923	0.8376	0.5827	0.6928
NR-CFI-Bi-FCM -SVM	0.0125	0.0606	0.9269	0.8063	0.8546

In conclusion, the improved $D_{\mathit{NR}-\mathit{CFI}}$ performs better in heterogeneous image CD. Compared with various classifiers, the SVM-based CD model utilized in this proposed method exhibits the highest level of stability. Furthermore, compared with competitive approaches, our method demonstrates the lowest instances of false and missed detections and achieves the highest Kappa coefficient across all three datasets.

Conclusion

This paper proposes a self-supervised change detection method based on a difference algorithm to overcome the problem of poor quality of the DI caused by heterogeneous image differences and noises. Although existing change detection methods also adopt the fusion idea, the difference in this paper is that the fused data is used to suppress the differences and noise of the heterogeneous images, which improves the neighborhood ratio difference algorithm. In order to reduce the labor and material cost of sample extraction, we improved the hierarchical FCM clustering algorithm. Unlike the existing sample extraction method, the changed sample and the unchanged sample extraction are independent in this paper. The standard difference image and the exponential difference image are used respectively to ensure the quality of the sample extraction and to avoid the problem of the accuracy of the change detection being dependent on the threshold value. Change detection is accomplished using the improved neighborhood ratio difference image method, which improves change detection accuracy. This process realizes self-supervised sample extraction and improves the efficiency of change detection. Experimenting with heterogeneous image datasets from three distinct scenes confirms the effectiveness of the proposed method.

Our approach exhibits high accuracy and stability compared to up-to-date CD methods. The ablation experiments of competitive classifiers also justify our method’s adoption of the SVM classifier. In addition, experiments have confirmed that adding fused images without auxiliary hyperspectral and high spatial resolution image data can generate high-quality DIs, which can be used to extract high-quality samples. Furthermore, the clustering method used does not overly depend on the thresholds.

However, the proposed method has two limitations that must be addressed in future work. First, the chosen parameters for calculating the exponential DI may not be optimal, and an iterative framework will be used to improve the CD accuracy. Second, when working with multi-source remote sensing images with complex spectral and textural features, the fusion method employed in this study might not yield a consistently stable outcome. Therefore, the fusion method has to be improved to enhance the adaptability of the proposed method.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by the National Key R&D Program of China (Grant No. 2022YFB3903604), the Key Research and Development Project of Lanzhou Jiao tong University (LZJTU-ZD YF2301), the Central Guided Local Science and Technology Development Funds Project (No. 22ZY1QA005), and the National Natural Science Foundation of China (No. 42161069).