Remotely sensed images and GIS data fusion for automatic change detection

Abstract

During the past years, researchers have put forward a large number of change detection techniques for using remotely sensed images and have summarised them from different viewpoints. However, most research has mainly focused on images versus images and 2D change detection; the detection results have been imprecise especially as altitude changes cannot be detected because of the lack of data. As multi-source data can be acquired more and more easily, change detection with multi-source data has become a hot issue. Because change detection with multi-source data can eliminate the effects of the atmosphere and topography and improve the ability to identify and extract objects, more accurate results can be obtained from change detection procedures. This article aims to integrate GIS data with images into applications of change detection. Change detection of linear, area and terrain features based on multi-source data is investigated and change detection based on artificial neural networks (ANN) and GIS data is also analysed. As the powerful GIS functions provide efficient tools for multi-source data processing and change detection analysis, we can expect more research taking this approach as a generic trend in change detection.

Keywords:

• change detection
• GIS
• data fusion
• buffer detection
• ANN

1. Introduction

Change detection is very important for economic construction and national defence. It is a core problem in resource and environmental monitoring, disaster monitoring, land cover/change, city expansion, geographic information update and military defence. Processing of multi-temporal images and change detection has been an active research field in remote sensing for decades. In recent years, great progress has been made to overcome technological obstacles with the development of new platforms and sensors. During the past years, researchers have put forward a large number of change detection techniques for remote sensing images and have summarised or classified them from different viewpoints (Singh 1989, Lu et al. 2004, Radke et al. 2005). Although many successful application cases have been reported on the monitoring and detection of environmental change, there are enormous challenges in applying multi-temporal imagery to derive timely information on the earth's environment and human activities. It has been generally agreed that change detection is a complicated and integrated process. No existing approach is optimal and applicable to all cases. Furthermore, at present, the degree of automation is low and prevents real-time applications. The previous reviews have classified the detection approaches and drawn many useful conclusions (Mouat et al. 1993, Coppin and Bauer 1996, Gary and Steve 2000, Li et al. 2006, Sui et al. 2008).

With the construction and application of all kinds of spatial databases, growing attention has been paid to timely data updating and change detection. Multi-source data integration has been proved to be an effective way to improve the change detection accuracy of remote sensing, and an effective means to make full use of the existing data about location and attributes of the objects from the database of DLG/GRG. The changes of many objects follow certain rules. For example, road objects, especially important roads, are often static with slow change, thus the road network in existing GIS databases can be employed to identify no-change roads to delineate new roads on the new image. Furthermore, GIS data and methods such as rules or specifications of spatial association, spatial clustering, spatial relation, spatial distribution, spatial evolvement and spatial feature can be integrated and used for detecting changes. It is an important development trend in change detection. In some applications such as urban GIS updating, and monitoring of military objects, the changing altitude of the objects needs to be detected. It is necessary to employ 3D change detection methods (Li and Zhu 2003, Wang et al. 2004). On the other hand, in recent years fast development of 3D reconstruction techniques, such as stereo-pairs formed from aerial and satellite images, Lidar and other techniques, have provided more convenient methods for 3D change detection. In this article, we integrate GIS data including DLG and DEM into applications of change detection. Four typical change detection experiments with image and GIS data, including change detection of linear features based on a buffer-detection algorithm, integrating image registration, feature extraction and change detection into a single process for detecting area features, land use/cover change detection based on ANN and GIS data, and 3D terrain change detection, are all discussed in detail.

2. Image registration for change detection

Although requirements for image pre-processing may vary between different change detection methods, image registration is an important and indispensable step. Precise registration to the multi-temporal images is required by many change detection methods. The accurate spatial registration of multi-temporal imagery is important since mis-registration will lead to largely spurious change detection results. It is generally agreed that geometrical registration accuracy of sub-pixel level is accepted (Dai and Khorram 1998, Zhang 2005).

Although geometrical registration of high accuracy is necessary for the pixel-based methods (such as the algebra method and classification), it is unnecessary for all change detection methods, especially for change detection between GIS data and an image. For the feature-based change detection methods like the object-oriented method, the so-called ‘buffer detection’ algorithm can be employed to compare the extracted features or objects, and in this way the harsh requirement of accurate registration can be avoided (Li et al. 2002). In order to avoid the impact of image registration on change detection, a good method is to develop a new algorithm that integrates image registration and change detection simultaneously. The core idea of this algorithm is to make full use of no-change objects as the foundation of image registration, iteratively registering images and detecting change until conditions converge. Accumulation of registration errors in traditional methods can be overcome in this method and detection accuracy can be improved. The main disadvantage of this method is that the algorithm is more complex and difficult than traditional algorithms.

3. Automatic change detection of linear features based on remote sensing images and GIS data

Road networks are one of the most important geo-spatial objects and are also typical linear features in images. With the fast development of cities and the expansion of urban areas, updating the road network is a key problem for updating geo-spatial information, especially in developing countries. Automatic change detection of road networks is the first and crucial step.

Extracting road networks from imagery with the existing DLGs can reduce the difficulty significantly. Generally coarse registration is needed and features extracted from images can be used to detect the changes from old maps. The so-called ‘buffer detection’ algorithm (Sui 2002) can be employed in change detection for features. The main idea of the ‘buffer detection’ algorithm is to make buffers for corresponding features in the new image and an old map by giving a buffer distance and computing the parts that fall into the buffer. If the ratio of these parts to the original feature is smaller than a pre-defined threshold, then this feature is considered as changed. The buffer distance can be deduced theoretically by root mean square error (RMSE) of feature extraction in the new image and an old map. For images with low resolution, the buffer area can be generated with road centre lines and a certain distance. For images with high resolution, the buffer area can be generated with road centre lines and parallel lines of road edges. All parallel lines inside the buffer area are searched, tracked and connected.

For changed roads and new roads, automatic road grouping and recognition are always necessary. Automatic road grouping according to whole relations can be composed of two parts. One is automatic grouping of similar road segments. This means grouping road segments one by one based on various similarity measurements in a given (usually small) area, and then the whole connection is performed. The other approach is extended road grouping. This involves making up for the information that was omitted when we processed parallel lines. Besides, the road's properties can be the evidence; the knowledge in the knowledge database can be the proof. As roads have wonderful network properties, road intersections derived from existing roads are important proofs, and rivers and vehicles that are verified are also reliable proofs. Especially for high-resolution remote sensing images, extracting vehicle information and green belts can be used as evidence to judge whether the candidates are roads or not, based on the template matching method and the analysis to the spectrum of green vegetation. Afterwards, the Bayesian network model that is used for testing the road network detection is established based on the information of the candidates’ roads, traffic and the green belts, and generates the final road network. Some detection results based on the whole framework proposed in this article are shown in Figure 1.

Figure 1. Automatic change detection and extraction results for a rural road network in low resolution images. The red lines in the original image are the vector lines of an old DLG. The red circles mark obvious errors or omissions. Main lines are unchanged roads, and other small lines are new roads. (a) Original image, (b) candidate road segments, (c) central lines, (d) central lines after feature grouping, (e) road after recognition and (f) detection and extraction results.

4. Automatic change detection of area features based on remote sensing images and GIS data

In order to avoid the impact of image registration on change detection, a good method is to integrate image registration, feature extraction and change detection into one whole process. The core idea is automatic feature matching and an iterative backup and validating procedure. The precondition is the existence of invariant features between the GIS information and remote sensing image, and the main problems lie in feature extraction and matching. This approach is very suitable for area features. Experiments have shown that this method can overcome the accumulation of registration errors in traditional methods and deliver good results.

At a certain scale, area object extraction from remotely sensed imagery is performed, and registration of vectors and imagery is realised based on extracted area features and the geometric registration model. Registration is an iterative process during which mismatching of corresponding objects is deleted, with higher registration precision expected in the next iteration. The result comprises well-registered objects that are also unchanged ground objects. Therefore, geometric model parameters with high precision are obtained. As for deleted GIS objects, their polygons or central lines are used as primary models to extract area objects in corresponding local image regions. Extracted objects are compared with the corresponding objects in the GIS data to determine whether they are changed or not.

The so-called label point determination algorithm is presented for polygon features from coarse to precise. Considering the similarity metrics, we propose an iterative extraction method enlightened by GIS knowledge for areal features. Furthermore, we design a hierarchical searching strategy. Employing a holistic iterative solution, we developed software, which when tested on the data of a district, gave a detection rate of 83.3%. The experiment is illustrated in Figure 2.

Figure 2. Change detection of area features. (a) The pre-orthorectification QuickBird image of Shanghai Pudong Zone, which consists of farmlands, ponds, vegetation, rivers, houses, etc., and is comparatively flat. (b) The pre-processed corresponding GIS data from 1998 in a local georeference system. (c) Change detection results of the total solution procedure, with the area objects changed after the period of time numbered. (a) QuickBird image in 2002, (b) pre-processed vector data in 1998 and (c) change detection result of area objects.

5. Automatic land use/cover change detection based on ANN, wavelet technique and GIS

In fact, there are still some problems when using only a new image and an old GIS dataset to detect the changes of line and area objects. The main problems lie in that we have to extract features from the old image only because we do not have any other information. However, feature extraction is very difficult and perhaps causes the failure of change detection. So if an existing old image corresponds with an old map, the difficulty of change detection can be reduced significantly.

The ANN has a strong ability for non-linear mapping, good self-adaptability and low demands for data distribution and it is suitable for multi-source data analysis. Integration of ANN, wavelet and other techniques for change detection can deliver better change detection results (Wang 1998). GIS data and knowledge can support ANN and other change detection techniques. Image grey levels, texture, relief information and other geo-data can be integrated to realise research on land use/cover change detection based on ANN, as shown in Figure 3 (Xiao 2001). Visually, land-type change is first represented as grey level change, and texture can only additionally discriminate partial types, such as town and plantation in agriculture lands. So change detection should use spectral, texture or other information selectively and orderly so as not to impair the precision of change detection. The proper procedure is described as follows. First, imagery must be pre-processed, including geometric rectification, radiometric correction and spectral similarity analysis. Second, a primary result of change detection is achieved after ANN-based change detection, and improvement can be made according to the result of elementary detection and spectral similarity analysis. Lastly, wavelet texture feature quantity and other auxiliary geo-data can help to validate changes of land types.

Figure 3. Change detection result and changed area based on ANN and GIS. (a) Imagery taken in 1992, (b) imagery taken in 1996, (c) change detection results and (d) changed area based on GIS data.

6. Automatic terrain change detection based on new remote sensing images and an old DEM/DOM data

Elevations of ground objects were always presumed to be constant in previous theory or applications of change detection in remote sensing. But these presumptions have not been examined, and if the elevation of the terrain surface has really changed, unexpected trouble could influence the result of 2-D change detection (Xia 2006). So, changes of terrain elevation need to be considered for change detection for large-scale spatial databases and 3D change detection should be considered.

With an existing database of control points or digital orthoimages (DOM) and a DEM, the orientation elements of a new enhanced stereoscopic pair can be calculated by image matching and spatial resection. The so-called VLL matching is employed to interpolate the elevation of grid points of the entire stereo model, and the elevations are compared with the old DEM. If the accuracy of a point is acceptable, it is considered as an unchanged point. If the difference of elevation exceeds an acceptable range, it is considered as a dubious point. During the process, multiple terrain factors or other related knowledge can be used to decide whether a certain ground point has changed or not. After editing and quality checking, changed points can be discriminated from dubious points, and unchanged points are considered in the next iteration. Final products include the results of change detection, an updated DEM and the orientation elements of a new stereo-pair. If new imagery is acquired by a digital camera with a POS system, orientation elements provided by POS may be regarded as initial values, and this is advantageous for faster computation.

In the process of producing a digital orthoimage by a digital photogrammetry workstation, the two orthoimages generated from the left and right photographs, respectively, of the same stereo model should be coincident in theory, i.e. there should be no parallax. If parallaxes do exist, and the orientation elements are assuredly precise, and there is no problem in the course of orthoimage matching, then the parallaxes occurring in the orthoimage pair reflect errors in the DEM, which are used to produce the orthoimage pair. According to Prof. Kraus’ research, elevation measuring accuracy of a stereo orthoimage pair is three times higher than the elevation accuracy of the DEM used for producing the stereo orthoimage pair. Therefore, the traditional process of updating a DEM by a digital photogrammetry workstation, during which three orientation steps and much manual work are inevitable, can be avoided. Parallax occurring on the orthoimage pair can be employed to refine an outdated DEM directly to realise change detection and updating.

In order to further investigate the approach of terrain change detection, source materials from 1999 and 1988 of Hanzhong Prefecture in Shanxi China, including aerial imagery, DEMs and DOMs, were used for experiments. Terrain data (DEM) was produced by a digital photogrammetric workstation by experienced operators in the Shanxi Bureau of Surveying and Mapping, and was regarded as standard data to evaluate the presented approaches. Figure 4(a) shows the result of the changed terrain area of the Hanzhong region with the change detection based on orthoimage matching. Obviously changed areas have been detected and marked in the image. Figure 4(b) shows the overlay of contours from the updated DEM and orthoimage with a contour interval of 5 m. It can be seen that contours are well overlaid on the surface of the terrain. So the updated DEM is of high accuracy and able to satisfy practical applications.

Figure 4. Terrain change detection based on a new image and an old DEM/DOM data. (a) Terrain changed area of Hanzhong region and (b) overlay of contour from updated DEM and orthoimage (contour interval: 5 m).

7. Conclusion

Change detection is a complicated process influenced by multiple factors. This article focuses on the integration of GIS and remote sensing images for change detection. Change detection of linear, area and terrain features based on DLG/DEM/DOM are separately investigated, and elementary experiments are illustrated in this article. Various semantic and non-semantic information is employed in the different steps. Diverse geo-data contribute to integrating different types of data to automatic and intelligent change detection.

The key problem of image-to-map change detection lies in object detection and segmentation, because feature and object extraction is difficult to implement. But with the development of image segmentation techniques, more and more researchers are studying them and they have obtained good results. Change detection, as well as registration, is very effective but many unsolved problems still exist and 3D change detection should be emphasised, especially for urban change detection.

Acknowledgements

The work described in this article was supported by Dr H.G. Sui, Dr X.D. Zhang, Dr P. Xiao and Dr S. Xia during their doctoral studies with me and funded by the National Natural Fund of China (NSFC) (No. 60602013) and the National Key Fundamental Research Plan of China (973) (No. 2006CB701300).