Scale-invariant feature transform based on the frequency spectrum and the grid for remote sensing image registration

Abstract

A novel image registration method is proposed based on the scale-invariant feature transform (SIFT). Our approach combines the image frequency spectrum with a regular grid. The method exploits features of SIFT, the greater number of feature points in high-frequency areas of an image and regular grid traits with a fine spatial distribution to extract evenly distributed feature points. Global correspondence error checking is used to remove false matching pairs. The registration of eight groups of remote sensing images shows that our method has well-distributed feature points in the spatial and scale domains, obtains an average of 15.78% matching pairs more than uniform robust (UR)-SIFT, and provides better registration accuracy than SIFT and UR-SIFT.

Keywords:

• remote sensing image registration
• scale-invariant feature transform
• feature detection
• feature point distribution

1. Introduction

Image registration is the process of overlaying two or more images of the same scene taken at different times from different viewpoints and/or by different sensors (Zitová and Flusser 2003). It is a vital step in remote image analysis, which includes remote image fusion (Yang, Kim, and Madden 2012; Yu et al. 2012), environmental surveillance, change detection (Jensen and Lulla 1987), and so on and has been studied by the professionals in the field of remote sensing image processing for several decades. With the rapid development of computer technology, aerospace engineering, and remote sensing techniques, remote sensing images can now provide hyper-spectral, high-resolution, and multi-source data that give a more exhaustive and accurate description of the characteristics of terrain objects (Zhang et al. 2005; Goncalves, Corte-Real, and Goncalves 2011). Thus, there is a high data volume, more local distortions, and control points. This greatly influences the accuracy and speed of registration (Hong and Zhang 2008).

Many automated remote sensing registration methods have been proposed. These methods fall into two main categories (Table 1):

1.

Area-based methods: These can be divided into methods that use image pixel values directly, including cross-correlation (Bamea et al. 1972) and mutual information (Chen, Varshney, and Arora 2003; Kern and Pattichis 2007), and those that use the frequency domain, including fast Fourier transform-based methods (Reddy and Chatterji 1996). Although most area-based methods have high positional accuracy, they require initial matching positions that can be easily affected by illumination differences. And they have a large amount of calculation.

2.

Feature-based methods: These methods extract and match salient features from two images. They are divided into methods that use low-level image features, such as edges (Hsu and Beuker 2002), corners (Wang et al. 1983), contours (Li, Manjunath, and Mitra 1995), lines (Medioni and Nevatia 1984), and regions (Goshtasby, Stockman, and Page 1986) from the images, and methods that use higher-level structural and syntactic descriptors (Brown 1992), including the scale-invariant feature transform (SIFT) (Lowe 1999, 2004) and speed-up robust features (Bay, Tuytelaars, and VanGool 2008). The form of the descriptor, as well as the type of feature used for matching, depends on the task to be performed. Compared with area-based methods, feature-based methods are more robust and more reliable, but they are usually less accurate and have eliminating mismatched pairs issue.

Because of its invariance to image scaling and rotation and its partial invariance to illumination and camera viewpoint changes, SIFT has been successfully applied to the registration of remote sensing images. SIFT's descriptor is also considered to perform better than similar descriptors (Mikolajczyk and Schmid 2005). Therefore, there are a number of SIFT-based methods. To improve the computational performance of feature matching, Yan and Rahul (2004), Duan et al. (2008), and Liu et al. (2010) employed principal component analysis, independent component analysis, and kernel independent component analysis, respectively, to remove high-order redundant information in the SIFT descriptors. El Rube, Sharks, and Salem (2009) combined SIFT and wavelet theory to reduce the computational load, and Mikolajczyk and Schmid (2005) presented a new gradient location and orientation histogram to make SIFT's descriptor more unique and robust. Morel and Yu (2009) also developed an improved method, ASIFT, to resolve the incomplete affine element of SIFT's descriptor. However, SIFT's strategy of selecting feature points is only based on a threshold of the contrast value and the ratio of principal curvatures, and it does not have the ability to control the distribution or quantity of feature points. There are a limited number of methods for resolving the distribution of SIFT feature points, which may lead to poor registration accuracy when there is local distortion with few feature points. Although Sedaghat, Mokhtarzade, and Ebadi (2011) developed a uniform robust (UR)-SIFT to improve the distribution of feature points, their method rarely obtains sufficient feature points of each scale to ensure scale invariance. Moreover, when SIFT-based methods are adopted to align remote sensing images, numerous false matches appear and registration accuracy is greatly reduced.

To addresses the uneven distribution problem of SIFT and the inadequate number of feature points in UR-SIFT, we propose a novel remote sensing registration method based on the frequency spectrum of an image and a regular grid (FS-G-SIFT) in this article. This method first uses SIFT and the combination of an image frequency spectrum and a regular grid to detect well-distributed feature points. Global correspondence error checking is then utilized to eliminate false matching pairs. FS-G-SIFT has high-quality features with a good distribution in both the scale and spatial domains and can be applied to various types of remote sensing images with wide ranges of scale, rotation, translation, illumination, and viewing point differences.

Table 1. Summary of the area-based and feature-based registration methods

	Area-based method	Feature-based method
Classical algorithm	Spatial domain: Cross-correlation (CC) and Mutual information (MI)	Point feature: Harris, scale-invariant feature transform (SIFT), and speed-up robust features (SURF)
		Line feature: line segment and outline
	Frequency domain: phrase correlation	Region feature: core of region
Advantage	Easy hardware implementation; benefit for the automatic registration	It is not sensitive to illustration change, has a low calculation amount, and can handle complex between-image distortions.
Disadvantage	It cannot deal with complicated geometric deformations, is very sensitive to the intensity changes, and has an extreme computational load.	The preprocess step is needed and the registration results are highly dependent on the result of this preprocess.
Application	Medical image	Remote sensing and computer vision

Table 2. Details of the data sets in the experiments

	No.	Image pairs	Transform type	Spectral mode	Acquisition date	Resolution	Image size
Data set 1	1	TerraSAR-X	-	Band: X	26 June 2009	2 m	580 × 701
		TerraSAR-X		Band: X			770 × 744
Data set 2	2	SPOT5	scale	Multi-spectral B: 1,2,3	4 October 2004	10 m	502 × 496
		SPOT5		Multi-spectral B: 2,4,3			628 × 620
	3	WorldView-II	scale	Multi-spectral B: 6,7,8	13 January 2010	2 m	853 × 663
		WorldView-II		Multi-spectral B: 5,6,4			913 × 865
	4	QuickBird	rotate	Multi-spectral B: 2,4,3	30 September 2003	2.44 m	781 × 735
		QuirkBird		Multi-spectral B: 1,2,3			737 × 749
	5	IKONOS	rotate	Multi-spectral B: 2,4,1	31 May 2005	4 m	605 × 595
		IKONOS		Multi-spectral B: 3,2,1			683 × 634
	6	SPOT5	rotate/scale	Panchromatic B: 1	4 October 2004	2.5 m	1077 × 948
		SPOT5		Panchromatic B: 1			1191 × 993
Data set 3	7	IKONOS	rotate/scale	Multi-spectral B: 1,2,3	31 May 2005	4 m	908 × 983
		SPOT5		Multi-spectral B: 2,1,3	4 October 2004	10 m	422 × 413
	8	SPOT5	rotate/scale	Multi-spectral B: 2,4,3	13 January 2010	10 m	372 × 455
		QuickBird		Multi-spectral B: 1,4,2	30 September 2003	2.44 m	1084 × 1296
Notes: No. is the number of the experimental group. Transform type is the transform between a pair of images. Spectral mode indicates that the image consists of some bands. “-” denotes that there are no transforms between two TerraSAR-X images, but there are some different scenes in them.

Table 3. Feature point detection results and matching results for data sets in Table 2

	SIFT			UR-SIFT			FS-G-SIFT
No.	N r/N s	N match	R mis	N r/N s	N match	R mis	N r/N s	N match	R mis	N rset/N sset
1	4403	3314	11.39%	1470	812	0.97%	1579	1060	0.46%	1626
	6167			2040			2244			2292
2	2706	752	27.48%	922	262	1.13%	964	275	1.81%	996
	5479			1405			1527			1557
3	4473	429	55.76%	1801	192	4.00%	2161	264	5.68%	2262
	7321			2504			3009			3159
4	6273	193	85.33%	1997	81	1.22%	2209	104	3.81%	2296
	4120			1927			2111			2208
5	3053	345	55.60%	1492	147	2.00%	1365	175	2.36%	1438
	3385			1808			1640			1732
6	11198	4011	26.33%	3636	1383	1.21%	3955	1400	1.56%	4084
	18156			4322			4590			4731
7	7743	74	76.65%	3122	41	4.65%	3399	44	4.34%	3570
	1659			619			688			697
8	2452	24	65.71%	635	7	22.2%	650	16	16.67%	677
	13899			4808			5619			5449
Notes: N r and N s are the number of feature points detected from the reference image and sensed image, respectively. N match is the number of matching pairs. R mis is the mismatch rate, which is the ratio of mismatched pairs to total matching pairs. N rset and N sset are the pre-set numbers of feature points in the reference and sensed images, respectively.

Table 4. Comparison results of RMSE_test and RT for data sets in Table 2

	RMSEtest			RT (s)
No.	SIFT	UR-SIFT	FS-G-SIFT	SIFT	UR-SIFT	FS-G-SIFT	R d_correct (%)
1	0.77	0.77	0.76	33.82	29.60	34.49	23.40
2	0.61	0.60	0.58	21.06	18.46	21.48	4.73
3	0.66	0.74	0.68	34.87	36.79	42.43	27.27
4	1.08	1.15	1.07	30.54	32.13	38.52	22.12
5	1.11	1.09	0.99	20.22	21.66	23.73	16.00
6	0.04	0.12	0.04	70.25	78.24	86.43	1.21
7	1.95	1.62	1.59	24.31	23.98	25.90	6.82
8	1.75	1.70	1.50	39.13	41.45	42.88	56.25
Notes: RMSEtest is the registration accuracy from the test points. RT is the processing time of extracting feature points, generating feature descriptors and getting the matching pairs. R d_correct is the ratio of additional matching pairs from FS-G-SIFT compared with UR-SIFT to the number of matching pairs from FS-G-SIFT.

Figure 1. Flowchart of FS-G-SIFT.

Figure 2. R _{d_correct} results for different values of the total desired number N.

Figure 3. RMSE_test comparison for the image pairs in Table 2 among SIFT, UR-SIFT, and FS-G-SIFT.

Figure 4. Matching results for two TerraSAR-X satellite images in Table 2.

Figure 5. Matching results for two WorldView-II satellite images in Table 2.

Figure 6. Matching results for two IKONOS and SPOT5 satellite images in Table 2.

Figure 7. Registration results of three image pairs in Table 2.

Figure 8. Nonuniform distribution of SIFT feature points for an IRS Image (900 × 900 pixels).

Figure 9. Ratio comparison of feature points in the scale domain for UR-SIFT, FS-G-SIFT, and candidate keypoints extracted from the DoG scale space for a WorldView-II satellite image (B 4/3/2) of Hangzhou, China.

2. Methods

2.1. Problems with SIFT and UR-SIFT

SIFT was proposed by Lowe in 1999 and completed in 2004. It is well known that SIFT is invariant to image scaling and rotation and partially invariant to illumination and camera viewpoint changes. UR-SIFT (Sedaghat, Mokhtarzade, and Ebadi 2011) was intended to improve the distribution of feature points of SIFT. Despite its outstanding characteristics, SIFT cannot obtain well-distributed feature points in the spatial or scale domains when applied to the registration of remote sensing images. Although UR-SIFT can improve the distribution of feature points, it does not always obtain an adequate number of points at some scales. These problems are now discussed in greater detail.

2.1.1. The uneven distribution problem of SIFT

The quantity, quality, and distribution of feature points determine the accuracy and reliability of remote sensing image registration. A relatively uniform distribution of feature points is essential for accurate piecewise transformation when there are local distortions in remote sensing images.

Despite the outstanding characteristics of SIFT, it cannot obtain an even distribution of feature points in the spatial or scale domains when applied to the registration of remote sensing images. As a precondition that there must be enough feature points in different scale layers, SIFT can be invariant to scale change. Although SIFT can extract many candidate keypoints, it only preserves stable ones that satisfy some threshold of the contrast value and the ratio of principal curvature. Therefore, SIFT cannot control both the total number of feature points and their number in different scale layers. That is, it lacks control over the quantity and distribution of feature points.

2.1.2. The insufficient points problem of UR-SIFT

Although UR-SIFT can improve the distribution of feature points, there are still some shortcomings. UR-SIFT uses a linear proportion to improve the distribution of feature points in the scale domain, but the proportion does not correspond to that of the candidate keypoints actually extracted from the difference-of-Gaussian (DoG) scale space. As a result, the required keypoints cannot be satisfied when the image layers possess large scale factors.

2.2. FS-G-SIFT

The proposed FS-G-SIFT initially uses feature point extraction to obtain a controllable number of well-distributed and stable feature points for remote sensing images. This is followed by global correspondence error checking. A flowchart of FS-G-SIFT is presented in Figure 1, and the following subsections discuss the details of each module.

2.2.1. Scale-space generation

First, FS-G-SIFT constructs the DoG scale space of an image (Lowe 2004). The Laplacian of Gaussian (LoG) scale space of an image, consisting of several octaves of blurred image layers, is constructed by the convolution of an image with a variable scale Gaussian function . For each octave, the first image is repeatedly convolved with Gaussian kernels with a constant multiplicative factor . This obtains different blurred images forming an octave. An image with twice the scale of the first in the previous octave is down-sampled by a factor of two as the first image in the next octave. The difference of two nearby scales separated by a constant factor is then defined as the DoG. By approximating the Laplacian function with its DoG function, this step can efficiently detect feature points.

The scale factor of the sth scale layer in the oth octave can be computed as follows:

where is the number of octaves, usually determined by the image size, is the number of middle image layers in each octave that can be used for extrema detection (its value is often three), the initial scale factor , and the constant factor (Lowe 2004).

Candidate keypoints are then obtained from the DoG scale space by detecting local maxima and minima of . A point is selected only if it is larger or smaller than its eight neighbors in the current image and 18 neighbors in the neighboring (higher and lower) scales. This guarantees that the candidate keypoints are extrema in both the spatial and scale domains.

2.2.2. Numbers of feature points to be extracted

According to signal processing theory, the convolution of the Gaussian function and 2D image in the spatial domain, whose result is a smoothed low-frequency image, corresponds to the product of their Fourier transforms in the frequency domain.

where and denote the Fourier transforms of the Gaussian function (Equation (3)) and image , respectively. The Fourier transform of the Gaussian function is expressed in Equation (4):

It can be observed that the Fourier transform of the Gaussian function is also a Gaussian function and that a greater variance in gives a smaller variance in . Characterizing the width of the Gaussian function, larger values of parameter in will give a narrower and a smoother image.

Hence, based on the process of scale-space generation, it can be determined that the LoG scale space consists of images with different low-frequency components. As the octave and layer increase, the variable scale factor increases and more high-frequency components are filtered. The DoG function is an approximation of the LoG function, so the DoG scale space possesses a similar frequency spectrum to the LoG scale space. In addition, feature points represent salient characteristics of the image, belonging to high-frequency components. Thus, the more high-frequency components there are in an image, the more feature points there will be. In other words, the number of feature points increases as the variable scale factor decreases.

Based on the above analysis, the scale-space distribution problem is addressed by assigning an appropriate number of feature points to each scale layer. This number is proportional to the scale factor of that layer. Owing to the smoothing characteristics of the DoG scale space, the number of feature points in each scale layer should be inversely proportional to the square of the scale factor. If it is intended that feature points are selected from the DoG scale space, the number of required feature points in the sth scale layer of the oth octave and the first scale layer of the first octave are denoted by and , respectively. We thus have

where represents the octave index, is the image layer index in the same octave, and and represent the scale factor in the sth scale layer of the oth octave and the first scale layer of the first octave, respectively.

Substituting Equations (1) and (5) into Equation (6), we obtain

Finally, based on Equations (6) and (7),

This guarantees a reasonable number of feature points in each scale layer whose ratios are similar to the candidate keypoints actually extracted from the DoG scale space, thus creating a suitable environment for scale invariance.

Furthermore, the spatial distribution problem of feature points is addressed by applying regular grids to each image layer and determining the desired number of feature points in each grid from the amount of information. The average gradient, number of actual candidate keypoints, and mean contrast of a grid are used to measure how much information it contains. The average gradient reflects the gray-change rate of an image and the expressive power of the image detail contrast, which is represented by

where is the width of a grid, is its height, and is the DoG response value of row and column in the grid. The number of candidate keypoints indicates the texture information of an image. A higher mean contrast value indicates the presence of strong and stable feature points. The contrast value is expressed by the absolute value of the DoG response. Thus, the number of feature points required for the grid in the sth scale layer of the oth octave is

where are the average gradient and the number and mean contrast of actual candidate keypoints in grid , respectively. are the total average gradient, the total number, and total mean contrast of candidate keypoints in an image layer, respectively, and are their respective weights, which satisfy the equation .

2.2.3. Accurate localization

After setting the number of feature points to be extracted in each scale layer and every grid, unstable points should be removed to obtain the desired number of stable feature points.

As the candidate keypoints from 2D discrete space are not necessarily real extrema, they are relocated using Taylor's interpolation equation. Unstable points are rejected when the absolute DoG values of their interpolations are less than 0.03 or when their , , or interpolation offsets exceed 0.5 (Lowe 2004). In addition, points close to edges are unstable and should be discarded because they have a larger principal curvature across the edge but a small one in the vertical direction. There is also a proportional relationship between the eigenvalue of the 2 × 2 Hessian matrix of each keypoint and its principal curvature. Hence, unstable points whose eigenvalue ratio is greater than a threshold of 10 are removed (Lowe 2004).Finally, the feature point extraction of FS-G-SIFT can be described as follows:

Algorithm 1: Feature point exatrction

Input: (the original image), (the desired feature points).

Output: (the extracted feature points).

 1: Establish the DoG scale space of and extract its candidate keypoints;

 2: Calculate the desired number of each scale layer in each octave;

 3:  for to do

 4:  for to do

 5: Use a regular mesh to grid the sth image layer;

 6: Calculate the desired number of grids in the sth image layer;

 7: For each grid in the sth image layer;

 8: Precisely locate the candidate keypoints and calculate their contrast ;

 9: Keep the greatest value of in the candidate keypoints, and remove the rest;

10: Remove keypoints whose principal curvature ratio is greater than a threshold value of 10;

11: Keep candidate keypoints with the greatest contrast values;

12:  end for

13:  end for

14: Obtain the extracted feature points .

2.2.4. Global correspondence error checking

After obtaining the original SIFT descriptors of the reference and sensed images, the ratio between the closest and second-closest neighbors of the target descriptor is exploited to obtain matching pairs. This is likely to yield false matching pairs and reduce the registration accuracy (Yu, Zhang, and Holden 2008). Global correspondence error checking is utilized to eliminate false matching pairs in this article. Based on a known global affine transformation model and a threshold of the root mean square error (RMSE), matching pairs with the greatest variance are regarded as false and are eliminated. This step is repeated until the RMSE of the remaining pairs is smaller than the threshold. A final transformation model is then solved by the remaining matching pairs. The global affine transformation model can be expressed by

where and are a matching pair of the reference image and sensed image, respectively, and are parameters of the transformation model.

Here, the RMSE, defined as RMSE_check, is taken as a measure of whether a matching pair is false:

where is the total number of matching pairs and the RMSE is given in units of pixels.The global correspondence error checking of FS-G-SIFT can be described as follows:

Algorithm 2: Global correspondence error checking

Input: (number of matching pairs), (matching pairs), (threshold of RMSE),

(initial transformation model).

Output: (correct matching pairs).

1: Calculate the variance of each matching pair ;

2: Calculate the of the matching pairs;

3: While and

4: find the greatest and eliminate it;

5: ←;

6: calculate the of the remaining matching pairs;

7: Obtain the correct matching pairs

2.3. Evaluation criteria

Several criteria are also adopted to quantitatively evaluate the method:

1.	Number of feature points and correct matching pairs: the number of feature points from the reference image and sensed image in each pair of the above data sets and the number of matching pairs are recorded. The ratio of the mismatched pairs (Rmis) is also used here.
2.	RMSEtest: This is widely used by many researchers to determine the registration accuracy. Some 10 test-point pairs selected manually for each test case cannot be set to zero by overfitting. Given the two point sets, their RMSE is calculated by Equation (12) and is herein referred to as RMSEtest.
3.	Running time (RT): defined as the execution time needed by the detection and matching scheme to complete the task. All experiments are performed on an Intel Core 2 6600 (2.40 GHz processor) with 3.25 GB of physical memory using the VS2008 compiler.
4.	Rd_correct: the ratio of the difference in the number of correct matching pairs between FS-G-SIFT and UR-SIFT to the number of correct matching pairs from FS-G-SIFT:

where a and b are the number of correct matching pairs obtained from FS-G-SIFT and UR-SIFT, respectively. This ratio is mainly used to compare FS-G-SIFT and UR-SIFT, which have similar purposes.

3. Experimental data

Three datasets (containing different scenes, different scales or rotations, and different sensors, respectively) are used to systematically test the robustness of our approach. Their details are presented in Table 2:

1.	Data set 1: a pair of images acquired by the same sensor of TerraSAR-X with different scenes. Considering the small scene differences, a good image registration result can be expected (http://www.dlr.de/desktopdefault.aspx/tabid-4237/6774_read-18112/).
2.	Data set 2: five sets of the same sensor image pairs, acquired by four different satellite sensors. They are all false-color images combining three different bands. To validate the invariance of FS-G-SIFT to scale, tests are performed on a pair of multi-spectral SPOT5 satellite images and a pair of WorldView-II satellite images, where there are scale differences (0.8) between the reference and sensed images. Another two tests on multi-spectral IKONOS and QuickBird satellite images with rotation differences (10°) are used to verify the invariance of FS-G-SIFT to rotation. Two panchromatic SPOT5 satellite images with both scale (0.8) and rotation (10°) differences are also tested.
3.	Data set 3: two pairs of images acquired by different sensors (QuickBird and SPOT5, SPOT5 and IKONOS) at different times. These have big differences in their gray values, aspect angles, and scales. Two multi-spectral SPOT5 images are, respectively, matched with a multi-spectral IKONOS and a multi-spectral QuickBird image, all of which are pseudo-color images.

4. Results and discussion

In this section, the performance of FS-G-SIFT is experimentally verified and is compared with that of the original SIFT (Lowe 2004) and UR-SIFT (Sedaghat, Mokhtarzade, and Ebadi 2011).

4.1. Experimental results

In all of our experiments, following Lowe (2004) and Sedaghat Mokhtarzade, and Ebadi (2011), we use , (Section 2.2), and a 100 × 100 regular grid. To attain a high registration accuracy, the threshold of RMSE_check (Section 2.2), used to eliminate false matching pairs, is set to . The weight parameters (Section 2.2) are empirically determined as .

Three data sets (Table 2) are tested for different desired numbers of feature points. As can be seen from R _{d_correct} comparison on condition that the total desired number of feature points are set to be different, shown in Figure 2, R _{d_correct} is greater than zero for all values of N is, which indicates that FS-G-SIFT obtains more matching pairs than UR-SIFT. When the desired number is set to be 0.4% of the image size, there are no abnormal values, and the number of matching pairs in each test is neither too small nor too large. Thus, the desired number of total feature points is set to 0.4% of the image size in the following experiments.

As observed from the feature point detection and matching results in Table 3, the original SIFT obtains the most feature points and matching pairs. The number of feature points detected by FS-G-SIFT is greater than that detected by UR-SIFT and closer to the set numbers. FS-G-SIFT also found more matching pairs than that in UR-SIFT. And the average increment of correct matching pairs (R _{d_correct}) is 15.78%, as shown in Table 4. We can see that only certain scenes difference in the first test, and the high resolution in No. 6 cause higher numbers of feature points and matching pairs to be found. Although there are many feature points in No. 7 and No. 8, the number of matching pairs is very small owing to the large geometrical distortions of the multi-sensor images. In addition, the R _mis value of FS-G-SIFT is much lower than that of SIFT, which indicates that there are fewer false matching pairs because of the global correspondence error checking.

The quantitative RMSE_test and RT results are given in Table 4, and Figure 3 shows the line graph of RMSE_test. It can be seen that the RMSE_test values for both SIFT and UR-SIFT are higher than that from FS-G-SIFT. This indicates that FS-G-SIFT has a better registration accuracy. RT measures the processing time of detecting feature points, generating feature descriptors, and matching features. The RT of FS-G-SIFT is close to that of SIFT and UR-SIFT.

Matching results for No. 1, No. 3, and No. 7 (Table 4) are presented in Figures 4–6, and their registration results are illustrated in Figure 7. As can be seen from Figure 4, there are many matching feature points in the overlapping region of the two images, whereas there are no feature points in the non-overlapping region of the two images. The number of false matching pairs is reduced by the corresponding error check. From Figure 5, we can see that feature points are well-distributed, and the matching pairs are also exactly matched in region-I and region-II. Figure 6 shows that although there are scale, rotation, and illustration changes, the matching pairs in region-I and region-II are accurate.

Hence, it can be seen that FS-G-SIFT is well able to register TerraSAR-X, multi-spectral, and multi-sensor images. The experimental results allow further conclusions to be drawn:

1.	On account of the well-distributed points in the scale and spatial domain, FS-G-SIFT has better registration accuracy than SIFT and UR-SIFT. By this means, the total number of feature points and their distribution can be controlled.
2.	There are fewer feature points and matching pairs obtained from FS-G-SIFT than from SIFT, which will decrease the time required to generate feature descriptors and match features. The running time for detecting well-distributed feature points is not a burden.
3.	The number of feature points detected by FS-G-SIFT is greater than that by UR-SIFT and is closer to the set number. This verifies the effectiveness of FS-G-SIFT in solving the inadequate number problem of UR-SIFT.
4.	Owing to the global correspondence error checking, FS-G-SIFT produces far fewer false matching pairs than SIFT.

4.2. Discussion

Experiments on different images with different transforms have shown that FS-G-SIFT can obtain better image registration accuracy than the original SIFT and UR-SIFT while maintaining the running time. It also can extract well-distributed feature points in both the spatial domain and the scale domain. We now take a closer look at the method and conduct some tests that help explain FS-G-SIFT's success.

A comparison (Figure 8(a) and (b)) of the spatial distribution of feature points detected by SIFT and FS-G-SIFT for an IRS image (http://www.satimagingcorp.com/gallery/irs-omen.html) verifies the better spatial distribution of feature points from FS-G-SIFT. The feature points in regions I and II (Figure 8(b)) indicate that there are neither too few nor too many feature points from our method, which cannot lead to an inaccurate registration result when there are local deformations or a large amount of calculation. This mainly results from the setting number of feature points in each grid, which is based on the grid's image information (Section 2.2). Hence, our method is especially appropriate to the remote sensing images with local deformations.

Compared with UR-SIFT, we show that FS-G-SIFT obtains more feature points and matching pairs (Table 3), which profits from the proper scale ratios based on the frequency spectrum of image layer from the DoG scale space (Section 2.2). The setting number of each scale from FS-G-SIFT is less than the candidate feature points extracted from DoG scale space (Figure 9). Thus, FS-G-SIFT can obtain the sufficient feature points in each scale and its characteristics of scale invariance are more robust, which guarantees the registration accuracy.

However, only the image size is used to control the number of feature points in this article. Image texture and other information that may affect the detection of feature points should be included, which will increase the computational complexity.

5. Conclusions

The main contribution of this article is a strategy for detecting evenly distributed feature points for SIFT. We draw on a combination of the image frequency spectrum and regular grids to obtain well-distributed feature points in both the scale domain and the spatial domain. Our image registration scheme is especially appropriate for images with many local deformations. Compared with UR-SIFT, FS-G-SIFT obtained more feature points and matching pairs. The experimental results show that FS-G-SIFT has better image registration accuracy than two different related methods (SIFT and UR-SIFT).

Acknowledgments

The authors thank Libao Zhang who is an associate professor at the College of Information Science and Technology at Beijing Normal University for his advice and comments. The authors thank the anonymous reviewers for their constructive comments, which substantially improved the quality of this article. This work was supported by the National Natural Science Foundation of China (No. 41272359, 41072245, 11001019), the Specialized Research Fund for the Doctoral Program of Higher Education of China (No. 20120003110032), and the Fundamental Research Funds for the Central Universities of China (No. 2012LZD05).

Notes