Scale-invariant feature transform based on the frequency spectrum and the grid for remote sensing image registration

Figures & data

Table 1. Summary of the area-based and feature-based registration methods

	Area-based method	Feature-based method
Classical algorithm	Spatial domain: Cross-correlation (CC) and Mutual information (MI)	Point feature: Harris, scale-invariant feature transform (SIFT), and speed-up robust features (SURF)
		Line feature: line segment and outline
	Frequency domain: phrase correlation	Region feature: core of region
Advantage	Easy hardware implementation; benefit for the automatic registration	It is not sensitive to illustration change, has a low calculation amount, and can handle complex between-image distortions.
Disadvantage	It cannot deal with complicated geometric deformations, is very sensitive to the intensity changes, and has an extreme computational load.	The preprocess step is needed and the registration results are highly dependent on the result of this preprocess.
Application	Medical image	Remote sensing and computer vision

Table 2. Details of the data sets in the experiments

	No.	Image pairs	Transform type	Spectral mode	Acquisition date	Resolution	Image size
Data set 1	1	TerraSAR-X	-	Band: X	26 June 2009	2 m	580 × 701
		TerraSAR-X		Band: X			770 × 744
Data set 2	2	SPOT5	scale	Multi-spectral B: 1,2,3	4 October 2004	10 m	502 × 496
		SPOT5		Multi-spectral B: 2,4,3			628 × 620
	3	WorldView-II	scale	Multi-spectral B: 6,7,8	13 January 2010	2 m	853 × 663
		WorldView-II		Multi-spectral B: 5,6,4			913 × 865
	4	QuickBird	rotate	Multi-spectral B: 2,4,3	30 September 2003	2.44 m	781 × 735
		QuirkBird		Multi-spectral B: 1,2,3			737 × 749
	5	IKONOS	rotate	Multi-spectral B: 2,4,1	31 May 2005	4 m	605 × 595
		IKONOS		Multi-spectral B: 3,2,1			683 × 634
	6	SPOT5	rotate/scale	Panchromatic B: 1	4 October 2004	2.5 m	1077 × 948
		SPOT5		Panchromatic B: 1			1191 × 993
Data set 3	7	IKONOS	rotate/scale	Multi-spectral B: 1,2,3	31 May 2005	4 m	908 × 983
		SPOT5		Multi-spectral B: 2,1,3	4 October 2004	10 m	422 × 413
	8	SPOT5	rotate/scale	Multi-spectral B: 2,4,3	13 January 2010	10 m	372 × 455
		QuickBird		Multi-spectral B: 1,4,2	30 September 2003	2.44 m	1084 × 1296
Notes: No. is the number of the experimental group. Transform type is the transform between a pair of images. Spectral mode indicates that the image consists of some bands. “-” denotes that there are no transforms between two TerraSAR-X images, but there are some different scenes in them.

Table 3. Feature point detection results and matching results for data sets in Table 2

	SIFT			UR-SIFT			FS-G-SIFT
No.	N r/N s	N match	R mis	N r/N s	N match	R mis	N r/N s	N match	R mis	N rset/N sset
1	4403	3314	11.39%	1470	812	0.97%	1579	1060	0.46%	1626
	6167			2040			2244			2292
2	2706	752	27.48%	922	262	1.13%	964	275	1.81%	996
	5479			1405			1527			1557
3	4473	429	55.76%	1801	192	4.00%	2161	264	5.68%	2262
	7321			2504			3009			3159
4	6273	193	85.33%	1997	81	1.22%	2209	104	3.81%	2296
	4120			1927			2111			2208
5	3053	345	55.60%	1492	147	2.00%	1365	175	2.36%	1438
	3385			1808			1640			1732
6	11198	4011	26.33%	3636	1383	1.21%	3955	1400	1.56%	4084
	18156			4322			4590			4731
7	7743	74	76.65%	3122	41	4.65%	3399	44	4.34%	3570
	1659			619			688			697
8	2452	24	65.71%	635	7	22.2%	650	16	16.67%	677
	13899			4808			5619			5449
Notes: N r and N s are the number of feature points detected from the reference image and sensed image, respectively. N match is the number of matching pairs. R mis is the mismatch rate, which is the ratio of mismatched pairs to total matching pairs. N rset and N sset are the pre-set numbers of feature points in the reference and sensed images, respectively.

Table 4. Comparison results of RMSE_test and RT for data sets in Table 2

	RMSEtest			RT (s)
No.	SIFT	UR-SIFT	FS-G-SIFT	SIFT	UR-SIFT	FS-G-SIFT	R d_correct (%)
1	0.77	0.77	0.76	33.82	29.60	34.49	23.40
2	0.61	0.60	0.58	21.06	18.46	21.48	4.73
3	0.66	0.74	0.68	34.87	36.79	42.43	27.27
4	1.08	1.15	1.07	30.54	32.13	38.52	22.12
5	1.11	1.09	0.99	20.22	21.66	23.73	16.00
6	0.04	0.12	0.04	70.25	78.24	86.43	1.21
7	1.95	1.62	1.59	24.31	23.98	25.90	6.82
8	1.75	1.70	1.50	39.13	41.45	42.88	56.25
Notes: RMSEtest is the registration accuracy from the test points. RT is the processing time of extracting feature points, generating feature descriptors and getting the matching pairs. R d_correct is the ratio of additional matching pairs from FS-G-SIFT compared with UR-SIFT to the number of matching pairs from FS-G-SIFT.

Figure 1. Flowchart of FS-G-SIFT.

Figure 2. R _{d_correct} results for different values of the total desired number N.

Figure 3. RMSE_test comparison for the image pairs in Table 2 among SIFT, UR-SIFT, and FS-G-SIFT.

Figure 4. Matching results for two TerraSAR-X satellite images in Table 2.

Figure 5. Matching results for two WorldView-II satellite images in Table 2.

Figure 6. Matching results for two IKONOS and SPOT5 satellite images in Table 2.

Figure 7. Registration results of three image pairs in Table 2.

Figure 8. Nonuniform distribution of SIFT feature points for an IRS Image (900 × 900 pixels).

Figure 9. Ratio comparison of feature points in the scale domain for UR-SIFT, FS-G-SIFT, and candidate keypoints extracted from the DoG scale space for a WorldView-II satellite image (B 4/3/2) of Hangzhou, China.