Heterogeneous self-supervised interest point matching for multi-modal remote sensing image registration

ABSTRACT

The implementation of multi-modal remote sensing image registration is still a challenge in various applications. In this paper, a novel framework called Heterogeneous SuperPoint Network is proposed for multi-modal remote sensing image registration. Heterogeneous SuperPoint Network operates on full-sized images and produces multi-modal interest points accompanied with fixed length descriptors in a single forward pass. Moreover, we develop a strategy for training a base detector that used in conjunction with heterogeneous homographic adaptation to generate pseudo-ground truth interest point labels for unlabeled images in a self-supervised fashion. Then, the interest points and point correspondences obtained by Heterogeneous SuperPoint Network are used to do the final matching with the nearest neighbor (NN) matching and Random sample consensus (RANSAC). Various image sets have been considered in this paper for the evaluation of the proposed approach. Experimental results demonstrate the superior performance of the proposed method over the existing design for the whole data set.

KEYWORDS:

• Image registration
• mutli-modal remote sensing images
• interest point matching
• deep learning

Disclosure statement

No potential conflict of interest was reported by the authors.

Data availability statement

The datasets used in this paper are available in the following domain resources: FLIR: https://www.flir.cn/oem/adas/adas-dataset-form/.

COCO2014: http://images.cocodataset.org/zips/train2014.zip

QXS-SAROPT: https://github.com/yaoxu008/QXS-SAROPT

VeDAI: https://downloads.greyc.fr/vedai/