A window size selection network for stereo dense image matching

ABSTRACT

Stereo dense image matching normally refers to per-pixel correspondence search between stereo pairs. However, their matching results actually depend on matching window sizes. Large windows usually obtain robust matching results in weak-textured regions, but serious mismatches in depth/disparity (parallax in the epipolar space) inconsistency regions. Small windows compute accurate matching results in depth/disparity inconsistency regions, but it may contain high matching uncertainties in weak-textured regions. To improve matching accuracies, this letter focuses on adaptively selecting appropriate matching windows for each pixel. In general, we propose a window size selection network (WSSN) with the basic assumption that appropriate window sizes are related to image textures and depth/disparity variations. WSSN firstly extracts both image texture features and disparity features by convolutional neural network and then utilizes the fully connected layers to conduct optimal window size selection. Experiments on an aerial image dataset show that our proposed method is capable of selecting appropriate matching windows for each pixel. It achieved the highest matching accuracy when compared with the matching results of a series of fixed matching windows and a state-of-the-art textured based window selection method.

Acknowledgments

The study is partially supported by the Office of Naval Research (Award No. N000141712928). We would like to thank the German Society for Photogrammetry, Remote Sensing and Geoinformation (DGPF) for providing the Vaihingen dataset. The first and second authors contribute equally.

Additional information

Funding

This work was supported by the Office of Naval Research [N000141712928].