A sub-pixel convolution-based improved bidirectional feature pyramid network for pansharpening

ABSTRACT

Due to the dramatic scale changes in different ground object characteristics, effectively utilizing multi-scale features of remote sensing images represents a major challenge. This letter proposes a sub-pixel convolution-based improved bidirectional feature pyramid network (SCIBFPN) to address the problem. To reduce the information loss in the process of image pre-processing and multi-scale feature fusion, this letter introduces sub-pixel convolution instead of up-sampling. To obtain richer spatial and spectral information, an improved bidirectional feature pyramid network-based sub-pixel convolution is combined with a residual network (ResNet) for feature extraction. In addition, to further enhance the spatial structure of the fused images, a panchromatic (PAN) image is used as a guide to direct the injection of spatial information. Experimental results from two real datasets show that the proposed method outperforms state-of-the-art methods in terms of both objective metrics and subjective visual evaluation.

KEYWORDS:

• Pansharpening
• feature pyramid network
• deep learning
• multi-scale
• sub-pixel convolution

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

The work was supported by the Henan Province Science and Technology Breakthrough Project [212102210102,212102210105].