Multi-scale feature fusion kernel estimation with masked interpolation loss for real-world remote sensing images super-resolution

ABSTRACT

In recent years, the application of deep learning to remote sensing image super-resolution has achieved promising results. However, most deep learning-based methods are often trained on remote sensing datasets constructed by bicubic downsampling, and their recovery effects on real-world remote sensing images are often unsatisfactory. This is because the process of generating low-resolution (LR) images from high-resolution (HR) images to construct training data pairs (LR-HR) using simple bicubic downsampling cannot reflect the degradation process of real-world remote sensing dataset images. In this paper, we propose a multi-scale feature fusion kernel estimation with masked interpolation loss for real-world remote sensing images super-resolution (MFFILSR) to address this problem. MFFILSR is divided into two stages: degenerate kernel estimation and SR network training. In the first stage, we propose a multi-scale feature fusion kernel estimation network that can effectively fuse multi-scale information, making the estimated downsampling kernel closer to the degradation patterns of real-world remote sensing images. In the second stage, we introduce a masked interpolation loss during generator training, by masking the interpolation loss, the artefacts of generated images can be effectively reduced. Extensive experiments show that MFFILSR has satisfactory super-resolution reconstruction performance for real-world remote sensing images.

KEYWORDS:

• Kernel estimation
• multi-scale feature fusion
• masked interpolation loss
• remote sensing
• image super-resolution

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

The work was supported by the Natural Science Foundation of Shandong Province, China (Grant No. ZR2019MF073), the Fundamental Research Funds for the Central Universities, China University of Petroleum (East China) (Grant No. 20CX05001A), the Major Scientific and Technological Projects of CNPC (No. ZD2019-183-008), the Creative Research Team of Young Scholars at Universities in Shandong Province (No.2019KJN019), the State Key Laboratory of Shale Oil and Gas Enrichment Mechanisms and Effective Development under Grants 33550000-22-ZC0613-0243.