GRF: guided residual fusion for pansharpening

ABSTRACT

Fusion of a multispectral (MS) image and a panchromatic (PAN) image is known as the pansharpening, which is a very important preprocessing step in remote sensing. Component substitution (CS)-based methods such as intensity-hue-saturation (IHS) may be the most widely used pansharpening methods and have been adopted in most of the professional software. However, the CS-based methods generally suffer from severe spectral distortions. In this paper, we propose a post-processing technique called guided residual fusion (GRF) to compensate the results of CS-based methods, which inherits high spatial fidelity from generalized IHS (GIHS, take it as an example) and greatly alleviates the spectral distortions. The main idea of our method is to estimate a residual map between the result of GIHS and the ideal high-resolution multispectral (HRMS) image. Specifically, we first obtain an approximate HRMS by inserting the pixels of the MS image according to the spatial resolution ratio between the PAN and MS. Most of the pixels in this HRMS are unknowns. We can get a masked residual map by subtracting the known values of HRMS and their corresponding counterparts of the result of GIHS. Then, we use an interpolation filter and an edge-preserving image smoothing filter to efficiently fill missing pixel values in the masked residual map, obtaining an accurate residual map. Extensive experiments on six real datasets (including datasets acquired by the Geoeye1, IKONOS, Landsat-8, QuickBird, SPOT6, and Worldview-3 sensors) demonstrate that our method outperforms the compared state-of-the-art methods, that is, the performance of our method ranks the best among all nine methods on three datasets (Geoeye1, IKONOS, and SPOT6), the second on two datasets (QuickBird and Worldview-3), and the third on the Landsat-8 dataset. The source code will be made public in https://ljy-rs.github.io/web.

KEYWORDS:

• Pansharpening
• image fusion
• residual map
• IHS
• guided image filtering
• remote sensing

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1. https://rscl-grss.org/coderecord.php?id=541.

2. https://github.com/Decri/Multi-Scale-and-Depth-CNN-for-Pan-sharpening.

3. https://github.com/sergiovitale/pansharpening-cnn-matlab-version.

Additional information

Funding

This work was supported by the National Key R&D Program of China (No. 2020YFC1521900) and National Natural Science Foundation of China (NSFC) (No. 42030102 and 41901398); National Key R&D Program of China [2020YFC1521900]