ABSTRACT
Pansharpening refers to the fusion of a multispectral image (MS) and a panchromatic image (PAN) to obtain a new image with the same spatial resolution as the PAN image and the same spectral resolution as the MS image. This paper describes a new, efficient, and accurate pansharpening architecture. The Vector-Quantized Variational AutoEncoder (VQ-VAE) is the foundation of the proposed method. The VQ-VAE model is trained to learn the non-linear mapping of degraded panchromatic image patches to high-resolution patches. This approach ensures that high-resolution patches can be recovered from low-resolution ones. After training on PAN patches, the VQ-VAE estimates high-resolution multispectral patches for each band of the original multispectral image before reconstructing the high-resolution multispectral image from the patches. The original multispectral image, the panchromatic image, and the estimated high-resolution multispectral image are combined through a modified Component Substitution (CS) process to obtain the pansharpened image. Three large satellite datasets from urban areas with 4-band spectral resolution (blue, green, red, and near-infrared) were used to evaluate the proposed pansharpening method’s performance. The effectiveness of the proposed method is demonstrated by the quantitative and visual results obtained compared to several literature approaches.
Disclosure statement
No potential conflict of interest was reported by the author(s).