AOHSR: attention mechanism-based optical image-guided hyperspectral image super-resolution

ABSTRACT

With the rapid development of hyperspectral imaging technology, hyperspectral image (HSI) has been applied in various fields. However, due to the constraint of hyperspectral imaging devices, the spatial resolution of HSI remains too low to accurately meet the needs of target detection and recognition, which shackles the further application of HSI. In this paper, an optical image-guided HSISR algorithm based on the attention mechanism (AOHSR) was proposed in order to effectively improve the resolution of HSI. With the help of the attention mechanism, the proposed algorithm can effectively retain the spectral information of HSI and the spatial information of the optical image. Even without high spatial resolution optical imaging in the test stage, favourable reconstruction results can still be achieved. Firstly, the framework of the proposed optical image-guided HSI SR algorithm based on the attention mechanism was introduced. The algorithm consists of three modules: spectral reservation module (SRM), optical image guidance module (OIGM) and spatial-spectral recovery module (SSRM). The SRM uses an improved channel attention mechanism to extract spectral features from low-resolution (LR) HSIs to preserve spectral information. Through an improved spatial attention mechanism, the OIGM extracts spatial information from high-resolution (HR) optical images, supplementing the information extracted from LR HSI. The SSRM is used to fuse the spectral information on HSI and the spatial information on optical images to reconstruct HR HSI. Then, a joint training algorithm for reconstruction loss and optical image content loss was also proposed, which can effectively guide to learn the reconstruction of detailed information of the spatial texture while ensuring the preservation of spectral information. The results of experiments on three open datasets reveal that the HSIs reconstructed by the proposed network performed better than the existing algorithms regardless of whether optical images in the test stage exist.

KEYWORDS:

• Hyperspectral image
• super-resolution reconstruction
• optical image guidance
• attention mechanismP

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was partially supported by the National Natural Science Foundation of China (Nos. 62076165, 61771319, 61871154), Natural Science Foundation of Guangdong Province (No. 2019A1515011307), Shenzhen Science and Technology Project (No. JCYJ20180507182259896, No.20200826154022001), Innovation Fund for Postgraduate of Shenzhen University (No. 315-0000470829) and the Other Project (Nos.2020KCXTD004, WDZC20195500201).