Hyperspectral image classification based on local feature decoupling and hybrid attention SpectralFormer network

ABSTRACT

Classification of hyperspectral image (HSI) is an essential step in various remote sensing tasks. Currently, transformer-based models have made significant contributions to HSI classification due to their remarkable ability to model the relation among long-range sequence data. However, HSI data exhibit strong nonlinear coupling and strong correlation, leading to a decrease in the classification performance of transformer-based models. The SpectralFormer network has achieved excellent performance in HSI classification, but its cross-layer adaptive fusion mechanism cannot effectively highlight critical information, affecting the representation ability of spectral-spatial features. To address the above issues, this paper proposes an HSI classification method based on local feature decoupling and SpectralFormer with a hybrid attention module (HAMSF) network. Specifically, this paper first uses the histogram of oriented gradient algorithm to preprocess HSI coupled data and achieve preliminary nonlinear decoupling of HSI data. Secondly, the hybrid attention module is embedded in a cross-layer adaptive fusion mechanism, generating refined neighbouring information before each information fusion to improve the spectral-spatial feature representation ability. Finally, the processed data are input into the HAMSF network for the final decoupling and classification of HSI. The experiment is carried out on three benchmark hyperspectral datasets, and the experimental results confirm that the proposed method has better classification performance compared to several transformer-based and classical HSI classification methods.

KEYWORDS:

• Nonlinear coupling
• hyperspectral image classification
• spectralformer network
• histogram of oriented gradient
• hybrid attention module

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data Availability statement

The data that support the findings of this study are available on www.ehu.eus/ccwintco/index.php/Hyperspectral_Remote_Sensing_Scenes

Additional information

Funding

This research was funded by the Foundation of the National Natural Science Foundation of China grant numbers 61973105, 61573130 and 52177039; The Key Technologies R & D Program of Henan Province of China [NO.212102210145, 212102210197 and NO.222102220016].