Three-branch attention deep model for hyperspectral image classification using cross fusion

ABSTRACT

Deep learning has achieved impressive success in computer vision, especially remote sensing. It is well known that different deep models are able to extract different kinds of features from remote sensing images. For example, the convolutional neural networks (CNN) can extract neighbourhood spatial features in the short-range region, the graph convolutional networks (GCN) can extract structural features in the middle- and long-range region, and the encoder-decoder (ED) can obtain the reconstruction features from an image. Thus, it is challenging to design a model that can combine the different models to extract fused features in a hyperspectral image classification task. To this end, this paper proposes a three-branch attention deep model (TADM) for the classification of hyperspectral images. The model can be divided into three branches: graph convolutional neural network, convolutional neural network, and deep encoder-decoder. These three branches first extract structural features, spatial-spectral joint features and reconstructed encoded features from hyperspectral images, respectively. Then, a cross-fusion strategy and an attention mechanism are employed to automatically learn the fusion parameters and complete the feature fusion.  Finally, the hybrid features are fed into a standard classifier for pixel-level classification. Extensive experiments on two real-world hyperspectral datasets (Houston and Trento) demonstrate the effectiveness and superiority of the proposed method. Compared with other baseline classification methods, such as FuNet-C and Two-Branch CNN(H), proposed method achieves the highest classification results. Specifically, overall classification accuracies of 93.25% and 95.84% were obtained on the Houston and Trento data, respectively.

Acknowledgment

This research is supported by Shaanxi Provincial Philosophy and Social Science Research Special Youth Project (Grant No.:2023QN0235), Shangluo Science and Technology Plan Project (Grant No.:2022-Y-0015) and Natural Science Research Project of Shangluo University (Grant No.:22KYZX06).

Disclosure statement

No potential conflict of interest was reported by the authors.