Semi-supervised hierarchical Transformer for hyperspectral Image classification

ABSTRACT

Transformer has achieved outstanding performance in many fields such as computer vision benefiting from its powerful and efficient modelling ability and long-range feature extraction capability complementary to convolution. However, on the one hand, the lack of CNN’s innate inductive biases, such as translation invariance and local sensitivity, makes Transformer require more data for learning. On the other hand, labelled hyperspectral samples are scarce due to the time-consuming and costly annotation task. To this end, we propose a semi-supervised hierarchical Transformer model for HSI classification to improve the classification performance of the Transformer with limited labelled samples. In order to perturb the samples more fully and extensively to improve the model performance, two different data augmentation methods are used to perturb the unlabelled samples, and two sets of augmented samples are obtained respectively. The pseudo-label obtained on the original unlabelled sample is used to simultaneously supervise the augmented sample obtained on this unlabelled sample. Among them, only the pseudo-labels above the threshold are retained. To further improve the model stability and classification accuracy, hierarchical patch embedding is proposed to eliminate the mutual interference between pixels. Extensive experiments on three well-known hyperspectral datasets validate the effectiveness of the proposed semi-supervised Transformer model. The experiments show that the model achieves excellent classification accuracy even when there are only 10 labelled samples in each category, which can effectively improve the classification performance of Transformer under small-scale labelled samples.

KEYWORDS:

• Hierarchical patch embedding
• hyperspectral image classification
• semi-supervised learning
• transformer

Acknowledgements

We would like to thank Prof. Melba Crawford for providing the Indian Pines 2010 data. In addition, the authors thank the National Center for Airborne Laser Mapping at the University of Houston, Texas, for providing the CASI Houston dataset and the IEEE Geoscience and Remote Sensing Society Image Analysis and Data Fusion Technical Committee for organizing the 2013 Data Fusion Contest.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1. The data were provided by Prof. N. Yokoya from the University of Tokyo and RIKEN AIP.

Additional information

Funding

This work was supported by Hebei Province Natural Science Foundation (No.E2021202179), Key Research and Development Project from Hebei Province (No.19210404D, No.20351802D, No.21351803D), the General Project of Science and Technology of Beijing Municipal Education Commission (KM202210005023).