Semantic interleaving global channel attention for multilabel remote sensing image classification

ABSTRACT

Multilabel remote sensing image classification (MLRSIC) has received increasing research interest. Taking the co-occurrence relationship of multiple labels as additional information helps to improve the overall performance. However, current methods only focus on using it to constrain the final feature which is output from a convolutional neural network (CNN). On the one hand, these methods need to exploit the potential of label correlation in feature representation fully. On the other hand, they increase the label noise sensitivity of the system, resulting in poor robustness. In this paper, a novel method called ‘Semantic Interleaving Global chaNnel Attention’ (SIGNA) is proposed for MLRSIC. First, the label co-occurrence graph is obtained according to the statistical information of the training set and fed into a graph neural network (GNN) to generate optimal semantic feature representations of each label. Next, the semantic features are interleaved with visual features which are extracted by CNNs to guide the overall features of the input image transform from the original feature space to the semantic feature space with embedded label relations. Then, global attention triggered by semantic interleaving is used to emphasize visual features in important channels. Finally, to make SIGNA easier to use and more optimized, multihead SIGNA-based feature adaptive weighting networks are proposed as plug-in blocks to plug into any layers of a CNN. For remote sensing images, better classification performance can be achieved by inserting the plug-in blocks into the shallow layers of CNNs. We conducted extensive experimental comparisons on three data sets: UCM, AID and DFC15. Experimental results demonstrate that the proposed SIGNA achieves superior classification performance compared to state-of-the-art (SOTA) methods. Notes that the codes of this paper will be open to the community for reproducibility research.

KEYWORDS:

• Remote sensing
• multilabel classification
• gnn
• channel attention
• label relation

Acknowledgements

We would like to thank Qingdao University of Technology for their technical support, as well as all those who participated in this paper. This work was supported in part by the National Natural Science Foundation of China under Grant 62171247.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability of statement

The code and data that support the findings of this study are available from the first author, Yongkun Liu (Email: [email protected]), upon reasonable request. Some useful information is also available at https://github.com/kyle-one/SIGNA.

Authors contribution

All the authors made significant contributions to this work. Project administration, K.X.; Innovations and original draft writing, Y.L.; Coding, K.N.; Review and editing, Y.Z. and J.Z. All authors have read and agreed to the published version of the manuscript.

Additional information

Funding

This work was supported in part by the National Natural Science Foundation of China under Grant 62171247.