Graph convolutional network with STC attention and adaptive normalization for skeleton-based action recognition

ABSTRACT

Graph Convolutional Network (GCN) have been widely used in the field of skeleton-based action recognition and have achieved exciting results. Introducing attention mechanism in the process of extracting skeleton features has always been a hot spot in GCN-related research. In this article, we design a new graph convolutional network, which combines the advanced decoupling graph convolutional network (DC-GCN) with spatial, temporal, channel (STC) series attention module and adaptive normalization (AN). The STC attention module helps the network tend to extract important information from skeleton features. In addition, in order to improve the adaptability of the normalization method to GCN, we design the AN module instead of the BN module, which can train the weights of different normalization methods, so that each normalization layer in the network adopts the most suitable normalization operation. The experimental results show that the accuracy of our method is competitive with the state-of-the-art action recognition methods.

KEYWORDS:

• Skeleton-based action recognition
• Skeleton features
• graph convolutional network
• STC attention module
• Spatial attention
• Temporal attention
• Channel attention
• adaptive normalization

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by the Fundamental Research Funds for the Central Universities of China under Grant LGYB202201; National Natural Science Foundation of China under Grant 62271160 and 61871142; Natural Science Foundation Heilongjiang Province of China under Grant LH2021F011; Fundamental Research Funds for the Central Universities of China under Grant 3072022TS0801; Key Laboratory of Advanced Marine Communication and Information Technology Open Fund under Grant AMCIT2103-03.

Notes on contributors

Haiyun Zhou

Haiyun Zhou was born in Changzhou, China, in 1980. Since 2015, she has been an Associate Professor with College of Public Security, Nanjing Forest Police College. Her research interests include image processing, computer vision, and pattern recognition, etc.

Xuezhi Xiang

Xuezhi Xiang was born in Harbin, China, in 1979. He received the B.Eng. degree in information engineering, and the M.Sc. and Ph.D. degrees in signal and information processing from Harbin Engineering University, China, in 2002, 2004, and 2008, respectively. He was a Post-Doctoral Fellow with the Harbin Institute of Technology from 2009 to 2011. From 2011 to 2012, he was a Visiting Scholar with the University of Ottawa. Since 2010, he has been an Associate Professor with the School of Information and Communication Engineering, Harbin Engineering University. He has authored over 40 articles. His research interests include image processing, computer vision, and pattern recognition, etc. Dr. Xiang is also a member of the Association for Computing Machinery and a Senior Member of the China Computer Federation.

Yujian Qiu

Yujian Qiu was born in Chongqing, China, in 1997. He received his Master degree from Harbin Engineering University China, in 2022. His research interests include image processing, computer vision and pattern recognition.

Xuzhao Liu

Xuzhao Liu was born in Harbin, China, in 1999. He received his B. Eng. degree from Harbin Engineering University China, in 2021. His research interests include image processing, computer vision and pattern recognition.