Real-time action detection and temporal segmentation in continuous video

ABSTRACT

Temporal segmentation of actions has been under intensive focus in the field of computer vision for a prolonged period. The present study proposed a template-based framework to resolve the issues concerning timeliness and real-time performance in the temporal segmentation in a continuous video. A complete action can be detected, based on the previous frames, and the action can be segmented immediately without waiting for the follow-up frames. Herein, characteristic frames are selected by a martingale-based method, followed by the formation of the corresponding motion history through backtracking along the characteristic frames, and the final segmentation is determined according to the recognition model trained by the extreme learning machine. In the experiment on the IXMAS database, the average rate of the detection of action reached 91%, and the accuracy in the frame level reached 83.5%. In the experiment on the 3D skeleton data based on Kinect, the detection rate reached 94%.

KEYWORDS:

• Action segmentation
• characteristic frames
• motion history image
• extreme learning machine

Disclosure statement

All the authors have no conflict of interest.

Notes on contributors

Xueping Liu was born in 1979. He is a doctoral candidate. His main research directions are action analysis and machine learning.

Yibo Li was born in 1963. He is a doctor, doctoral student supervisor. His main research directions are image processing, pattern recognition, and flight control.

Qing Shen was born in 1988. He is a doctoral candidate. His main research directions are action analysis and machine learning.

Additional information

Funding

This work was supported by the National Nature Science Foundation of China (Nos. 61103123, 61203170).