A Dual Stream Model for Activity Recognition: Exploiting Residual- CNN with Transfer Learning

ABSTRACT

Visual content has a protagonist role in this age of data revolution. These days, computer vision research community is fascinated towards application of convolution neural networks and transfer learning for various image and video analysis tasks. Residual connection in CNN can facilitate the training process in the deep networks. This paper investigates and uses deep residual networks with fusion based dual stream pre-trained models for activity recognition from video streams. The architecture is further trained and evaluated using standard video actions benchmarks of UCF-101, HMDB-51 and NTU RGB. Performance of depth-based variants of residual networks is also analysed. The proposed approach not only provides competitive results but also better at exploiting pre-trained model and annotated image data.

KEYWORDS:

• Activity recognition
• convolution networks
• residual networks
• spatio-temporal features
• transfer Learning

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Notes on contributors

Roshan Singh

Roshan Singh received M.Tech degree in Information Technology from the Guru Gobind Singh Indraprastha University, Delhi, India in 2009. He Completed his M.Sc. degree in Computer Science from University of Allahabad, Allahabad, India, in 2007. He has around 5 years of Teaching and 5 years of System Administration Experience. Currently, he is a Ph.D. Research Scholar in the Department of Computer Engineering at the Indian Institute of Technology (Banaras Hindu University), Varanasi (U.P.) India. He has been working on Image processing and Video Processing. His research area includes video segmentation, classification, tracking and human activity analysis in videos.

Rajat Khurana

Rajat Khurana has done M.Tech from the Department of Computer Science and Engineering at the I.K. Gujral Punjab Technical University Main Campus, Kapurthala, Punjab, India. He received his B.Tech. Degree in Computer Science & Engineering from Guru Nanak Dev University Amritsar in 2017. Currently, he is Assistant Professor in Department of Computer Science and Engineering at the IKG Punjab Technical University, Kapurthala, Punjab India. He has been working on computer vision and machine learning. His research includes activity recognition, classification and object tracking.

Alok Kumar Singh Kushwaha

Alok Kumar Singh Kushwaha is working as Associate Professor in the Department of Computer Sc. & Engineering Guru Ghasidas University, Bilaspur, Chhattisgarh India. He received Ph.D. degree in Computer Science and Engineering from Indian Institute of Technology (Banaras Hindu University), Varanasi (U.P.) India in 2015 and completed his M.Tech in Computer Science from the Devi Ahilya University, Indore, India in 2011. He has worked as Asst. Professor in GLA University, Mathura, India. He is a member of Computer Society of India. He has been working on Image processing and Video Processing. His research area includes video segmentation, classification, tracking and human activity analysis in videos.

Rajeev Srivastava

Rajeev Srivastava is working as a Professor and Head in the Department of Computer Sc. and Engineering, Indian Institute of Technology (Banaras Hindu University), IIT (BHU), Varanasi, Uttar Pradesh, India. He received his B.E. in Computer Engineering from Gorakhpur University, India, his M.E. degree in Computer Technology and Applications and PhD degree in Computer Engineering both from the University of Delhi, Delhi, India. He has around 20 years of teaching and research experience. He has around more than 100 research publications to his credit. He has also authored one book and edited two books in the areas of image processing and computer vision published from internationally reputed publishers from Germany and the United States. His research interests include image processing, computer vision, pattern recognition and algorithms.