210
Views
1
CrossRef citations to date
0
Altmetric
Research papers

A novel key-frame selection-based sign language recognition framework for the video data

, , , & ORCID Icon
Pages 156-169 | Received 16 Mar 2019, Accepted 07 May 2020, Published online: 18 Jun 2020
 

ABSTRACT

Sign language is a medium of communication for people with hearing disabilities. Static and dynamic gestures are identified in a video-based sign language recognition and translated them into humanly understandable phrases to achieve the communication objective. However, videos contain redundant Key-frames which require additional processing. Number of such Key-frames can be reduced. The selection of particular Key-frames without losing the required information is a challenging task. The Key-frame extraction algorithm is used which helps to speed-up the sign language recognition process by extracting essential key-frames. The proposed framework eliminates the computation overhead by picking up the distinct Key-frames for the recognition process. Discrete Wavelet Transform (DWT), Discrete Cosine Transform (DCT), and Histograms of Oriented Gradient (HOG) are used for unique features extraction. We used the bagged tree, boosted tree ensemble method, Fine KNN, and SVM for classification. We tested methodology on video-based datasets of Pakistani Sign Language. It achieved an overall 97.5% accuracy on 37 Urdu alphabets and 95.6% accuracy on 100 common words.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes on contributors

Fakhar Ullah Mangla is working as an assistant professor in university of Sargodha, Pakistan. He did his MS in computer science from International Islamic University, Islamabad, Pakistan. His area of research includes Machine Learning, Image Processing and Computer Vision. He is author of several research articles published in reputed peer reviewed journals.

Aysha Bashir has recently completed her MS in computer science from University of Sargodha, Pakistan and is teaching at BS level in the department of computer science. Here area of research includes Machine Learning, Image Processing and computer vision.

M. Ikram Lali is working as a Professor at the Department of Information Sciences, University of Education Lahore, Pakistan. He obtained Masters in Software Engineering and Ph.D. Computer Science degrees from COMSATS, Islamabad, Pakistan. He has been visiting research fellow at University of Groningen Netherlands during his PhD under RuG Fellowship. He has served at Minintry of IT, University of Gujrat and University of Sargodha, Pakistan for several years. He is collaborating with different research groups at national and international level. He is senior member ACM. He is author of more than 70 research articles which have been published in conferences and reputed journals. He is reviewer of many high impact factor journals and reputed conferences. His areas of interests include machine learning, social network data analysis, Image Processing and Computer Vision.

Dr. Ahmad Chan Bukhari is as an Assistant Professor and Director of Healthcare Informatics at St. John's University, New York. He received his Ph.D. in Computer Science from the University of New Brunswick, Canada and then went on to complete his postdoctoral fellowship at Yale School of Medicine where he worked with Stanford University Center of Expanded Data Annotation and Retrieval (CEDAR) to develop data submission pipelines to improve scientific experimental reproducibility. His current research efforts are concentrated on addressing several core problems in the area of healthcare informatics and data science. He particularly focuses on devising techniques to semantically confederate heterogeneous biomedical data and to further develop clinical predictive models for diseases predictions. These techniques further alleviate many data access-related challenges faced by healthcare providers. Dr. Bukhari is a senior IEEE member and a distinguished ACM speaker who serves as an editorial board member of multiple scientific journals. In September 2019, he has been awarded with the IEEE Technological Innovation Award. His research work has been published in top-tier journals and picked by various scientific blogs and international media.

Dr Basit Shahzad received his M.Sc. degree from the National University of Science and Technology (NUST), Islamabad, Pakistan, and Ph.D. degree from University Technology Petronas, Malaysia. Dr Shahzad was a Postdoctoral Researcher at the Computer Laboratory, University of Cambridge, U.K. Dr Shahzad has served as an Assistant Professor at COMSATS Institute of Information Technology, Islamabad and at King Saud University, Riyadh. He is currently a Visiting Scientist at the University of Cambridge and a Visiting Fellow of Macquarie University, Australia. He is a Collaborating Researcher with the Hagenberg Centre for Software Competence, Austria. Dr Shahzad is currently working as Assistant Professor of Software Engineering at the National University of Modern Languages, Islamabad. Dr Shahzad has numerous publications in journals and at conferences of international repute and has a very active research profile. He has editorial role in several conferences and journals of high repute and has edited a number of special issues in the areas of software engineering, social networks, and mobile healthcare. His research and teaching career spans over 16 years. Dr Shahzad is a Reviewer of several high impact journals. He serves on the Program Committee of several distinguished conferences.

Reprints and Corporate Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

To request a reprint or corporate permissions for this article, please click on the relevant link below:

Academic Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

Obtain permissions instantly via Rightslink by clicking on the button below:

If you are unable to obtain permissions via Rightslink, please complete and submit this Permissions form. For more information, please visit our Permissions help page.