Recognition of Indian Sign Language Using ORB with Bag of Visual Words by Kinect Sensor

ABSTRACT

Recognition of Indian Sign Language (ISL) could bridge the gap between deaf-mute people and society. Hand recognition is a key requirement for ISL recognition system. In this paper, the hand region is segmented from the depth image using the Microsoft Kinect Sensor in the cluttered environment. The depth image obtained is then used to implement supervised machine learning by extracting and training the features of images. Here, by comparing various methods, it is depicted that ORB (Oriented FAST and Rotated BRIEF) outruns others in terms of accuracy. ORB is invariant to scale, rotation, and lighting conditions. ORB is also fused with various classification techniques to gain the optimum result. The method is applied to images of ISL 0–9 and is also compared with some standard datasets. Tuning of the ORB with k-NN classification produces an average recognition accuracy of 93.26% with ISL dataset.

KEYWORDS:

• Bag of visual words
• Indian sign language (ISL)
• Kinect sensor
• k-NN
• ORB

Additional information

Notes on contributors

Jayesh Gangrade

Jayesh Gangrade received MTech degree with honours in computer science and engineering from MANIT Bhopal 2008. He has worked as a project engineer in a research center in Wipro Technology Bangalore. He is currently a PhD candidate in the Department of Computer Science and Engineering at the Maulana Azad National Institute of Technology, Bhopal, India. His research interests are in the areas of computer vision, image processing, and pattern recognition.

Jyoti Bharti

Jyoti Bharti is currently working as an assistant professor in the Department of Computer Science and Engineering, MANIT Bhopal. She earned his PhD in image processing at Maulana Azad National Institute of Technology Bhopal India. She has 15 years of teaching experience; her research interests include image processing, computer vision, and biometrics. She published more than 20 articles in a refereed journal. Email: [email protected]

Anchit Mulye

Anchit Mulye is currently pursuing BE degree in computer science & engineering from Institute of Engineering and Science, IPS Academy, Indore. His research interests are in the areas of machine learning, computer vision, and data analytics. Email: [email protected]