Recognition of Japanese finger spelling gestures using neural networks

Abstract

Effective communication with the hearing and speech impaired often requires at least a basic working knowledge of sign language gestures, without which a memo pad and pen, or a mobile phone's notepad is indispensable. The aim of this study was to build a neural network that could be used to recognize static finger-hand gestures of the yubimoji, the Japanese sign language syllabary. To build the network, signal inputs from a data glove interface were taken for each of the static yubimoji gestures. The network was trained and tested 10 times using a multilayer perceptron model. Overall, only 18 of the 41 static gestures were successfully recognized. One of the reasons was attributed to the inability of the data glove to measure gesture directions particularly for yubimoji gestures with similar finger configurations. Future work will focus on these problems as well as the inclusion of dynamic yubimoji gestures.

Keywords:

• Yubimoji (Japanese finger spelling)
• Neural network
• Multilayer perceptron
• Mean squared error
• Linear correlation coefficient

Acknowledgments

The authors wish to acknowledge the cooperation of Ms Kaori Fukino of the Gunma (Japan) Federation of the Hearing Impaired.

Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the paper.