Modelling Facial Features for Emotion Recognition and Synthesis

ABSTRACT

Emotion recognition and synthesis is one of the most important challenges for effective human–computer interaction. In this paper, a novel approach for emotion recognition is presented by modelling facial feature deformations. The presented approach is based on the fact that facial features such as lip, nose, eyes, and eyebrows get deformed due to variation in emotions. To measure the change in shapes of different facial features, landmark points are extracted around the facial features. Thin plate spline (TPS) is used to model the deformation of these landmark points. The basic property of TPS mapping function is that it is capable of computing rigid as well as non-rigid transformations between neutral and emotion image frames. The rigid transformation parameters represent affine parameters caused by head movement and non-rigid transformation parameters are used as representatives of facial feature deformation caused by emotion. To prove the modelling ability of TPS, non-rigid parameters are fed to support vector machine for emotion recognition. Moreover, an attempt is made to synthesize emotion by using TPS warping function. The mean of non-rigid transformation for an emotion is used as a template to warp the neutral image to emotion image. To evaluate the proposed approach, extended Cohn-Kanade database and JAFFE database are used and experimental results show 95% and 70% accuracy for them, respectively.

KEYWORDS:

• Emotion recognition
• Facial expression recognition
• Facial feature points
• Human-Computer Interaction
• Support vector machine
• Thin Plate Spline

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Notes on contributors

Neeru Rathee

Neeru Rathee is working on facial expression analysis from GGSIPU and has received her M.Tech. from DU. She is currently working with Maharaja Surajmal Institute of Technology, C-4, Janakpuri, New Delhi. Her research areas include image processing, computer vision, and artificial intelligence.

E-mail: [email protected]

Dinesh Ganotra

Dinesh Ganotra is currently with Indira Gandhi Delhi Technical University for Women (Formerly Indira Gandhi Institute of Technology). His research areas include panorama vision, speech processing, and image processing.

E-mail: [email protected]