A concept for emotion recognition systems for children with profound intellectual and multiple disabilities based on artificial intelligence using physiological and motion signals

Abstract

Purpose

This study proposes a concept for emotion recognition systems for children with profound intellectual and multiple disabilities (PIMD) based on artificial intelligence (AI) using physiological and motion signals.

Methods

First, the heartbeat interval (R–R interval, RRI) of a child with PIMD was measured, and the correlation between the RRI and emotion was briefly tested in a preliminary experiment. Then, a concept based on AI for emotion recognition systems for children with PIMD was created using physiological and motion signals, and an emotion recognition system based on the proposed concept was developed using a random forest classifier taking as inputs the RRI, eye gaze, and other data acquired using low physical burden sensors. Subsequently, the developed emotion recognition system was evaluated, validating the proposed concept. Finally, we proposed a validated concept for emotion recognition systems.

Results

A correlation was found between the RRI and emotion. The emotion recognition system was created based on the proposed concept and tested. According to the results, the recognition rate of “negative” and “not negative” of 70.4% ± 6.1% (Mean ± S.D.) of the developed emotion recognition system was higher than 48.5% ± 5.0% of an unfamiliar person used as a control.

Conclusion

The results indicate that the proposed concept for emotion recognition systems is useful for communicating with children with PIMD.

IMPLICATIONS FOR REHABILITATION

• A new concept based on artificial intelligence for emotion recognition systems for children with profound intellectual and multiple disabilities (PIMD) using physiological and motion signals is proposed.

• An emotion recognition system based on the proposed concept developed using a random forest classifier taking as inputs the heartbeat interval, eye gaze, and other data acquired using low physical burden sensors were tested in terms of the emotion recognition rate.

• The recognition rate of “negative” and “not negative” of the developed system (i.e., 70.4% ± 6.1%) is higher than that of an unfamiliar person (i.e., 48.5% ± 5.0%).

• The proposed concept for emotion recognition systems may be useful for communicating with children with PIMD.

Keywords:

• Children with profound intellectual and multiple disabilities
• emotion recognition
• heartbeat interval
• artificial intelligence
• physiological signal
• motion signal
• communication assist system

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by JSPS KAKENHI Grant Number [19K11330].