Measurement of neuropsychiatric symptoms in the older adults with mild cognitive impairment based on speech and facial expressions: a cross-sectional observational study

Abstract

Objectives

To examine the association between speech and facial features with depression, anxiety, and apathy in older adults with mild cognitive impairment (MCI).

Methods

Speech and facial expressions of 319 MCI patients were digitally recorded via audio and video recording software. Three of the most common neuropsychiatric symptoms (NPS) were evaluated by the Public Health Questionnaire, General Anxiety Disorder, and Apathy Evaluation Scale, respectively. Speech and facial features were extracted using the open-source data analysis toolkits. Machine learning techniques were used to validate the diagnostic power of extracted features.

Results

Different speech and facial features were associated with specific NPS. Depression was associated with spectral and temporal features, anxiety and apathy with frequency, energy, spectral, and temporal features. Additionally, depression was associated with facial features (action unit, AU) 10, 12, 15, 17, 25, anxiety with AU 10, 15, 17, 25, 26, 45, and apathy with AU 5, 26, 45. Significant differences in speech and facial features were observed between males and females. Based on machine learning models, the highest accuracy for detecting depression, anxiety, and apathy reached 95.8%, 96.1%, and 83.3% for males, and 87.8%, 88.2%, and 88.6% for females, respectively.

Conclusion

Depression, anxiety, and apathy were characterized by distinct speech and facial features. The machine learning model developed in this study demonstrated good classification in detecting depression, anxiety, and apathy. A combination of audio and video may provide objective methods for the precise classification of these symptoms.

Keywords:

• Neuropsychiatric symptoms
• mild cognitive impairment
• facial analysis
• speech analysis
• machine learning

Acknowledgments

The authors would like to thank Professor Liming Zhang for facilitating data collection. We also thank Liangwei Xu for his assistance with data processing. The authors would like to thank all participants in this research.

Authors’ contributions

Zhou Ying: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Visualization, Resources, Software, Validation, Writing - original draft. Xiuyu Yao: Formal analysis, Writing - Original Draft. Wei Han: Methodology, Formal analysis, Visualization. Yingxin Li: Conceptualization, Methodology, Supervision. Jiajun Xue: Investigation, Project Administration. Zheng Li: Conceptualization, Data curation, Formal analysis, Methodology, Project administration, Supervision, Funding acquisition, Resources, Writing - review & editing.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

This work was supported by the Peking Union Medical College Research Fund [grant number: PUMCSON201901], and the Graduate Student Innovation Fund of Peking Union Medical College [grant number: 2019-1002-93].