System Development and Evaluation of Human–Computer Interaction Approach for Assessing Functional Impairment for People with Mild Cognitive Impairment: A Pilot Study

Abstract

Non-pharmacological treatments have gained significant attention in the field of cognitive impairment. Among them, human–computer interaction-based (HCI) methods have emerged as a promising approach due to their broad applicability and convenience in assessing symptoms associated with this progressively debilitating condition. However, existing rehabilitation training systems for cognitive impairment lack effective assessment methods to meet the diverse rehabilitation needs of users. In this article, we surveyed existing HCI-based cognitive rehabilitation training systems and analyzed their advantages and shortcomings. Drawing from the insights gained from these systems, we propose a novel Leap Motion-based building block training system that incorporates system software capable of generating highly realistic virtual scenes, with the added capability of user behavior detection using Kinect. We conducted user testing of this new system, comparing the performance of a representative cohort with mild cognitive impairment (MCI) (n = 9) to that of disease-free participants (n = 10). Additionally, we conducted ergonomic experiments to assess the system’s performance in elderly people. The experimental results revealed significant differences between the MCI cohort and the control cohort. Specifically, the MCI cohort exhibited a reduced range of motion and longer task completion times compared to the control cohort. These findings have the potential to contribute to the differentiation of cognitive levels. In conclusion, our analysis of existing cognitive rehabilitation training systems provides valuable insights for researchers working on the development of future innovative cognitive rehabilitation training systems and enriches the non-pharmacological treatment models for cognitive impairment. Furthermore, the observed relationship between behavioral data, task completion times, and cognitive levels in older adults offers useful insights for the design of HCI-based approaches for diagnosing and assessing the treatment of MCI.

Keywords:

• Mild cognitive impairment
• cognitive training
• human–computer interaction
• non-pharmacological treatment
• hand–brain coordination
• puzzle game

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This project was supported by the National Natural Science Foundation of China (Grant No. 52205275) and the Natural Science Foundation of Shandong Province (ZR2022QE297).

Notes on contributors

Tian Su

Tian Su is an undergraduate at the School of Software, Shandong University. Her main research interests include human–computer interaction and computer graphics.

Zixing Ding

Zixing Ding is an undergraduate at the School of Software, Shandong University. Her main research interests include human–computer interaction.

Lizhen Cui

Lizhen Cui is a professor and the Chair of the School of Software. He is also the Co-Chair of the Joint SDU-NTU Centre for Artificial Intelligence Research (C-FAIR), Shandong University. His research interests include big data management and analysis, AI theory, and application.

Lingguo Bu

Lingguo Bu is a Professor at Shandong University. He has experience as a Postdoctoral Research Fellow at Nanyang Technological University, Singapore. His main research interests include human–computer interface, neuroergonomics, and industrial design.