Comprehensible instructions from assistive robots for older adults with or without cognitive impairment

ABSTRACT

The purpose of this study was to reveal comprehensible instructions from an assistive robot for older adults, across cognitive levels and characteristics. Participants included 19 older adults with or without cognitive impairment. We administered cognitive tests assessing all major domains (e.g., memory and attention). Participants were required to listen to robot instructions carefully, and perform three activities of daily living (e.g., taking medicine) with three different types of instructions. In instruction pattern 1 (IP1), the robot informed seniors of the task in one sentence, while in instruction patterns 2 and 3 (IP2 and IP3), the steps of each activity were split into two and three sentences, respectively. Participants with lower cognitive level showed lower task performance with IP1, whereas almost all participants completed tasks with IP2 and IP3. Cognitive domains such as working memory significantly affected task performances. Participants with lower attention made mistakes in taking their medicine. The results imply that step-by-step instructions should be used for older people with lower levels of cognitive function, especially working memory, and repeated instructions may be required for lower attention. Types of instruction should be selected depending on cognitive characteristics.

KEYWORDS:

• activities of daily living
• cognitive impairment
• older adults
• robotics

Introduction

The number of older adults with mild cognitive impairment (MCI) and dementia is rapidly increasing with aging global population (Alzheimer’s Association, 2020; Hu et al., 2016). According to the World Health Organization (2018), dementia is one of the main causes of dependency, and long-term care systems need to be developed in all countries. In Japan, more than 6 million older adults had support needs certification in long-term care insurance system, and the number of persons needing long-term care has dramatically increased in recent years (Japanese Ministry of Health, Labour and Welfare, 2016). The shortage of care workers is one of the major issues in many countries in such critical times.

As a solution to this, assistive technologies such as robots, aimed at maintaining the independent living of older people, have been developed in recent years (Nishiura et al., 2019; Smith & Astell, 2017). Assistive technologies are reported to have a high potential for promoting independence when utilized effectively (Chaurasia et al., 2016; Inoue et al., 2012), increasing the quality of life (Orpwood et al., 2010, 2007), reducing caregiver burden (Czarnuch & Mihailidis, 2011; Wang et al., 2017), and cutting cost of care (O’Keefe et al., 2010). In terms of assistive robots, Wang et al. (2017) revealed that tele-operated support robots assisted in activities of daily living (ADL) and reduced care burden in older adults with mild-to-moderate dementia. Additionally, a robot with a head, camera, microphones, two arms for making gestures, etc. was perceived as particularly useful in providing reminders and safety checks, companionship, reassurance, and reducing care burden (Law et al., 2019). Thus, assistive robots are evidently effective to support lives for older people with cognitive impairment.

However, evidence of the effectiveness and adaptability of assistive robots is still limited. There were no discussions about what instructions and how approaches from assistive robots are suitable for people with dementia. Dementia is a syndrome with multiple causes like Alzheimer’s disease (AD), dementia with Lewy bodies (DLB), etc. (Gale et al., 2018), and is characterized by symptoms such as memory deficit, attention deficit, disorientation (Fujii et al., 2018; Little, 2018). In addition, severity of symptoms changes easily (Lorenz et al., 2017). As a result, older people could use assistive robots for unmet needs and cognitive functions for some time, and could not use them for a long period with positive effects (Kenigsberg et al., 2019; Lorenz et al., 2017). Therefore, it is important that robot instructions are understandable based on individual cognitive characteristics and levels.

In the present study, we tried to reveal easily understandable instructions from an assistive robot for older adults to perform ADL. The main purpose of the present study was to clarify the comprehensible verbal instructions from a robot depending on cognitive levels and characteristics of cognition. The research goal was to adapt assistive robots to individuals for sustainable use.

Materials and methods

Study design

A cross-sectional design was used to establish the appropriate level of instructions from a robot. We focused on cognitive levels and domains in older adults.

Robot used in experiments

The communication robot called “PaPeRo” was used (see Figure 1). Inoue et al. (2012) stated that “the robot is equipped with speech recognition, speech synthesis, facial image recognition, autonomous mobility, head motion, light indication, and tactile sensors.” Its key functions are to provide required information to users, to attract attention, to prompt actions/activities, and to communicate (Inoue et al., 2012). It was developed as a platform for further research.

Figure 1. The information support robot, “PaPeRo” (NEC). Colors are visible in the online version of the article (Inoue et al., 2012); http://dx.doi.org/10.3233/TAD-120357.

Procedure

In the experiment, PaPeRo asked the participants to do three daily activities: “Task A,” to take medicine, “Task B,” to measure blood pressure, and “Task C,” to clean up the experiment room. There were 3 different types of instructions for each task. In the instruction pattern 1 (IP1), PaPeRo informed subjects of the tasks in one sentence. For example, it said, “Please take daytime medicine,” in Task A. On the other hand, in instruction patterns 2 and 3 (IP2 and IP3), the processes of the activities were split into two and three steps, respectively (see Table 1). One experiment session consisted of the participant performing the tasks A through C with IP1, IP2 or IP3 in 1 day. All participants received three experiment sessions (three days within one week) with all instruction patterns. The order of tasks and instruction patterns were selected randomly (Nishiura et al., 2014). Therefore, a total of nine data (3 tasks with 3 IPs) in each participant was taken. Because PaPeRo was not able to work autonomously as a research platform, the examiner controlled PaPeRo, and also observed the behavior of the participants.

Table 1. The instruction patterns (IPs) of the robot.

	Task A	Task B	Task C
IP1	1. Please take daytime medicine.	1. Please measure blood pressure and record its value on the paper.	1. Please clean up the room according to the sign.
IP2	1. At first, please take battled water from the fridge, and pull it to the glass. 2. Next, please pick up daytime medicine and take it.	1. At first, please wrap the cuff on your upper arm, and push a start button. 2. Next, please record the blood pressure value on the paper.	1. At first, please move the tea pot and the tea cup to the kitchen Table 2. Next, please hang up the jacket, fold up pants, and put them back in the closet.
IP3	1. At first, please take bottled water from the fridge. 2. Next, please pull water to the glass. 3. Finally, please pick up daytime medicine and take them.	1. At first, please wrap the cuff on your upper arm. 2. Next, please push a start button. 3. Finally, please record the blood pressure value on the paper.	1. At first, please move the tea pot and the tea cup to the kitchen Table 2. Next, please hang up the jacket and put it back in the closet. 3. Finally, please, fold up pants and put them back in the closet.

The robot simply informed in IP1. On the other hand, it presented one by one in the IP2 and IP3 respectively according to this table.

The experiment took place in a residential room (Figure 2). PaPeRo was placed at a fixed position in the middle of a small table in front of the participants. A medicine box, freely soluble candies (alternative pills), an apparatus for measuring blood pressure, a piece of paper to record the blood pressure value, a pencil, a tea-pot, and a tea-cup were placed on the other table, and one jacket and two pants were placed on the bed.

Figure 2. The layout of the experiment room. The participant was always required to have a seat in front of the “PaPeRo” in order to carefully listen to what the robot talked. The experimental goods such as the medicine box and the apparatus for measuring blood pressure were placed on the table. Similarly, the jacket and the two pants were put on the bed.

Participants were briefed about PaPeRo and the experimental procedure. First, the examiner explained that the participants should answer PaPeRo when its ears turn red, because this indicates that it has entered preparatory mode for voice recognition. Participants were also required to carefully listen to instructions, answer “yes” if they understood what PaPeRo said, and perform the tasks based on the instructions. The participants were told not to ask the examiner any questions except in emergency situations.

Participants

In total, 19 older people with or without dementia (4 men and 15 women, mean age: 81.5 ± 5.8 (SD) years, range: 71–94 years) were invited to participate in paid nursing homes in an urban community. Two of them had AD, and one had DLB. Based on the Mini-Mental State Examination (MMSE) (Folstein et al., 1975), other two participants were considered to have mild cognitive impairment (MCI), though the diagnosis was not histopathologically confirmed. The remaining 14 participants were healthy older adults who scored more than the MMSE cutoff (24 points). All lived independently, and received no physical care. They had no difficulty in hearing in their daily lives, and were able to orally communicate.

The protocol of the study was explained to the participants and their families and was presented in a written document. One family member signed an informed consent form for participation in the study in case the participant was diagnosed with dementia.

Assessments

MMSE and Neurobehavioral Cognitive Status Examination (COGNISTAT) (Kiernan et al., 1987) were performed just before experiments. MMSE is used to evaluate cognitive function using a scale ranging from 0 to 30, with lower scores indicating more severe impairment. COGNISTAT is a cognitive assessment tool used to identify people with cognitive impairment. It assesses 10 domains of cognitive functioning: orientation, attention, language comprehension, repetition, naming, constructions, memory, calculations, reasoning similarities, and judgment.

Statistical analysis

The process of completing each task was split into steps. Task A took seven steps to complete. Similarly, Task B was divided into five steps, while Task C was divided into seven steps (see Table 2). The examiner determined the performance rate (PR: %), which was the number of completed steps divided by the total number of steps.

Table 2. Each task was separated in several steps.

	Task A	Task B	Task C
Processes of tasks	1. Taking the bottled water from the fridge, 2. Pulling water to the glass, 3. Finding the medicine box, 4. Opening the medicine box, 5. Selecting the daytime cell, 6. Picking up the daytime pills, 7. Drinking the daytime pills.	1. Selecting the apparatus for measuring blood pressure, 2. Wrapping the manchette around her upper arm, 3. Pushing the start button, 4. Reading the measured result, 5. Writing the values down on the indicator paper.	1. Finding the instructing sign for clearing the experiment room, 2. Silent reading the sign, 3. moving the tea pot and the tea cup to the kitchen table, 4. Hanging up the jacket, 5. putting the jacket back in the closet, 6. Folding up the two pants, 7. putting the pants back in the closet.

Task A and Task C were 7 steps, whereas Task B was 5 steps.

In terms of the relationship between cognitive level and task performance, correlation of MMSE total score with PR in each task was analyzed using the Spearman’s rank-correlation coefficient. To identify the appropriate IP according to the characteristics of cognitive function, multiple regression analysis was used to predict PR based on each score in COGNISTAT. A p-value of less than 0.05 was considered statistically significant. IBM SPSS Statistics 22.0 (IBM Corporation, Armonk, NY, USA) was used for the analyses.

Ethical considerations

This study was approved by the Ethical Committee of the National Rehabilitation Center for Persons with Disabilities prior to commencing the research (C25143). All participants provided written consent before data collection.

Results

All participants completed all the tasks. They showed positive motivation to interact with PaPeRo and perform tasks. Over 15 participants showed a PR of 100% in tasks A through C with IP2 and IP3, and the mean of PR was more than 90.0 % (see Table 3). Therefore, only the results of tasks A through C performed with IP1 were used for analyses.

Table 3. It shows the mean of PR (%) in each task and instruction (n = 19).

	Task A	Task B	Task C
IP1	72.6 ± 21.3	77.6 ± 35.3	58.8 ± 35.7
IP2	96.8 ± 10.0	92.1 ± 16.8	89.5 ± 16.8
IP3	97.9 ± 6.3	98.7 ± 5.7	91.6 ± 18.0

Most participants completed tasks with IP2 and IP3.

Cognitive level and task performance

The MMSE total scores were significantly positively correlated with PR in tasks B and C (Spearman’s rank correlation coefficient (rs) Task B – rs = 0.78, p = 0.00088; Task C – rs = 0.58, p = 0.014), while there was no correlation between MMSE total scores and PR in Task A (Task A – rs = 0.31, p = 0.24) (see Figure 3). This indicated that participants with lower cognitive levels showed lower task performance in tasks B and C with IP1, while almost all participants completed tasks with IP2 and IP3.

Figure 3. Scatter plot for correlation between MMSE points and PR with IP1. There was a significant positive correlation in Tasks B and C, while there was no correlation in Task A.

• = 1 person, ° = 2 persons, ▲ = 4 persons, ● = 5 persons.

Cognitive characteristics and task performance

Multiple regression analysis with factors (each item in COGNISTAT) as dependent variable (PR) showed that repetition and judgment significantly exerted positive effects on task performances (PR) in Task A (95% Confidence Interval [CI] = 6.58 to 60.74, p = 0.021 for repetition; 95% CI = – 30.75 to – 2.90, p = 0.024 for judgment). Similarly, constructions showed significant effect on PR in Task C (95% CI = 0.21 to 32.0, p = 0.048) (see Table 4). In terms of Task B, regression expression was valid for prediction (p = 0.048) (see Figure 4).

Table 4. Summary of multiple regression analysis for variable predicting PR with IP1 for each task (n = 19).

	Task A			Task B			Task C
Variable	B	SE B	β	B	SE B	β	B	SE B	β
Orientation	4.04	2.91	0.50	4.72	3.43	0.35	−0.44	5.03	−0.032
Attention	−4.79	3.18	−0.67	2.97	3.74	0.25	−4.31	5.49	−0.36
Language comprehension	−8.77	4.25	−1.19	6.48	5.00	0.53	2.68	7.35	0.22
Repetition	33.66	11.74	3.42*	3.80	13.82	0.23	9.28	20.32	0.56
Naming	−29.79	13.17	−1.82	−6.87	15.50	−0.25	−2.57	22.78	−0.093
Constructions	−0.018	3.99	−0.001	23.54	4.70	0.14	16.13	6.90	0.65
Memory	−3.72	4.17	−0.30	11.12	4.91	0.54	3.97	7.21	0.19*
Calculations	37.65	20.21	0.81	−19.82	23.79	−0.26	−12.92	34.96	−0.17
Reasoning similarities	6.68	6.67	0.28	2.80	7.85	0.071	−3.94	11.54	−0.099
Judgment	−16.8	6.04	−0.93*	−9.47	7.11	−0.32	0.34	10.45	0.11
R^2		0.68			0.84			0.66
F		1.70			4.14*			1.54

*p < 0.05.

Figure 4. It shows the scatter plot for multiple regression analysis in Task B with IP1.

Two participants with lower scores of attention in COGNISTAT made a mistake in Task A, although their MMSE points were more than the cutoff value. PaPeRo had asked them to take daytime pills, but instead they took morning pills.

Discussion

The first result showed that people with a lower cognitive level showed lower performances with one sentence instructions (IP1). Second, the domains of cognitive functioning such as repetition, judgment, constructions, and attention affected performance in each task.

The major finding of this study indicated that older people with cognitive deficits might easily understand what to do next if a task is divided into steps and instructed by PaPeRo. It is almost consistent with the previous reports that one verbal instruction in a single step enabled persons with mild or moderate Alzheimer’s disease to perform ADL (Lancioni et al., 2008, 2012). Additionally, Stanyon et al. (2016) reported that health care workers tend to try to speak with reduced speech complexity, in short sentences, and not using abstract ideas. Thus, technical verbal instruction and communication is very important for people with dementia to take action, and increase communicative behavior (Small et al., 2003; Stanyon et al., 2016; Wells et al., 2000). In this study, we found out that communication robots also need to use step-by-step instructions for people with cognitive disfunction, as along with daily conversations in health care settings.

The secondary finding showed that repetition and judgment significantly affected performance in Task A. Repetition is mainly influenced by hearing function and working memory (Kiernan et al., 1987). In this study, the participants with no hearing impairment were required to listen to the PaPeRo carefully, consider what to do, and perform the tasks, wherein working memory was needed above all other cognitive functions. This finding was supported by a Liu et al. (2019) study which reported that higher deficit of working memory reduced sentence comprehension for both healthy older people and people with AD. Moreover, working memory and judgment are also components of executive function, which affects ADL in the early stages of dementia (Martyr & Clare, 2012; Royall et al., 2002). Therefore, the participants with lower working memory (repetition in COGNISTAT) and judgment might have had lower executive function, which would have led to failures in completing Task A (main ADL among older adults). Thus, our study revealed not only that the executive function was important to perform ADL but also assistive robots are able to fill in gaps by using step-by-step instructions like IP2 and IP3. In terms of Task B, there were no significant affected items in COGNISTAT, but the regression formula was valid. Because measuring blood pressure was complicated task with using tools, global cognitive function might be needed.

Another interesting finding was that two healthy older adults with low scores of attention made mistakes in selecting the medicine. Getzmann et al. (2014) reported that deficits in attention lead to difficulties in speech perception in noisy situations. The most important finding in this study was that older people with attention deficit might make mistakes in their daily lives even though they are healthy older adults judged by cognitive assessment. We emphasized information should be repeated to enable people with lower attention to acquire correct information (Inoue et al., 2012; Nishiura et al., 2014).

In this study, there are some limitations. First, we collected the data from older persons both with or without dementia. People with or without dementia might have different attitudes to the assistive robot and lead different outcomes. Second, a sample size might be too small to apply the present study to a larger population. In a future study, we will try multicenter studies and analyze the results by dividing groups according to with or without dementia.

In conclusion, step-by-step instructions should be used for older people with lower levels of cognitive function, especially working memory, and repeated instructions may be required for lower attention. Types of instruction should be selected depending on cognitive characteristics.

Declaration of interest statement

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Acknowledgments

The authors are grateful to all participants and supporting staffs in Seikatsu Kagaku Un-Ei Co., Ltd. We disclosed receipt of the following financial support for the research, authorship, and/or publication of this article; JST “Strategic Promotion of Innovation Research and Development” Grant Number JPMJSV1011, Japan. The authors confirm that the data supporting the findings of this study are available within the article and its supplementary materials.

Additional information

Funding

This work was partly supported by the JST “Strategic Promotion of Innovation Research and Development” under Grant Number [JPMJSV1011] and JSPS KAKENHI Grant Number JP19KT0004.