Publication Cover
Assistive Technology
The Official Journal of RESNA
Volume 34, 2022 - Issue 5
1,328
Views
1
CrossRef citations to date
0
Altmetric
Articles

Comprehensible instructions from assistive robots for older adults with or without cognitive impairment

, PhDORCID Icon, , PhD, , PhD & , PhD
Pages 557-562 | Accepted 11 Feb 2021, Published online: 03 May 2021

ABSTRACT

The purpose of this study was to reveal comprehensible instructions from an assistive robot for older adults, across cognitive levels and characteristics. Participants included 19 older adults with or without cognitive impairment. We administered cognitive tests assessing all major domains (e.g., memory and attention). Participants were required to listen to robot instructions carefully, and perform three activities of daily living (e.g., taking medicine) with three different types of instructions. In instruction pattern 1 (IP1), the robot informed seniors of the task in one sentence, while in instruction patterns 2 and 3 (IP2 and IP3), the steps of each activity were split into two and three sentences, respectively. Participants with lower cognitive level showed lower task performance with IP1, whereas almost all participants completed tasks with IP2 and IP3. Cognitive domains such as working memory significantly affected task performances. Participants with lower attention made mistakes in taking their medicine. The results imply that step-by-step instructions should be used for older people with lower levels of cognitive function, especially working memory, and repeated instructions may be required for lower attention. Types of instruction should be selected depending on cognitive characteristics.

Introduction

The number of older adults with mild cognitive impairment (MCI) and dementia is rapidly increasing with aging global population (Alzheimer’s Association, Citation2020; Hu et al., Citation2016). According to the World Health Organization (Citation2018), dementia is one of the main causes of dependency, and long-term care systems need to be developed in all countries. In Japan, more than 6 million older adults had support needs certification in long-term care insurance system, and the number of persons needing long-term care has dramatically increased in recent years (Japanese Ministry of Health, Labour and Welfare, Citation2016). The shortage of care workers is one of the major issues in many countries in such critical times.

As a solution to this, assistive technologies such as robots, aimed at maintaining the independent living of older people, have been developed in recent years (Nishiura et al., Citation2019; Smith & Astell, Citation2017). Assistive technologies are reported to have a high potential for promoting independence when utilized effectively (Chaurasia et al., Citation2016; Inoue et al., Citation2012), increasing the quality of life (Orpwood et al., Citation2010, Citation2007), reducing caregiver burden (Czarnuch & Mihailidis, Citation2011; Wang et al., Citation2017), and cutting cost of care (O’Keefe et al., Citation2010). In terms of assistive robots, Wang et al. (Citation2017) revealed that tele-operated support robots assisted in activities of daily living (ADL) and reduced care burden in older adults with mild-to-moderate dementia. Additionally, a robot with a head, camera, microphones, two arms for making gestures, etc. was perceived as particularly useful in providing reminders and safety checks, companionship, reassurance, and reducing care burden (Law et al., Citation2019). Thus, assistive robots are evidently effective to support lives for older people with cognitive impairment.

However, evidence of the effectiveness and adaptability of assistive robots is still limited. There were no discussions about what instructions and how approaches from assistive robots are suitable for people with dementia. Dementia is a syndrome with multiple causes like Alzheimer’s disease (AD), dementia with Lewy bodies (DLB), etc. (Gale et al., Citation2018), and is characterized by symptoms such as memory deficit, attention deficit, disorientation (Fujii et al., Citation2018; Little, Citation2018). In addition, severity of symptoms changes easily (Lorenz et al., Citation2017). As a result, older people could use assistive robots for unmet needs and cognitive functions for some time, and could not use them for a long period with positive effects (Kenigsberg et al., Citation2019; Lorenz et al., Citation2017). Therefore, it is important that robot instructions are understandable based on individual cognitive characteristics and levels.

In the present study, we tried to reveal easily understandable instructions from an assistive robot for older adults to perform ADL. The main purpose of the present study was to clarify the comprehensible verbal instructions from a robot depending on cognitive levels and characteristics of cognition. The research goal was to adapt assistive robots to individuals for sustainable use.

Materials and methods

Study design

A cross-sectional design was used to establish the appropriate level of instructions from a robot. We focused on cognitive levels and domains in older adults.

Robot used in experiments

The communication robot called “PaPeRo” was used (see ). Inoue et al. (Citation2012) stated that “the robot is equipped with speech recognition, speech synthesis, facial image recognition, autonomous mobility, head motion, light indication, and tactile sensors.” Its key functions are to provide required information to users, to attract attention, to prompt actions/activities, and to communicate (Inoue et al., Citation2012). It was developed as a platform for further research.

Figure 1. The information support robot, “PaPeRo” (NEC). Colors are visible in the online version of the article (Inoue et al., Citation2012); http://dx.doi.org/10.3233/TAD-120357.

Figure 1. The information support robot, “PaPeRo” (NEC). Colors are visible in the online version of the article (Inoue et al., Citation2012); http://dx.doi.org/10.3233/TAD-120357.

Procedure

In the experiment, PaPeRo asked the participants to do three daily activities: “Task A,” to take medicine, “Task B,” to measure blood pressure, and “Task C,” to clean up the experiment room. There were 3 different types of instructions for each task. In the instruction pattern 1 (IP1), PaPeRo informed subjects of the tasks in one sentence. For example, it said, “Please take daytime medicine,” in Task A. On the other hand, in instruction patterns 2 and 3 (IP2 and IP3), the processes of the activities were split into two and three steps, respectively (see ). One experiment session consisted of the participant performing the tasks A through C with IP1, IP2 or IP3 in 1 day. All participants received three experiment sessions (three days within one week) with all instruction patterns. The order of tasks and instruction patterns were selected randomly (Nishiura et al., Citation2014). Therefore, a total of nine data (3 tasks with 3 IPs) in each participant was taken. Because PaPeRo was not able to work autonomously as a research platform, the examiner controlled PaPeRo, and also observed the behavior of the participants.

Table 1. The instruction patterns (IPs) of the robot.

The experiment took place in a residential room (). PaPeRo was placed at a fixed position in the middle of a small table in front of the participants. A medicine box, freely soluble candies (alternative pills), an apparatus for measuring blood pressure, a piece of paper to record the blood pressure value, a pencil, a tea-pot, and a tea-cup were placed on the other table, and one jacket and two pants were placed on the bed.

Figure 2. The layout of the experiment room. The participant was always required to have a seat in front of the “PaPeRo” in order to carefully listen to what the robot talked. The experimental goods such as the medicine box and the apparatus for measuring blood pressure were placed on the table. Similarly, the jacket and the two pants were put on the bed.

Figure 2. The layout of the experiment room. The participant was always required to have a seat in front of the “PaPeRo” in order to carefully listen to what the robot talked. The experimental goods such as the medicine box and the apparatus for measuring blood pressure were placed on the table. Similarly, the jacket and the two pants were put on the bed.

Participants were briefed about PaPeRo and the experimental procedure. First, the examiner explained that the participants should answer PaPeRo when its ears turn red, because this indicates that it has entered preparatory mode for voice recognition. Participants were also required to carefully listen to instructions, answer “yes” if they understood what PaPeRo said, and perform the tasks based on the instructions. The participants were told not to ask the examiner any questions except in emergency situations.

Participants

In total, 19 older people with or without dementia (4 men and 15 women, mean age: 81.5 ± 5.8 (SD) years, range: 71–94 years) were invited to participate in paid nursing homes in an urban community. Two of them had AD, and one had DLB. Based on the Mini-Mental State Examination (MMSE) (Folstein et al., Citation1975), other two participants were considered to have mild cognitive impairment (MCI), though the diagnosis was not histopathologically confirmed. The remaining 14 participants were healthy older adults who scored more than the MMSE cutoff (24 points). All lived independently, and received no physical care. They had no difficulty in hearing in their daily lives, and were able to orally communicate.

The protocol of the study was explained to the participants and their families and was presented in a written document. One family member signed an informed consent form for participation in the study in case the participant was diagnosed with dementia.

Assessments

MMSE and Neurobehavioral Cognitive Status Examination (COGNISTAT) (Kiernan et al., Citation1987) were performed just before experiments. MMSE is used to evaluate cognitive function using a scale ranging from 0 to 30, with lower scores indicating more severe impairment. COGNISTAT is a cognitive assessment tool used to identify people with cognitive impairment. It assesses 10 domains of cognitive functioning: orientation, attention, language comprehension, repetition, naming, constructions, memory, calculations, reasoning similarities, and judgment.

Statistical analysis

The process of completing each task was split into steps. Task A took seven steps to complete. Similarly, Task B was divided into five steps, while Task C was divided into seven steps (see ). The examiner determined the performance rate (PR: %), which was the number of completed steps divided by the total number of steps.

Table 2. Each task was separated in several steps.

In terms of the relationship between cognitive level and task performance, correlation of MMSE total score with PR in each task was analyzed using the Spearman’s rank-correlation coefficient. To identify the appropriate IP according to the characteristics of cognitive function, multiple regression analysis was used to predict PR based on each score in COGNISTAT. A p-value of less than 0.05 was considered statistically significant. IBM SPSS Statistics 22.0 (IBM Corporation, Armonk, NY, USA) was used for the analyses.

Ethical considerations

This study was approved by the Ethical Committee of the National Rehabilitation Center for Persons with Disabilities prior to commencing the research (C25143). All participants provided written consent before data collection.

Results

All participants completed all the tasks. They showed positive motivation to interact with PaPeRo and perform tasks. Over 15 participants showed a PR of 100% in tasks A through C with IP2 and IP3, and the mean of PR was more than 90.0 % (see ). Therefore, only the results of tasks A through C performed with IP1 were used for analyses.

Table 3. It shows the mean of PR (%) in each task and instruction (n = 19).

Cognitive level and task performance

The MMSE total scores were significantly positively correlated with PR in tasks B and C (Spearman’s rank correlation coefficient (rs) Task B – rs = 0.78, p = 0.00088; Task C – rs = 0.58, p = 0.014), while there was no correlation between MMSE total scores and PR in Task A (Task A – rs = 0.31, p = 0.24) (see ). This indicated that participants with lower cognitive levels showed lower task performance in tasks B and C with IP1, while almost all participants completed tasks with IP2 and IP3.

Figure 3. Scatter plot for correlation between MMSE points and PR with IP1. There was a significant positive correlation in Tasks B and C, while there was no correlation in Task A.

• = 1 person, ° = 2 persons, ▲ = 4 persons, ● = 5 persons.
Figure 3. Scatter plot for correlation between MMSE points and PR with IP1. There was a significant positive correlation in Tasks B and C, while there was no correlation in Task A.

Cognitive characteristics and task performance

Multiple regression analysis with factors (each item in COGNISTAT) as dependent variable (PR) showed that repetition and judgment significantly exerted positive effects on task performances (PR) in Task A (95% Confidence Interval [CI] = 6.58 to 60.74, p = 0.021 for repetition; 95% CI = – 30.75 to – 2.90, p = 0.024 for judgment). Similarly, constructions showed significant effect on PR in Task C (95% CI = 0.21 to 32.0, p = 0.048) (see ). In terms of Task B, regression expression was valid for prediction (p = 0.048) (see ).

Table 4. Summary of multiple regression analysis for variable predicting PR with IP1 for each task (n = 19).

Figure 4. It shows the scatter plot for multiple regression analysis in Task B with IP1.

Figure 4. It shows the scatter plot for multiple regression analysis in Task B with IP1.

Two participants with lower scores of attention in COGNISTAT made a mistake in Task A, although their MMSE points were more than the cutoff value. PaPeRo had asked them to take daytime pills, but instead they took morning pills.

Discussion

The first result showed that people with a lower cognitive level showed lower performances with one sentence instructions (IP1). Second, the domains of cognitive functioning such as repetition, judgment, constructions, and attention affected performance in each task.

The major finding of this study indicated that older people with cognitive deficits might easily understand what to do next if a task is divided into steps and instructed by PaPeRo. It is almost consistent with the previous reports that one verbal instruction in a single step enabled persons with mild or moderate Alzheimer’s disease to perform ADL (Lancioni et al., Citation2008, Citation2012). Additionally, Stanyon et al. (Citation2016) reported that health care workers tend to try to speak with reduced speech complexity, in short sentences, and not using abstract ideas. Thus, technical verbal instruction and communication is very important for people with dementia to take action, and increase communicative behavior (Small et al., Citation2003; Stanyon et al., Citation2016; Wells et al., Citation2000). In this study, we found out that communication robots also need to use step-by-step instructions for people with cognitive disfunction, as along with daily conversations in health care settings.

The secondary finding showed that repetition and judgment significantly affected performance in Task A. Repetition is mainly influenced by hearing function and working memory (Kiernan et al., Citation1987). In this study, the participants with no hearing impairment were required to listen to the PaPeRo carefully, consider what to do, and perform the tasks, wherein working memory was needed above all other cognitive functions. This finding was supported by a Liu et al. (Citation2019) study which reported that higher deficit of working memory reduced sentence comprehension for both healthy older people and people with AD. Moreover, working memory and judgment are also components of executive function, which affects ADL in the early stages of dementia (Martyr & Clare, Citation2012; Royall et al., Citation2002). Therefore, the participants with lower working memory (repetition in COGNISTAT) and judgment might have had lower executive function, which would have led to failures in completing Task A (main ADL among older adults). Thus, our study revealed not only that the executive function was important to perform ADL but also assistive robots are able to fill in gaps by using step-by-step instructions like IP2 and IP3. In terms of Task B, there were no significant affected items in COGNISTAT, but the regression formula was valid. Because measuring blood pressure was complicated task with using tools, global cognitive function might be needed.

Another interesting finding was that two healthy older adults with low scores of attention made mistakes in selecting the medicine. Getzmann et al. (Citation2014) reported that deficits in attention lead to difficulties in speech perception in noisy situations. The most important finding in this study was that older people with attention deficit might make mistakes in their daily lives even though they are healthy older adults judged by cognitive assessment. We emphasized information should be repeated to enable people with lower attention to acquire correct information (Inoue et al., Citation2012; Nishiura et al., Citation2014).

In this study, there are some limitations. First, we collected the data from older persons both with or without dementia. People with or without dementia might have different attitudes to the assistive robot and lead different outcomes. Second, a sample size might be too small to apply the present study to a larger population. In a future study, we will try multicenter studies and analyze the results by dividing groups according to with or without dementia.

In conclusion, step-by-step instructions should be used for older people with lower levels of cognitive function, especially working memory, and repeated instructions may be required for lower attention. Types of instruction should be selected depending on cognitive characteristics.

Declaration of interest statement

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Acknowledgments

The authors are grateful to all participants and supporting staffs in Seikatsu Kagaku Un-Ei Co., Ltd. We disclosed receipt of the following financial support for the research, authorship, and/or publication of this article; JST “Strategic Promotion of Innovation Research and Development” Grant Number JPMJSV1011, Japan. The authors confirm that the data supporting the findings of this study are available within the article and its supplementary materials.

Additional information

Funding

This work was partly supported by the JST “Strategic Promotion of Innovation Research and Development” under Grant Number [JPMJSV1011] and JSPS KAKENHI Grant Number JP19KT0004.

References

  • Alzheimer’s Association. (2020). 2020 Alzheimer’s disease facts and figures. Alzheimer’s & Dementia: The Journal of the Alzheimer’s Association, special report, 1–94. https://doi.org/10.1002/alz.12068.
  • Chaurasia, P., McClean, S. I., Nugent, C. D., Cleland, I., Zhang, S., Donnelly, M. P., Scotney, B. W., Sanders, C., Smith, K., Norton, M. C., & Tschanz, J. (2016). Modelling assistive technology adoption for people with dementia. Journal of Biomedical Informatics, 63, 235–248. https://doi.org/10.1016/j.jbi.2016.08.021
  • Czarnuch, S., & Mihailidis, A. (2011). The design of intelligent in-home assistive technologies: Assessing the needs of older adults with dementia and their caregivers. Gerontechnology, 10(3), 169–182. https://doi.org/10.4017/gt.2011.10.3.005.00
  • Folstein, M. F., Folstein, S. E., & McHugh, P. R. (1975). “Mini-mental state”. A practical method for grading the cognitive state of patients for the clinician. Journal of Psychiatric Research, 12(3), 189–198. https://doi.org/10.1016/0022-3956(75)90026-6
  • Fujii, M., Butler, J. P., & Sasaki, H. (2018). Core symptoms and peripheral symptoms of dementia. Geriatrics & Gerontology International, 18(6), 979–980. https://doi.org/10.1111/ggi.13283
  • Gale, S. A., Acar, D., & Daffner, K. R. (2018). Dementia. The American Journal of Medicine, 131(10), 1161–1169. https://doi.org/10.1016/j.amjmed.2018.01.022
  • Getzmann, S., Falkenstein, M., & Wascher, E. (2014). ERP correlates of auditory goal-directed behavior of younger and older adults in a dynamic speech perception task. Behavioural Brain Research, 278, 435–445. https://doi.org/10.1016/j.bbr.2014.10.026
  • Hu, K., Riemersma-van Der Lek, R. F., Patxot, M., Li, P., Shea, S. A., Scheer, F. A., & Van Someren, E. J. (2016). Progression of dementia assessed by temporal correlations of physical activity: Results from a 3.5-year, longitudinal randomized controlled trial. Scientific Reports, 6(1), 27742. https://doi.org/10.1038/srep27742
  • Inoue, T., Nihei, M., Narita, T., Onoda, M., Ishiwata, R., Mamiya, I., Shino, M., Kojima, H., Ohnaka, S., Fujita, Y., & Kamata, M. (2012). Field-based development of an information support robot for persons with dementia. Technology and Disability, 24(4), 263–271. https://doi.org/10.3233/TAD-120357
  • Japanese Ministry of Health, Labour and Welfare. (2016, November). Long-term care insurance system of Japan. Health and Welfare Bureau for the Elderly. https://www.mhlw.go.jp/english/policy/care-welfare/care-welfare-elderly/index.html
  • Kenigsberg, P. A., Aquino, J. P., Bérard, A., Brémond, F., Charras, K., Dening, T., Droës, R. M., Gzil, F., Hicks, B., Innes, A., Nguyen, S. M., Nygård, L., Pino, M., Sacco, G., Salmon, E., Van Der Roest, H., Villet, H., Villez, M., Robert, P., & Manera, V. (2019). Assistive technologies to address capabilities of people with dementia: From research to practice. Dementia (London, England), 18(4), 1568–1595. https://doi.org/10.1177/1471301217714093
  • Kiernan, R. J., Mueller, J., Langston, J. W., & Van Dyke, C. (1987). The neurobehavioral cognitive status examination: A brief but quantitative approach to cognitive assessment. Annals of Internal Medicine, 107(4), 481–485. https://doi.org/10.7326/0003-4819-107-4-481
  • Lancioni, G. E., La Martire, M. L., Singh, N. N., O’Reilly, M. F., Sigafoos, J., Pinto, K., & Minervini, M. G. (2008). Persons with mild or moderate Alzheimer’s disease managing daily activities via verbal instruction technology. American Journal of Alzheimer’s Disease and Other Dementias, 23(6), 552–562. https://doi.org/10.1177/1533317508328181
  • Lancioni, G. E., Singh, N. N., O’Reilly, M. F., Sigafoos, J., & Olivetti Belardinelli, M. (2012). Technology-based intervention to help persons with minimally conscious state and pervasive motor disabilities perform environmentally relevant adaptive behavior. Cognitive Processing, 13(S1), S219–22. https://doi.org/10.1007/s10339-012-0485-5
  • Law, M., Sutherland, C., Ahn, H. S., MacDonald, B. A., Peri, K., Johanson, D. L., Vajsakovic, D. S., Kerse, N., & Broadbent, E. (2019). Developing assistive robots for people with mild cognitive impairment and mild dementia: A qualitative study with older adults and experts in aged care. BMJ Open, 9(9), e031937. https://doi.org/10.1136/bmjopen-2019-031937
  • Little, M. O. (2018). Reversible dementias. Clinics in Geriatric Medicine, 34(4), 537–562. https://doi.org/10.1016/j.cger.2018.07.001
  • Liu, X., Wang, W., Wang, H., & Sun, Y. (2019). Sentence comprehension in patients with dementia of the Alzheimer’s type. PeerJ, 7, e8181. https://doi.org/10.7717/peerj.8181
  • Lorenz, K., Freddolino, P. P., Comas-Herrera, A., Knapp, M., & Damant, J. (2017). Technology-based tools and services for people with dementia and carers: Mapping technology onto the dementia care pathway. Dementia (London, England), 18(2), 725–741. https://doi.org/10.1177/1471301217691617
  • Martyr, A., & Clare, L. (2012). Executive function and activities of daily living in Alzheimer’s disease: A correlational meta-analysis. Dementia and Geriatric Cognitive Disorders, 2-3(2–3), 189–203. https://doi.org/10.1159/000338233
  • Nishiura, Y., Inoue, T., & Nihei, M. (2014). Appropriate talking pattern of an information support robot for people living with dementia: A case study. Journal of Assistive Technologies, 4(4), 177–187. https://doi.org/10.1108/JAT-12-2013-0035
  • Nishiura, Y., Nihei, M., Nakamura-Thomas, H., & Inoue, T. (2019). Effectiveness of using assistive technology for time orientation and memory, in older adults with or without dementia. Disability and Rehabilitation. Assistive Technology, Advance online publication, 1–7. https://doi.org/10.1080/17483107.2019.1650299
  • O’Keefe, J., Maier, J., & Freiman, M. (2010). Assistive technology for people with dementia and their caregivers at home: What might help (RTI Project Report No. 0209351.004.008). Administration on Aging. https://www.dhs.wisconsin.gov/regulations/training/a8-b8-ho2.pdf
  • Orpwood, R., Chadd, J., Howcroft, D., Sixsmith, A., Torrington, J., Gibson, G., & Chalfont, G. (2010). Designing technology to improve quality of life for people with dementia: User-led approaches. Universal Access in the Information Society, 9(3), 249–259. https://doi.org/10.1007/s10209-009-0172-1
  • Orpwood, R., Sixsmith, A., Torrington, J., Chadd, J., Gibson, G., & Chalfont, G. (2007). Designing technology to support quality of life of people with dementia. Technology and Disability, 19(2), 103–112. https://doi.org/10.3233/TAD-2007-192–307
  • Royall, D. R., Lauterbach, E. C., Cummings, J. L., Reeve, A., Rummans, T. A., Kaufer, D. I., LaFrance, W. C., & Coffey, C. E. (2002). Executive control function: A review of its promise and challenges for clinical research. A report from the committee on research of the American neuropsychiatric association. The Journal of Neuropsychiatry and Clinical Neurosciences, 4(4), 377–405. https://doi.org/10.1176/jnp.14.4.377
  • Small, J. A., Gutman, G., Makela, S., & Hillhouse, B. (2003). Effectiveness of communication strategies used by caregivers of persons with Alzheimer’s disease during activities of daily living. Journal of Speech, Language, and Hearing Research: JSLHR, 2(2), 353–367. https://doi.org/10.1044/1092-4388(2003/028)
  • Smith, S. K., & Astell, A. J. (2017). Independent LIVING FUNCTIONS FOR THE ELderly (IN-LIFE): Supporting communication in dementia. Studies in Health Technology and Informatics, 242, 16–22. https://eprints.whiterose.ac.uk/121323/8/AAATE_SKS.pdf
  • Stanyon, M. R., Griffiths, A., Thomas, S. A., & Gordon, A. L. (2016). The facilitators of communication with people with dementia in a care setting: An interview study with healthcare workers. Age and Ageing, 1(1), 164–170. https://doi.org/10.1093/ageing/afv161
  • Wang, R. H., Sudhama, A., Begum, M., Huq, R., & Mihailidis, A. (2017). Robots to assist daily activities: Views of older adults with Alzheimer’s disease and their caregivers. International Psychogeriatrics, 29(1), 67–79. https://doi.org/10.1017/S1041610216001435
  • Wells, D. L., Dawson, P., Sidani, S., Craig, D., & Pringle, D. (2000). Effects of an abilities-focused program of morning care on residents who have dementia and on caregivers. Journal of the American Geriatrics Society, 48(4), 442–449. https://doi.org/10.1111/j.1532-5415.2000.tb04704.x
  • World Health Organization. (2018, February 5). Fact sheet/ageing and health. https://www.who.int/en/news-room/fact-sheets/detail/ageing-and-health