Online awareness of performance on a functional cognitive assessment in individuals with stroke: A case-control study

ABSTRACT

Unawareness is a significant barrier to cognitive rehabilitation following acquired brain injury. Little is known about online awareness of cognitively-based instrumental activities of daily living (C-IADL) after stroke, particularly C-IADLs that emphasize executive functions. Our goal was to evaluate in stroke patients (1) online awareness during and immediately after a C-IADL task that emphasizes executive functions and (2) the association between awareness and performance on the C-IADL task. Seventy-seven stroke patients on an acute inpatient rehabilitation unit and 77 control participants completed the 10-item Weekly Calendar Planning Activity (WCPA-10), a standardized C-IADL task that requires working memory, planning, shifting, and inhibition. Trained examiners observed the use of a self-checking strategy and self-recognition of errors during the task. Immediately after the task, participants estimated their accuracy, and rated their own performance, which was compared with objective accuracy. Relative to the control group, stroke patients overestimated their accuracy, less often recognized errors, and less frequently used a self-checking strategy. Overestimation was associated with worse overall performance on the WCPA-10. Findings suggest that poor online awareness of C-IADL performance is common in stroke patients undergoing acute inpatient rehabilitation. Increasing awareness through metacognitive interventions should be a core focus of early post-stroke cognitive rehabilitation.

KEYWORDS:

• Self-Awareness
• insight
• cerebrovascular disease
• executive function
• rehabilitation

Introduction

Individuals with acquired brain injuries commonly experience diminished awareness of deficits, which has significant negative effects on rehabilitation, functional outcomes (Ownsworth & Clare, 2006), and psychosocial functioning (Ownsworth & Fleming, 2005). Individuals with acquired brain injuries may exhibit unawareness for motor, sensory, or cognitive impairments. Awareness of cognitive deficits moderates the association between objective cognitive impairment and everyday function (Griffen et al., 2011), such that patients who have impairments but are aware of them can compensate for deficits and better preempt or self-regulate cognitive errors. In contrast, poor awareness is typically marked by a lack of general awareness of one’s limitations, and/or overestimation of performance on cognitively-demanding tasks, resulting in limited compensation and self-regulation of errors.

As described in the Dynamic Comprehensive Model of Awareness, self-awareness has been divided into general awareness or metacognitive knowledge and online awareness (Toglia, 2018; Toglia & Kirk, 2000). General awareness is based on one’s accumulated experiences and stored memories and includes beliefs and self-perceptions of one’s abilities and limitations, that are independent from a particular task or situation. For example, a patient who reports memory or attention complaints in an interview, can be said to have good general awareness. In contrast, online awareness is task-dependent and is activated within the context of an activity. It includes appraisal of task difficulty before an activity (i.e., anticipatory awareness), error recognition and correction during an activity, and self-appraisal of performance immediately after an activity (Toglia & Foster, 2021). The latter can be operationalized as the discrepancy between perceived performance and objective performance on a task or activity (Rotenberg-Shpigelman et al., 2014). Awareness during and immediately after a task have also been described as emergent awareness (Crosson et al., 1989), which was the focus of our current study. There is some evidence that the skills needed for moment-to-moment error monitoring during a task may be different than the ability to reflect on and self-appraise one’s performance immediately after a task (Dean et al., 2017; Robertson & Schmitter-Edgecombe, 2015; Toglia, 2018).

Online awareness of one’s performance is particularly important for everyday functional activities (Robertson & Schmitter-Edgecombe, 2015; Vossel et al., 2013). Recognition of errors in-the-moment and accurate self-appraisal after a task is crucial for adaptation and modification of actions for safety, independence, and successful task outcomes. Although unawareness is known to occur after stroke (Al Banna et al., 2015; Leung & Liu, 2011) – particularly for visual attention in hemispatial neglect (Chen & Toglia, 2019), language skills after aphasia (Dean et al., 2017), and memory impairment (Boosman et al., 2014) – there are few studies that have investigated online awareness of performance specifically as it relates to cognitively-based instrumental activities of daily living (C-IADL). Rotenberg-Shpigelman and colleagues (2014) found deficits in online awareness in a sample of patients that was predominantly stroke, which manifested as an overestimation of performance on instrumental activities of daily living (IADL) task. However, these IADL tasks did not have high cognitive demands and included making two hot beverages and using the telephone to obtain information regarding store hours. Similarly, Barrett and colleagues found that stroke patients overestimated performance on a medication management task, though again the task was not specifically chosen to measure C-IADL (Barrett et al., 2014).

The need to understand online awareness within the context of cognitively-demanding functional activities, particularly those that involve working memory and executive functions, is underscored by the frequency of working memory and executive dysfunction post-stroke (Turunen et al., 2018) and the importance of these cognitive domains in post-stroke outcomes (Jaywant et al., 2018). Executive functions encompass skills such as planning, organization, shifting, and inhibition that subserve efficient goal-directed behavior (Friedman & Miyake, 2017). Working memory, which is at times subsumed under the umbrella of executive functions, refers to the ability to maintain, manipulate, and update information “online” in the service of goal-directed behavior (Chai et al., 2018). Knowledge of how a stroke affects online awareness in C-IADLs involving working memory and executive functions may help to develop tailored and targeted metacognitive interventions.

The goal of this study was to evaluate – relative to an age-matched healthy control group – stroke patients’ online awareness on an objective, psychometrically-sound, performance-based C-IADL task, the 10-item Weekly Calendar Planning Activity (WCPA-10) that places high demands on executive functions. Online awareness of one’s performance was operationalized as in a previous study by evaluating self-recognition of errors and use of a self-checking strategy during the task, and discrepancies between perceived accuracy and objective accuracy immediately after the task (Arora et al., 2021). Thus, we used two separate measures of online awareness during the task and two separate measures of awareness after the task. We hypothesized that individuals with stroke would show reduced online awareness relative to healthy control participants, in awareness during and after the task (i.e., emergent awareness), which would be associated with worse overall performance on the WCPA-10. We also explored if lesion location (left vs. right hemisphere) affected awareness in the stroke group.

Methods

Design

This study consisted of follow-up analyses to a previous study that examined differences in individuals with stroke and healthy age-matched adults in performance on the WCPA-10 (Jaywant et al., 2021). In the current study, we extended those findings to focus on awareness.

Participants

Seventy-seven individuals with stroke undergoing acute inpatient rehabilitation and 77 healthy age matched controls from an existing normative database (n = 324) were included in this study. Participants with stroke were all admitted to acute inpatient rehabilitation on a 22-bed general rehabilitation unit at a large academic medical center. Inclusion criteria included: alert and oriented, could attend for a minimum of 20 min, were English-speaking, were able to follow two-step commands, and were cognitively independent in routine or basic self-care activities of daily living (ADL). Individuals with stroke were excluded from this study if the severity of vision, language, or cognitive deficits precluded administration of study assessments. If a prospective participant met inclusion/exclusion criteria, and the treating occupational therapist identified the need for a functional cognitive assessment, then the patient was asked by the occupational therapy unit manager or research assistant if results of the WCPA could be included in this study. All stroke participants provided written informed consent and study procedures were approved by the Weill Cornell Medicine Institutional Review Board.

Healthy control participants, age 18 years or older, were recruited via snowballing techniques by graduate occupational therapy students from the greater New York City area. Participants were living independently in the community, did not have subjective cognitive complaints (assessed by Patient-Reported Outcomes Measurement Information System (PROMIS) Cognitive Abilities Short-Form Version 2.0) and, for participants over age 65, had a score >24 on the Montreal Cognitive Assessment (Arora et al., 2021). Note that among healthy control participants, only those at or above age 65 years were administered the Montreal Cognitive Assessment (n = 46) following American Academy of Neurology guidelines (Foster et al., 2019). Participants were excluded if they reported a history of neurologic disease, hospitalization for a psychiatric illness, had a Montreal Cognitive Assessment score < 24, or were unable to read and write in English. Recruitment and assessment of healthy control participants was granted an exempt approval by the Mercy College Institutional Review Board. Verbal consent was obtained after a consent script was read aloud, and a written copy of the script was provided to each participant.

To select an age-matched set of healthy control participants from the larger pool (n = 324; Arora et al., 2021), a research assistant sorted the database according to age, and secondarily by gender. Stroke participants were matched to a healthy control participant by age within one year, and gender if possible. Match with gender was not always possible because the normative database had more females than males. If two or more healthy participants matched a stroke participant, then selection was random.

Measures

10-item Weekly Calendar Planning Activity (WCPA)

Participants were administered the 10-item version of the WCPA, an objective, ecologically valid and performance-based measure of executive functions. It is the short-form (inpatient adaptation) of the original 17 item version, the latter of which has demonstrated validity, reliability (Holmqvist et al., 2020; Lahav et al., 2018), and sensitivity to executive dysfunction and to change following rehabilitation (Goverover et al., 2020; Lahav & Katz, 2020; Toglia, 2015; Toglia et al., 2017). We have previously shown that individuals with stroke have worse accuracy, efficiency, and strategy use, relative to control participants, on the short-form WCPA-10 (Jaywant et al., 2021).

The WCPA-10 takes approximately 10-15 min to complete. It requires the examinee to input 10 appointments into a mock weekly calendar/schedule while following a set of specific rules and guidelines. Appointments are either fixed at a certain date/time, or flexible with multiple date/time options. The examinee has to keep track of multiple rules (e.g., cannot enter appointments on a pre-determined day, cannot cross off items once entered) in working memory while shifting between the appointment sheet, calendar, and instructions sheet. On occasion, appointment times conflict, which requires the examinee to flexibly manage the conflict. Task rules are explained verbally just prior to beginning the task. An 8 × 11 inch paper with task instructions is also placed on the table and can be referred to by the examinee throughout the task. The examiner periodically attempts to distract the examinee with pre-specified questions, which the examinee must inhibit. During the task, the examiner closely observes the participant for strategies used and self-recognition of errors. Further description of WCPA is available at: https://multicontext.net/weekly-calendar-planning-activity

For stroke participants, the WCPA-10 was administered as part of standard of care on the inpatient rehabilitation unit by occupational therapists. For healthy control participants, the WCPA-10 was administered by occupational therapy graduate students to participants in the community. Following completion of the WCPA-10, participants completed a post-task interview with the examiner. Participants were asked to estimate how many appointments out of 10 they entered accurately. Participants then completed a self-rating using four questions that ascertained their perceptions of task difficulty and the efficiency and accuracy of their methods used (Table 1a). Participants responded to each question on a 1–4 rating scale in which 1 = agree, 2 = somewhat agree, 3 = somewhat disagree, 4 = disagree. All participants were also asked if they were familiar with calendars/schedules such as the one used by the WCPA-10. As we have previously reported (Jaywant et al., 2021), 53% of the stroke group and 64% of the control group reported familiarity with the format, though familiarity was not associated with accuracy or other performance variables on the WCPA-10.

Table 1. Post-Task Self Rating Questions (a) and Awareness Categorization Scheme (b).

(a)
Post-Task Self-Rating Questions	Agree (1)	Somewhat Agree (2)	Somewhat Disagree (3)	Disagree (4)
This task was easy for me
I used efficient methods to complete this task
I completed this task accurately
I kept track of everything I needed to do
Average self-rating:
(b)
Category assigned based on Average Self Rating	Average Self-Rating	Category Assigned Based on Accuracy Score	Accuracy Score
1	1-1.5	1	9–10
2	1.75-2.25	2	7–8
3	2.5-3.0	3	5–6
4	3.25-4.0	4	<5

Quantification of awareness

During Task Awareness. We estimated online awareness following prior examples in the literature (Arora et al., 2021; Boosman et al., 2014; Dean et al., 2017; Rotenberg-Shpigelman et al., 2014). To evaluate online awareness during the task, the examiner was trained to observe whether the participant used a self-checking strategy. Examples of a self-checking strategy include a participant verbalizing that they are double-checking their work, or the examiner observing the participant to check the appointments they have entered against the list of appointments. The use of a self-checking strategy was dichotomized (observed vs. not). Further, the examiner observed whether the participant self-recognized errors. Self-recognition of errors was operationalized as verbal or non-verbal (e.g., deep sigh or shaking head) acknowledgement of an error, or an attempt to correct a mistake (e.g., drawing a line or arrow from the error to the correct location of the appointment). The percentage of each participant’s total errors that were recognized was used as the outcome, where 0% indicates that the participant did not recognize any errors and 100% indicates that the participant recognized all errors made. For statistical analyses, we included only those participants who made two or more errors (n = 73 stroke participants, n = 55 control participants) in order to ensure that participants had errors to self-recognize and to mitigate potential ceiling/floor effects for participants who made only one error.

After Task Awareness. We estimated online awareness after the task using patients’ self-appraisal of performance as compared to objective performance. We calculated the discrepancy (awareness discrepancy) between the actual number of appointments correctly entered out of 10 and each participant’s estimation of the number of appointments entered correctly. We classified poor vs. good awareness based on this discrepancy in two ways. We initially created a binary classification in which we defined ‘poor’ awareness as overestimation by > 2 appointments (i.e., estimated – observed > 2) and ‘good’ awareness as all other cases (i.e., estimated – observed ≤ 2). We then created a more fine-grained classification with three levels defined as poor awareness (estimated – observed > 2), mild unawareness (estimated – observed = 1-2), and good awareness (estimated – observed < 1). Note that individuals who underestimated their performance (i.e., had a negative discrepancy score) are also included in the ‘good’ awareness category for simplicity of interpretation and because underestimation implies that the participant recognized errors made.

Next, we created a categorization scheme (awareness level) based on participants’ mean responses to the post-task interview/self-rating questions (Table 1) and based on the number of appointments accurately entered. For each participant, we computed the average self-rating (Table 1a) and then assigned a category label (1 through 4) based on that average self-rating (Table 1b). The number of appointments correctly entered (accuracy) was also assigned a separate 1 through 4 category label. We then computed the discrepancy between the category label the participant was assigned based on their average self-rating, and the category label the participant was assigned based on their accuracy. For example, if a participant’s average self-rating category was 1 (indicating they perceived no difficulty), but their accuracy category was 4 (assigned category based on accuracy score of 4), then the category discrepancy would be 3 indicating significant overestimation. This enabled us to quantify after-task online awareness in a manner that mitigated potential ceiling effects that could be introduced in patients who made few errors.

Of note, we did not calculate anticipatory awareness, as the WCPA-10 does not include questions to assess this aspect of online awareness. Anecdotally, we have found that most individuals (patients and controls) do not have strong anticipatory awareness of the WCPA-10 because the task includes subtle challenges and demands on executive control (e.g., distractions, appointments that conflict) that are not explicitly stated in the instructions and that examinees would not be aware of beforehand.

Procedure

In the stroke group, all procedures were conducted while individuals were admitted to the acute inpatient rehabilitation unit. The Montreal Cognitive Assessment was performed by the occupational therapist as a standard of care cognitive screen on admission and the score was extracted from the patient’s chart. Demographic and clinical characteristics were recorded by the occupational therapist on a cover sheet. The WCPA-10 was administered by the treating occupational therapist. Occupational therapists all used the WCPA-10 routinely. They were provided with a series of in-services that involved videos and role playing. An OT manager/ supervisor who was proficient in the WCPA-10 supervised staff who administered the WCPA to stroke participants.

For healthy control participants, the Montreal Cognitive Assessment and WCPA-10 were administered by occupational therapy graduate students in a quiet evaluation room at Mercy College and/or in a quiet, distraction free environment within a community setting or person’s home depending on the individual’s travel preference. Graduate students participated in a rigorous training program including readings, critique of videos, practice sessions, submission of scored practice tests conducted independently on practice subjects, and individual competency testing through role playing scenarios.

Statistical analyses

We used chi-square analyses and non-parametric inferential statistics (Mann–Whitney U) with alpha = .05 to investigate group differences between individuals with stroke and healthy control participants based on the above awareness variables. We repeated these analyses within the stroke group to evaluate the effect of lesion location (left vs. right hemisphere [n = 62]; we excluded individuals with unknown lesion locations or bilateral lesions for this analysis). Effect sizes were computed using phi (ϕ) and r respectively. We used Spearman rank-order correlations to evaluate the association between awareness and performance on the WCPA-10, separately in the stroke group and control group. These inferential statistics were accompanied by graphical inspection of data Table 2.

Table 2. Frequency (N[%]) of responses by group (Stroke, Healthy Control) on the Post-Task Interview and Self-Rating

	Agree (1)	Somewhat Agree (2)	Somewhat Disagree (3)	Disagree (4)
Stroke
This task was easy for me	25 (33%)	31 (40%)	10 (13%)	9 (12%)
I used efficient methods to complete this task	39 (51%)	23 (30%)	4 (5%)	9 (12%)
I completed this task accurately	31 (40%)	23 (30%)	9 (12%)	11 (14%)
I kept track of everything I needed to do	40 (52%)	18 (23%)	8 (10%)	8 (10%)
Healthy Control
This task was easy for me	32 (42%)	42 (54%)	3 (4%)	0 (0%)
I used efficient methods to complete this task	37 (48%)	31 (40%)	7 (9%)	2 (3%)
I completed this task accurately	32 (42%)	30 (39%)	8 (10%)	7 (9%)
I kept track of everything I needed to do	47 (61%)	20 (26%)	7 (9%)	3 (4%)

Results

Demographic and clinical characteristics

Individuals with stroke and healthy control participants did not differ in age (Stroke Mean = 66.1, SD = 14.1; Healthy Control Mean = 66.0, SD = 14.0), gender (Stroke, Males = 39 [51%], Females = 38 [49%]; Healthy Control, Males = 36 [47%], Females = 41 [53%]), or education (in years, Stroke Mean = 14.7, SD  = 1.9; Healthy Control Mean = 14.6, SD = 2.6). The stroke group had a larger percentage of Black participants (21% vs 14%) and a smaller percentage of Hispanic participants (4% vs. 20%) relative to the control group. Stroke participants had an average score on the Montreal Cognitive Assessment in the mildly impaired range (Mean = 23.3, SD = 3.6), which was significantly lower than the healthy control participants (Mean = 26.2, SD = 1.7). Stroke patients were evaluated on average 18.1 (SD = 14.6) days post-stroke. Thirty-eight (49%) patients in the stroke group had right hemisphere strokes, 28 (36%) had left hemisphere strokes, 8 (10%) had bilateral strokes, and 3 (4%) had stroke locations that were unknown or unavailable.

Awareness: self-Checking strategy

A 2 × 2 chi-square analysis (Group [Stroke, Control], Self-Checking Strategy [observed vs. not]) demonstrated that a significantly larger percentage (62%) of stroke participants did not use a self-checking strategy compared to control participants (39%), χ²= 7.71, p = .005, ϕ = .23 (small effect).

Awareness: self-recognition of errors

Stroke participants had a trend towards lower self-recognition of errors relative to healthy control participants (Stroke: median proportion errors recognized = 13% (IQR 19%); healthy control: median percent errors recognized = 22% (IQR 26%); Mann–Whitney U = 2333, p = .09, effect size r = .11 (small effect)).

Awareness Discrepancy: Observed Accuracy vs. Estimated Accuracy.

Figure 1 displays a frequency histogram of discrepancy scores (estimated accuracy – observed accuracy) by group, where positive values indicate overestimation and negative values indicate underestimation. A clear right-ward shift towards overestimation is observed in the stroke group.

Figure 1. Frequency distribution of discrepancy between estimated accuracy (number correct out of 10) and observed accuracy on the 10-item Weekly Calendar Planning Activity. Positive values of Awareness Discrepancy indicate overestimation and negative values indicate underestimation.

Frequency histogram with bars depicting for the stroke group and the control group, the number of participants who overestimated or underestimated their performance. There is a clear shift in the stroke group demonstrating a tendency to overestimate.

A 2 × 2 chi-square analysis using the binary definition of awareness discrepancy (Group [Stroke, Control], Awareness [Poor, Good]), indicated that a significantly larger percentage of stroke participants had poor awareness (67%) compared to control participants (38%), χ²= 12.5, p = .001, ϕ = .29 (small-medium effect) (Figure 2). Similarly, there was no difference in those with left vs. right hemisphere strokes and proportion errors recognized, Mann–Whitney U = 488, p = .86, effect size r = .02 (small effect).

Figure 2. Percent of sample classified as having poor awareness (estimated accuracy – observed accuracy > 2) vs. good awareness, by group.

Stacked bar graph with two bars, one for the stroke group and one for the control group, depicting the percentage of each group that was classified as having good awareness vs. poor awareness. There is a larger percentage of the stroke group classified as having poor awareness.

A similar finding was observed when awareness discrepancy was separated into three groups (Poor Awareness, Mild Unawareness, Good Awareness). A 2 × 3 chi-square analysis indicated a statistically significant Group x Awareness Discrepancy effect (χ²= 13.2, p = .001, ϕ = .29 (medium effect)). Among the stroke group, the majority of participants were classified as having poor awareness (67%) while a smaller percentage were classified as having mild unawareness (18%) and good awareness (15%). Among the healthy control group, the distribution of classifications indicated 38% of the sample had poor awareness, 40% had mild unawareness, and 22% had good awareness (Figure 3).

Figure 3. Percent of sample classified as having poor awareness (estimated accuracy – observed accuracy > 2), mild unawareness (estimated accuracy – observed accuracy = 1-2) vs. good awareness, by group.

Stacked bar graph with two bars, one for the stroke group and one for the control group, depicting the percentage of each group that was classified as having normal awareness vs. mild unawareness vs. poor awareness. In the stroke group, there is a larger classified as having poor awareness, and a smaller percentage classified as having mild unawareness. The percentage of normal awareness is approximately the same in the stroke group and control group.

Awareness level: discrepancy between participant self-Rating and task accuracy

We found consistent results when evaluating awareness using the fourth method of categorizing participant self-ratings and accuracy, and calculating the discrepancy between these categories for each participant (“awareness level”). A chi-square analysis evaluating Group (Stroke, Control) and Awareness Level (Aware or Underestimates, Overestimates by 1 Category, Overestimates by 2 Categories, Overestimates by 3 Categories) approached significance (χ²= 6.95, p = .07, ϕ = .22 (small effect)). As shown in Figure 4, the stroke group had a slightly greater percentage of participants who overestimated by 2 or 3 categories (25% and 15%, respectively), relative to the control group (17% and 5%, respectively). The percentage of individuals who overestimated by 1 category was similar in the stroke group (27%) and the control group (30%). The control group had a larger percentage of individuals who were categorized as “Aware or Underestimates” (48%) relative to the stroke group (33%).

Figure 4. Unawareness by group as calculated by Awareness Level, the discrepancy between participants’ self-rating category (1-4) and accuracy category (also 1-4).

Stacked bar graph with two bars, one for the stroke group and one for the control group, depicting the percentage of each group that was classified as having overestimated by 3 categories, overestimated by 2 categories, overestimated by 1 category, or classified as “aware or underestimates”. There is a larger percentage of the stroke group classified as having overestimated by 3 or 2 categories.

Association between awareness and performance

Among both the stroke group and the control group, awareness was significantly associated with performance on the WCPA-10 such that greater unawareness was associated with worse performance. In the stroke group, awareness discrepancy and overestimation (estimated accuracy – actual accuracy) correlated negatively with total number of accurate responses on the WCPA-10 (rs = -.58, p < .01). Overestimation based on awareness level (difference in categorization based on self-rating vs. actual performance) also correlated negatively with total number of accurate responses on the WCPA-10 (rs = -.38, p < .01). Similar patterns were observed in the control group, though the strength of the correlation coefficients was higher. Awareness discrepancy (estimated accuracy – actual accuracy) correlated negatively with total number of accurate responses on the WCPA-10 (rs = -.80, p < .01). Awareness level (difference in categorization based on self-rating vs. actual performance) also correlated negatively with total number of accurate responses on the WCPA-10 (rs = -.70, p < .01).

Association between awareness and lesion location

Within the stroke group, there was a significant association between awareness level (discrepancy between participant self-rating and task accuracy) and lesion location (left hemisphere vs. right hemisphere), χ²= 9.49, p = .02, ϕ = .39 (medium effect). A greater percentage of individuals with right hemisphere strokes (24%) overestimated their performance by three categories than did individuals with left hemisphere strokes (0%). Relatedly, a smaller percentage of individuals with right hemisphere strokes (19%) overestimated their performance by one category than did individuals with left hemisphere strokes (44%). There was no association between lesion location and use of a self-checking strategy (p = .13), proportion errors self-recognized (p = .86), and awareness discrepancy separated into two groups (p = .90) or into three groups (p = .97).

Discussion

The goal of this study was to evaluate online awareness during and immediately following a C-IADL task that emphasizes executive functions in individuals with stroke, relative to a healthy age-matched control group. We found consistent evidence of impaired online awareness of performance in stroke participants relative to control participants across multiple measures. Relative to the healthy control group, the stroke group less frequently self-recognized errors and used a self-checking strategy during the task, and more frequently overestimated their accuracy (i.e., had a higher perceived accuracy than observed accuracy), and had a greater discrepancy between their self-rating of performance and their actual performance, after the task. Overestimation of accuracy after the task was associated with worse performance on the WCPA-10 in the stroke group and control group, though the association was stronger in the control group. Finally, on one of our awareness variables (awareness level) individuals with right hemisphere strokes more frequently overestimated their performance by a large magnitude (3 categories) than did individuals with left hemisphere strokes.

Our findings are consistent with studies that have found individuals with stroke to exhibit unawareness of hemispatial neglect (Chen & Toglia, 2019), aphasia (Dean et al., 2017) memory deficits (Boosman et al., 2014), and activities of daily living (Barrett et al., 2014; Rotenberg-Shpigelman et al., 2014). Importantly, we extend these prior studies to demonstrate poor online awareness on a C-IADL task that places high demands on executive skills. Executive functions are strongly associated with everyday function and disability (Cahn-Weiner et al., 2002; Jaywant et al., 2018) and thus our findings have direct relevance for daily activities after stroke. On a task that requires accurately managing a schedule and calendar such as the WCPA-10, individuals with stroke and executive dysfunction may omit appointments, inadequately manage conflicting appointments (e.g., double book or schedule one at the wrong time), or make errors in specific details of an appointment such as its date, length, or details; for those who have poor online awareness, such errors are less likely to be adaptively managed or self-corrected. This is because individuals who lack awareness may not recognize the need to adopt compensatory strategies or to self-regulate their performance. Consistent with this notion, self-awareness is known to impact independence in IADLs (Yeo et al., 2021) and to moderate the association between cognitive impairment and daily function (Villalobos et al., 2020). Similarly, in the present study, we found that overestimation of performance correlated significantly with worse overall performance on the WCPA-10. Our findings suggest that improving self-awareness, for example through metacognitive strategy training (Copley et al., 2020; Jaywant et al., 2020), should be emphasized in post-stroke cognitive rehabilitation. Metacognitive strategy training is designed to increase awareness of cognitive errors and to develop strategies to mitigate errors. It has level 1 evidence of efficacy in acquired brain injury as demonstrated through multiple randomized controlled trials and has been recommended as a practice standard for treating executive dysfunction by the Cognitive Rehabilitation Task Force of the American Congress of Rehabilitation Medicine (Cicerone et al., 2019; Ownsworth et al., 2017; Schmidt et al., 2013).

In this study, we chose to focus on online awareness of performance. This is an aspect of awareness that emphasizes an individual’s self-perception on a task or activity, as ascertained during a task or immediately afterwards (Toglia & Kirk, 2000). It is distinct from general awareness, which refers to an individual’s general sense of having a deficit, and anticipatory awareness, which refers to an individuals’ perception of having difficulties on a task or activity in the future. We found that there was a statistically significant difference in awareness between the stroke group and age-matched healthy control group on both a during-task measure (use of a self-checking strategy) and an after-task measure (discrepancy between estimated accuracy and observed accuracy). We also observed a similar though non-significant trend on an additional during-task measure (self-recognition of errors) and an after-task measure (discrepancy between self-rating and objective performance). Effect sizes were small-medium across definitions of awareness. These results underscore the relevance of online awareness in stroke and highlight the importance of assessing online awareness within the context of functional cognitive activities in clinical practice. Many patient-reported outcome measures and interview tools focus on general awareness (e.g., Sherer et al., 1998) but our findings argue for the necessity of ascertaining online awareness in conjunction with general awareness during clinical care. While we did not directly compare general vs. online awareness, nor did we assess anticipatory awareness, prior research has demonstrated that even among stroke patients with intact intellectual awareness, online awareness can be impaired and contribute to poor functional outcomes (Chen & Toglia, 2019).

Interestingly, our age-matched healthy control group also tended to overestimate their accuracy, though less frequently than the stroke group. This finding accords with another study using the WCPA-10 that demonstrated greater unawareness in older adults relative to younger adults (Arora et al., 2021), as well as previous research indicating greater overestimation of performance on a cognitive task in older versus younger adults (Harty et al., 2013). Prior studies have also related overestimation of abilities in older adults to unsafe driving and poor financial decision making (Huang et al., 2020; Sunderaraman et al., 2020). This body of research suggests that aging itself is associated with decrements in awareness, which may occur as a result of age-related changes in brain networks thought to underlie self-awareness, primarily the default mode network (Therriault et al., 2018; Vannini et al., 2017).

We explored the association between lesion location and awareness in the stroke group. On one awareness variable – the discrepancy between participant self-rating and task accuracy (awareness level) – we found a significant and medium sized effect. Individuals with right hemisphere strokes more commonly severely overestimated performance whereas those with left hemisphere strokes more frequently mildly underestimated performance. This finding is consistent with the known role of right hemisphere lesions in anosognosia for motor deficits and hemispatial neglect (Barrett, 2021; Monai et al., 2020). Our finding suggests a role of right hemisphere dysfunction in unawareness of executive function on a functional cognitive measure. Our analysis was exploratory, however, and should be interpreted with caution, particularly as we did not see lesion location differences in our other measures of awareness.

Among limitations of this study, we were not able to administer a neuropsychological battery to participants, including impairment-based neuropsychological measures of executive function. However, this reflects the reality of working with an acutely ill population in an inpatient setting. Prior research has demonstrated that the WCPA-10 is sensitive to executive dysfunction in patient populations such as multiple sclerosis, attention-deficit/hyperactivity disorder, and acquired brain injury (Doherty, 2018; Goverover et al., 2020; Lahav et al., 2018). We did not have access to neuroimaging, which could have enabled further exploration of the structural and lesion-based correlates of poor awareness (e.g., lesion size and location). Incorporating structural and functional neuroimaging to elucidate the mechanisms of unawareness after stroke is an important area of future research, as this may help match patients to treatments and to develop targeted interventions. Another possible limitation is the potential for ceiling effects on the short-form WCPA-10. However, we have shown a relatively broad distribution of scores on the WCPA-10 in this sample, as well as stroke-control differences across performance variables on the WCPA-10 including accuracy and strategy use (Jaywant et al., 2021). Further, our measure of awareness level mitigated the possibility of ceiling effects that could have occurred in the awareness discrepancy measure. Because we only assessed stroke patients undergoing acute inpatient rehabilitation, our results may not generalize to community-dwelling patients with chronic stroke or those undergoing outpatient rehabilitation.

In summary, we demonstrated that relative to an age-matched healthy control group, individuals with stroke exhibit worse online awareness of performance on a functional C-IADL task that is associated with executive functions. Individuals with stroke overestimate their performance and less commonly use a self-checking strategy. Poor online awareness of performance is associated with worse task performance. Our findings suggest that online awareness of functional cognitive performance is commonly impaired after stroke and have the potential to impact daily function and rehabilitation.

Disclosure statement

JT is author of a published assessment used in this study (WCPA) and receives royalties from the publisher, American Occupational Therapy Association.

Additional information

Funding

The authors received no specific funding for this work. AJ receives salary support from a K12 career development award from Georgetown University (subaward)/National Institute of Child Health and Human Development (Grant/Award Number: 1K12-HD093427-04).