Quality of self-reported cognition: effects of age and gender on spatial navigation self-reports

Abstract

Subjective measures of cognitive abilities are often used in various environments, such as clinical, experimental, and professional settings. Here, we assess the quality of such measures, specifically looking into the impact of age and gender. Spatial navigation ability will be used as an exemplary case, given its large individual variation and relevance to the healthy aging process. With a navigation experiment and a self-report questionnaire, the objective and subjective navigation performance of 7150 participants (age 18–89 years) was measured. Results showed the participants provided informative estimates of their cognitive performance. However, strong systematic biases were present related to age and gender. Overestimation increased with increasing age. Overestimation was also found for males, whereas underestimation was found for females. Consideration of such biases is recommended when implementing self-report measures of cognition and considering the potential impact these biases may have on cognitive functioning itself.

Keywords:

• Self-report
• cognition
• spatial navigation
• aging
• gender

1. Introduction

Human cognitive abilities are typically assessed in an objective setting with standardized tests of specific cognitive domains. This form of assessment is traditionally applied in clinical, educational and professional settings (see e.g. Cichetti, 1994). Subjective reports on cognitive abilities can also be informative, especially in the clinical domain. For instance, when screening for cognitive abnormalities indicative of certain neurological phenomena, like mild cognitive impairment and dementia, subjective cognitive complaints could be an important source of information (e.g. Dufouil, Fuhrer, & Alpérovitch, 2005; Hohman, Beason-Held, Lamar, & Resnick, 2011). However, some have questioned the usefulness of measures of such cognitive complaints (e.g. Edmonds, Delano-Wood, Galasko, Salmon, & Bondi, 2014; Slavin et al., 2010). Literature thus shows the quality of such subjective reports in relation to cognitive measures are unclear and that there are possibly other psychological factors involved (Slavin et al., 2010). A likely factor to affect these subjective reports is age. Subjective cognitive reports have been reported to show age-related decline, in absence of a relation with objective functioning (Ponds, Van Boxtel, & Jolles, 2000). This raises the question how well individuals can assess their own cognitive abilities and which factors may affect this. Therefore, the aim of the current study is to assess subjective ratings of cognitive ability in relation to objective performance.

We focus on spatial navigation ability, as an exemplary cognitive domain, given its particular relevance for the aging population. It is one of the first cognitive functions to decline with age and is suggested to be of critical importance in the early detection of pathological aging (Klencklen, Després, & Dufour, 2012; Lester, Moffat, Wiener, Barnes, & Wolbers, 2017; Lithfous, Dufour, & Després, 2013). So for clinical use, spatial navigation offers a highly important domain for which we aim to uncover the quality of self-reported performance and aging effects on this quality.

Whether we walk to another room in our house or travel longer distances outside, we rely on our navigation ability. The research domain of spatial navigation is increasing in popularity over the past decade, especially in relation to healthy and pathological aging (e.g. Coughlan, Laczó, Hort, Minihane, & Hornberger, 2018). Many of the research questions concerning navigation ability relate to the relatively large individual differences that are found in this ability. Age and gender are thought to substantially explain for such differences. A decline with older age is frequently reported and there is a strong link to age-related hippocampal volume changes (Coutrot et al., 2018; Head & Isom, 2010; Moffat, 2009; Van der Ham, Claessen, Evers, & van der Kuil, 2020). Gender differences are typically less straightforward and can be explained by differences in strategy use during navigation (Cutmore, Hine, Maberly, Langford, & Hawgood, 2000; Grön, Wunderlich, Spitzer, Tomczak, & Riepe, 2000).

In literature there is a wide array of measurement tools available to assess navigation ability. Popular choices include human analogues of experimental designs used in rodents (e.g. Astur, Tropp, Sava, Constable, & Markus, 2004) and tasks assessing path integration (Wolbers, Wiener, Mallot, & Büchel, 2007). Recently, such tasks have been implemented in a game like application to assess navigation performance in a very large sample across different ages and cultural backgrounds (see e.g. Coutrot et al., 2018)

Most of these tools concern objective behavioral assessment of performance. Yet, subjective measures are also frequently used. Such subjective measures of navigation ability can serve experimental as well as clinical purposes. Experimentally, questionnaires like the Santa Barbara Sense of Direction Scale are often administered, as it has shown to provide a reliable measure of navigation ability in general (Hegarty, Richardson, Montello, Lovelace, & Subbiah, 2002). This scale is frequently used to identify good and bad navigators in a sample of participants (e.g. Janzen, Jansen, & van Turennout, 2008). Clinically, the Wayfinding Questionnaire (WQ) has shown to be a reliable tool to detect navigation impairment in patients with acquired brain injury and to substantially correlate with performance on objective navigation tests in this population (Claessen, Visser-Meily, de Rooij, Postma, & van der Ham, 2016; de Rooij, Claessen, van der Ham, Post, & Visser-Meily, 2017; van der Ham, Kant, Postma, & Visser-Meily, 2013).

Given the large variation between individuals in objective measures of navigation ability, it is informative to assess whether similar variation is also present in subjective, self-report measures of navigation ability. Consequently, the causes of such variation should be considered in the interpretation of self-reported navigation ability. Although scarcely studied, there are a few indications that both age and gender might affect such self-reports. Taillade, N’Kaoua, and Sauzéon (2016) report that young and old adults were comparable in self-rated navigation ability and that therefore the elderly may overestimate their abilities in comparison to younger adults. In contrast, Ariel and Moffat (2018) find a decline in terms of confidence in spatial abilities with increasing age. Furthermore, they found that monitoring of spatial abilities was similar for young and old adults with the exception of allocentric navigation skills. With regard to gender, results converge towards higher self-ratings for males compared to females (see Condon et al., 2015).

The current study was designed to assess the quality of self-reported navigation ability in relation to objective, behavioral navigation performance. We used a very large-scale dataset in which 7150 Dutch and Flemish participants (age range 18–89) performed a navigation task battery as the objective measure and filled out the Wayfinding Questionnaire with the ‘Navigation and orientation’ subscale as a self-reported, subjective measure of navigation ability. The characteristics of this sample allow for the analysis of how age and gender may affect such self-reported navigation ability measures across the lifespan. The limited amount of literature on this topic leads to the following hypotheses: older individuals overestimate their navigation performance in comparison to younger individuals. Furthermore, females rate their navigation ability substantially lower than males.

The inclusion of the Wayfinding Questionnaire also allows for examination of the ‘Spatial anxiety’ subscale. Spatial anxiety refers to a feeling of anxiety specifically related to spatial situations, such as a fear of getting lost. This phenomenon is often found to be elevated in individuals with navigation complaints and shows informative variation in mentally healthy individuals (e.g. van der Ham et al., 2013). It should be noted that the concept of spatial anxiety lacks elaborate empirical support and it remains to be determined whether it is a specific, separate form of anxiety or an expression of general anxiety in spatial situations. In relation to gender, several reports indicate that spatial anxiety is rated higher by females compared to males (e.g. Lawton, 1994; Lawton & Kallai, 2002). Studies specifically examining spatial anxiety in relation to age have not been reported. Other forms of anxiety have been found to be common with older age (Beekman et al., 1998), which may also be expected for spatial anxiety.

As it is well established that several questionnaires on navigation ability have experimental and clinical merit, this study aims to investigate the potential impact individual factors may have on their outcomes. Literature on this matter is limited, but suggests substantial impact of age and gender on these measures. This may have particular implications for the use of self-reported levels of cognitive performance in both experimental and clinical settings. Experimentally, such measures are frequently used to create meaningful subgroups of participants and clinically such self-reports are heavily relied on in the screening process for medical conditions. Age and gender are typically not considered in the interpretation of such self-reports. Age and gender should potentially be included in such interpretations to equate outcomes of different demographic groups and create meaningful individualized analyses.

2. Methods

2.1. Participants

Figure 1 provides an illustration of the total sample of participants in relation to the sample included in the analyses reported here. A total of 7771 participants (5027 females, 2736 males, 8 nonbinary, age range 8–100) performed the experiment in full. Out of these 7596 participants, 462 were children between the ages of 8–17, who were removed from the sample, as were the nonbinary participants. Only 7 participants were over the age of 89. To keep the age bins equal and informative, the very small number of participants over 89 years of age was also removed. Lastly, the participants were selected based on their completion of the full questionnaire. The remaining 7150 participants were adults 18–89 years old. The study was approved by the local ethical committee at Leiden University, and in accordance with the declaration of Helsinki (2013) each participant provided informed consent prior to the experiment. Children under the age of 16 had to provide the consent of a parent or legal guardian. Individuals with neurological or psychiatric conditions were explicitly asked to abstain from participation. Recruitment occurred through a variety of national and local media, organized by The Weekend of Science, a Dutch annual event organized by the Secretary of Education, Science and Culture, to promote science to the general public. Data collection ran from October 2017-November 2018. The experiment was introduced to the participants as a serious and formal experiment, open to all healthy individuals, aged 8 and up.

Figure 1. Flowchart of participants. All participants who opted to fill out the Wayfinding Questionnaire in full are included, leading to the final sample of 7150 participants.

2.2. Materials

The experiment as used here is described in full by van der Ham, Claessen, Evers, and van der Kuil (2020). It was made available through a web-based environment (www.navigerenkunjeleren.nl). The experiment consisted of demographic questions, a navigation task battery consisting of five tasks, and the Wayfinding Questionnaire as an optional addition to the experiment. Out of a total of 11.887 participants 60.2% volunteered to fill out the Wayfinding Questionnaire in full, resulting in the sample of participants discussed here. A comparison of the demographics of the total sample and those who filled out the WQ in full showed that they are highly similar in age composition, education level, and spatial experience (see Van der Ham et al., 2020). The demographic questions concerned gender, age, and education level. For purposes beyond the scope of the current study additional questions were used, aimed at measuring spatial experience and living environment.

For the navigation task battery, a short video was shown (69 s), of a route through a desert-like environment with muted colors. The route lead past eight distinguishable landmarks, with salient colors (oil drums, a shield, a crate, a boat, a car, a shipping container, a gemstone, and a buoy) placed at separate intersections. At the endpoint of the route, a spaceship was placed. The narrative used was that the participant had landed on an unknown planet and through the video would find their way to the space-ship that could take them back home. The instructions were to pay attention to all elements of the route, not revealing what specific questions would be asked afterwards.

Five different tasks followed the video, to reflect the cognitive complexity of navigation ability: landmark, location – egocentric, location – allocentric, path – route, and path – survey. As it was the only task including distractor items, the landmark task was always shown first, the order of the rest of the tasks was fully random. The landmark task entailed the presentation of eight items, four of which were present in the video, the other four were distractor items, leading to a chance level performance of 50%. Landmarks were randomly assigned to participants, ensuring all 8 landmarks were used throughout all measurements. In the location egocentric task, participants were shown a landmark and were asked which of six provided options showed an arrow pointing in the direction of the spaceship, at the end of the route. The six arrows would be exactly 60 degrees different from one another, covering 360 degrees in total. Chance level performance was therefore 16.7%, and a total of four trials were presented. Again, a random selection of landmarks was presented to each participant. For the location-allocentric task, participants were shown a landmark together with a map of the environment, with 4 possible locations indicated with the letters A, B, C, D. They were asked to indicate at which of the four locations the landmark was positioned. Therefore, chance level performance was at 25% for this task. Four trials were presented, one for each of four randomly selected landmarks. The path-route task entailed a response to the question in which direction the route continued for a given landmark. Depending on the landmark, two or three possible directions were provided; left, right, and straight ahead, mean chance level was 44% (range 37.5–50%). This was repeated for four randomly selected landmarks. The path – survey task consisted of 3 landmarks presented simultaneously, for which the two landmarks that were closest together should be selected. It was stressed that this should be measured from a bird’s eye perspective, and thus relying on the mental representation of the environment a participant had made. This was again repeated for four sets of landmarks. For all five subtasks, performance was measured in percentage accurate responses.

The Wayfinding Questionnaire (Claessen et al., 2016; de Rooij et al., 2017; van der Ham et al., 2013) consists of 22 items, covering three subscales: Navigation and orientation (11 items), Distance estimation (3 items), and Spatial anxiety (8 items). The Navigation and orientation subscale is considered to reflect self-reported navigation ability and the mean score on this subscale (range 1–7, low to high ability) is used in the analyses. The score on each subscale is reflected by the mean response to all items with the subscale.

2.3. Procedure and design

Participants started the experiment by providing informed consent by clicking the appropriate button on the opening screen, after reading the relevant information. Next, the demographic questions were presented. This was followed by a screen indicating the video would be played when the participant clicked a button, warning them to be focused on the video and to avoid any distractions during the experiment. Next, the five tasks were presented and the participant could either finalize their participation or opt to fill out the Wayfinding Questionnaire in its entirety. Total duration was around 15 min including the Wayfinding Questionnaire.

All participants received the same questions and tasks. The order of tasks was random, apart from landmark knowledge, which always came first, due to its content.

2.4. Statistical analyses

First, the relation between the objective measurements and the self-reported scores was determined by means of correlations between the 5 navigation tasks and the three subscales of the Wayfinding Questionnaire. Next, the quality of the self-reports, the ‘estimation quality’ was calculated by standardizing all five objective subtask scores into z-scores, based on the mean and standard deviation of all participants pooled together. The same approach was used to standardize the Navigation and orientation subscale of the Wayfinding Questionnaire. Estimation quality was operationalized as the standardized subjective navigation WQ subscale score subtracted from the mean standardized objective score of the five navigation tasks (Mean z-score (5 subtasks) – z-score (WQ Navigation and orientation subscale)). This resulted in a value that indicates underestimation when it is larger than 0 and overestimation when it is below 0. A value of 0 indicates a perfect estimation. An ANCOVA was performed with gender and age group as between subject factors, and education level, spatial experience and spatial anxiety as covariates. The following age groups were composed: 18–29, 30–39, 40–49, 50–59, 60–69, 70–79, 80–89 years old. In addition to examining estimation quality, we similarly analyzed the reported levels of spatial anxiety; with an ANCOVA with age group and gender as between subject factors, and education level and spatial experience as covariates. Significant effects including age group were tested with Bonferroni corrected post-hoc tests.

3. Results

The mean scores for both the objective and the subjective measures are provided in Table 1 for each age group. First, the correlations between each of these measures was calculated (see Table 2). Each of the objective and subjective measures correlate significantly with one another, with the exception of distance estimation with the path route task and spatial anxiety with the location egocentric task. All correlations are positive, except for spatial anxiety, which correlates negatively with the objective and subjective performance measures, indicating lower spatial anxiety level is linked to higher spatial performance and with higher subjective level of spatial performance.

Table 1. Descriptive statistics and mean scores of all adults (18–89 years old).

Age group	N	N Female	N Male	Age	Objective measures					Subjective measures
					LM	LE	LA	PR	PS	NO	DE	SA
18–29	1034	750	284	23.2 (3.2)	0.91 (0.11)	0.33 (0.24)	0.60 (0.27)	0.73 (0.23)	0.57 (0.24)	4.70 (1.17)	3.80 (1.35)	3.02 (1.24)
30–39	604	410	194	34.6 (2.9)	0.89 (0.12)	0.31 (0.23)	0.58 (0.29)	0.72 (0.22)	0.59 (0.24)	4.84 (1.17)	4.23 (1.40)	2.74 (1.23)
40–49	902	612	290	44.8 (2.9)	0.88 (0.13)	0.31 (0.23)	0.54 (0.29)	0.70 (0.24)	0.58 (0.25)	4.76 (1.22)	4.36 (1.38)	2.87 (1.31)
50–59	1646	1164	482	54.8 (2.8)	0.87 (0.13)	0.30 (0.22)	0.51 (0.28)	0.67 (0.24)	0.61 (0.26)	4.63 (1.20)	4.28 (1.37)	3.01 (1.39)
60–69	1969	1239	730	64.3 (2.9)	0.86 (0.13)	0.30 (0.23)	0.46 (0.28)	0.61 (0.24)	0.61 (0.26)	4.75 (1.14)	4.44 (1.36)	3.07 (1.41)
70–79	890	460	430	73.1 (2.7)	0.84 (0.13)	0.31 (0.22)	0.43 (0.28)	0.57 (0.25)	0.59 (0.27)	4.88 (1.11)	4.59 (1.27)	3.04 (1.32)
80–89	105	50	55	82.7 (2.4)	0.84 (0.14)	0.29 (0.22)	0.40 (0.28)	0.56 (0.25)	0.55 (0.25)	4.86 (1.11)	4.59 (1.16)	3.16 (1.39)

Objective measures: LM = Landmark, LE = location egocentric, LA = location allocentric, PR = path route, PS = path survey, Subjective measures, Wayfinding Questionnaire subscales: NO = navigation and orientation, DE = distance estimation, SA = spatial anxiety.

Standard deviation in parentheses.

Table 2. Correlation matrix for all five objective navigation measures and the three subjective navigation measures.

	LM	LE	LA	PR	PS	NO	DE
LE	.037**	–
LA	.132***	.060***	–
PR	.116***	.056***	.139***	–
PS	.105***	.052***	.111***	.062***	–
NO	.065***	.041***	.080***	.113***	.057***	–
DE	.010	−.029*	.075***	.010	.066***	.627***	–
SA	−.054***	−.018	−.083***	−.086***	−.030*	−.542***	−.384***

Objective measures: LM = Landmark, LE = location egocentric, LA = location allocentric, PR = path route, PS = path survey, Subjective measures, Wayfinding Questionnaire subscales: NO = navigation and orientation, DE = distance estimation, SA = spatial anxiety.

* p < .05.

** p < .01.

*** p < .001.

Next, estimation quality was calculated for each participant, resulting in a single value reflecting objective navigation performance, subtracted by subjective navigation performance (see Figure 2). Education level, F (1,7150) = 25.64, p < .001, η²p = .004, spatial experience, F (1,7150) = 9.85, p < .005, η²p = .001, and spatial anxiety, F (1,7150) = 1600.03, p < .001, η²p = .183, all showed significant main effects in estimation quality and were therefore included as covariates in the ANCOVA. The ANCOVA showed a significant main effect of gender, F (1,7150) = 116.66, p < .001, η²p = .016, as well as age group, F (6,7150) = 25.60, p < .001, η²p = .021. The interaction between both variables did not reach significance, F < 1.0. For gender, the data showed that males had a significantly lower estimation quality, indicating overestimation (negative score), compared to females, who showed underestimation (positive score). For age, a gradual decline with higher age was found, with significant differences between each of the first four age groups (18–59) and the oldest three groups (60–89) (p < .001 in all cases). Additionally, estimation quality score was significantly higher for the 60–69 group, in comparison to the 70–79 group (p = .001). No significant interaction of age group and gender was present, F < 1.0.

Figure 2. Mean estimation quality for each age group and gender. Mean estimation quality = standardized subjective navigation WQ subscale score subtracted from the mean standardized objective score of the five navigation tasks. Error bars reflect standard error of the mean.

For the spatial anxiety subscale the effects of age group and gender were also analyzed (See Figure 3). As education level, F (1,7150) = 57.27, p < .001, η²p = .008, and spatial experience, F (1,7150) = 137.78, p < .001, η²p = .019, both showed significant main effects for spatial anxiety, they were included as covariates in the analyses. This ANCOVA showed a significant main effect of gender, F (1,7150) = 179.51, p < .001, η²p = .025, and of age group, F (1,7150) = 4.33, p < .001, η²p = .004. Females reported a higher level of spatial anxiety than males. With the Bonferroni corrected alpha of .05/7 = .007, the difference between the 18–29 and 30–39 year olds was significant (p = .003). Participants in their thirties reported significantly lower rates of spatial anxiety than those in their twenties. No significant interaction of age group and gender was present, F < 1.0.

Figure 3. Mean spatial anxiety scores for each age group and gender. Error bars reflect standard error of the mean.

4. Discussion

In this study we aimed to investigate subjective self-reports of cognition in relation to objective cognitive measures and used spatial navigation as an example case. The effect of gender and age may have a significant impact on the use of cognitive self-reports in a range of applications including medical, educational, vocational and experimental applications. Previous research is scarce but supports a significant impact of both age and gender on self-reports of navigation ability. The accuracy of self-reported performance may drop with older age and males may provide higher self-reports than females (e.g. Condon et al., 2015; Taillade et al., 2016). With a very large-scale examination of both subjective and objective navigation measures we studied this matter across the lifespan, with inclusion of a self-report measure of spatial anxiety.

The quality of self-reported ability scores is substantial, as evidenced by significant correlations between the subjective and objective scores. The Navigation and orientation subscale of the Wayfinding Questionnaire correlates with all other measures, whereas the Distance estimation scores are somewhat less convincing. Therefore, the Navigation and orientation subscale in particular can be considered an informative measure of navigation performance. The same procedure was followed for the Spatial anxiety scores, and as expected these are negatively related to objective performance which is in line with previous findings (e.g. van der Ham et al., 2013). With higher navigation performance comes lower spatial anxiety.

As reported in a related study, a clear age related decline was found for navigation ability overall (Van der Ham et al., 2020), which is in line with other large scale reports on the impact of age on spatial cognition (Coutrot et al., 2018; Head & Isom, 2010; Moffat, 2009). The current results substantiate previous findings with regard to age. With aging comes a clear decline of the estimation quality as defined here. This means that the older the participants, the more they overestimated their objective ability. When examining the raw scores, this effect appears to be specifically due to stable subjective scores across adulthood, while objective performance drops with age. A likely explanation for this pattern is that individuals may continue to refer to their performance at a younger age and not be aware of age-related decline taking place (Taillade et al., 2016). None of the age related effects were affected by gender and thus indicate highly similar aging processes for males and females on these measures. It should be noted that no information concerning neurological health was available, and as such there is a risk of pathological aging, in particular for the older participants. Participants were asked to refrain from participation if they were familiar with any such conditions.

There was a distinct difference regarding gender, with an overall overestimation for males and underestimation for females. This further clarifies gender differences in subjective cognitive ratings. Here we see that both genders deviate from estimation quality of 0, which indicates a deviation from a fully accurate subjective score. Both overestimation and underestimation take place. Together with the aging pattern, this shows that the most accurate self-reports are provided by the youngest adult males and oldest adult females. Young adult females and old adult males show the largest misestimation in opposite directions.

In addition to subjective reports on cognition, spatial anxiety was also of interest and was included as one of the three subscales of the Wayfinding Questionnaire. In agreement with existing findings, spatial anxiety was present to a larger extent in females compared to males. The impact of age was limited, with participants in their thirties showing the lowest level of spatial anxiety especially compared to younger adults, after which visual inspection of the data indicates a gradual increase. This pattern matches the aging process found for other forms of anxiety (Beekman et al., 1998).

The current findings show that the Wayfinding Questionnaire is an appropriate tool to assess navigation ability in a brief and simple way. Yet age and gender appear to be important, especially in adults, and should therefore be considered factors in the interpretation of the scores. Although self-reports are clearly correlated with objective performance, we have found tendencies to over- and underestimate. When used to detect cognitive problems in a clinical setting, appropriate norm tables should be used. To that end, the current dataset provides sufficient input given its substantial size and age range. Furthermore, brief forms of objective assessment are also recommended, to further explore potential cases of navigation impairment.

In short, previous research has shown conflicting findings with regard to self-reports of cognitive abilities, which are especially relevant in the detection of pathological aging. The current findings show that for spatial cognition in particular, individuals are able to provide informative estimates of their cognitive performance, but they show systematic biases which can be largely explained by the age and gender of the individual. Consideration of such biases is recommended when implementing self-report measures of cognition and considering the potential impact these biases may have on objective level of cognitive functioning.

Disclosure statement

No potential conflict of interest was reported by the author(s).