2,057
Views
0
CrossRef citations to date
0
Altmetric
General

Changes in perceived ageism during the COVID-19 pandemic: impact on quality of life and mental well-being among Dutch adults aged 55 and older

, , , &
Pages 2490-2498 | Received 21 Dec 2022, Accepted 20 Mar 2023, Published online: 28 Apr 2023

Abstract

Objectives: The COVID-19 pandemic brought ageism to the forefront of public discourse. Negative ageism incurs more negative self-perceptions of aging, which affects physical and mental functioning. Whether negative ageism as perceived and experienced by older adults has worsened as the pandemic lingered, and how such changes impact quality of life (QoL) and mental well-being (MWB), remain urgent questions.

Method: In a sample of adults aged 55 or older (n = 500), we aimed to address this by administering the Perceived Ageism Questionnaire twice during the pandemic (T1: between October 2020 and May 2021; T2: on average 45 wk after T1).

Results: Higher levels of perceived negative ageism were associated with lower QoL and MWB, at least partially through its unfavorable effects on self-perceptions of aging, even after controlling for ageism experiences in the preceding year (at T2, corrected for T1). Furthermore, we found that perceived negative ageism increased from T1 to T2, which had negative implications for QoL/MWB. Opposite effects were found for perceived positive ageism, although less consistently.

Conclusion: These patterns reveal that ageism as perceived and experienced by adults of 55 or older became stronger and more negative throughout the COVID-19 pandemic, which had detrimental implications for individuals’ QoL and MWB. These disconcerting findings emphasize the importance of combatting negative ageism in our society.

Introduction

In response to the outbreak of the COVID-19 pandemic, governments adopted strict measures to contain the virus and protect the older generation, who were among the most vulnerable and at high-risk members of our society (Remuzzi & Remuzzi, Citation2020). Despite good rationale for preventing acute health impacts, these measures amplified intrapersonal and interpersonal ageist perceptions and brought negative ageism (i.e. stereotyping, prejudices and discrimination against older adults on the grounds of their age) to the forefront of public discourse (Naughton et al., Citation2021; Silva et al., Citation2021). Age-based narratives in the news media grew more negative, and older adults were the focus of public and political (online) discussions. They were portrayed as a homogenous and vulnerable group of individuals that were unable to contribute much to society (Ayalon et al., Citation2021; Previtali et al., Citation2020; Swift & Chasteen, Citation2021), and who had to be protected or fought for, rather than fought along with (Lagacé et al., Citation2021). This led to the misconception that the virus only affected older adults (Aronson, Citation2020; Swift & Chasteen, Citation2021), and prompted governments to introduce age-related guidelines, such as special early shopping hours for older adults. In many countries (including the Netherlands), these were followed by legislative proposals to ‘isolate the elderly’, with some governments even implementing these (see for overview: WHO, Citation2021b). Using chronological age as a criterion for responses and decision making increases pigeonholing and overlooks the diverse capacities and needs of older adults (Ayalon et al., Citation2021). Moreover, the distinction between young and old also led younger individuals to develop more negative views about aging, with some even starting to direct their anger and resentment about the situation towards older adults (Ayalon et al., Citation2021).

Frequent exposure to stereotypes, prejudices and discrimination on the basis of one’s age can have significant consequences for the physical and mental health of older individuals. Negative ageism has been linked to earlier death, slower recovery from disability, and worsened mental health and well-being (Han & Richardson, Citation2015; H. Kim et al., Citation2019; Levy et al., Citation2002; Lyons et al., Citation2018; WHO., Citation2021a), and therefore negatively impacts successful aging (Swift et al., Citation2017).

A commonly described pathway through which ageism operates, is via altered self-perceptions of aging (Han & Richardson, Citation2015; H. Kim et al., Citation2019; Kotter-Grühn & Hess, Citation2012). When individuals are frequently exposed to negative age-related stereotypes, prejudices and discrimination, these can become self-fulfilling prophecies and shape individuals attitudes about their own aging (Marques et al., Citation2020). This can be explained through either the stereotype embodiment theory (Levy, Citation2009), postulating that stereotypes are internalized throughout the lifespan and can have detrimental impacts on health when these become self-relevant, or stereotype threat (Swift et al., Citation2017), the predicament in which individuals are afraid to conform to existing stereotypes, as a result of which they actually perform worse and thereby impact their attitude towards aging. Subsequently, this can lead to lower levels of physical and functional (mental) health, cognitive functioning, and resilience (e.g. Bellingtier & Neupert, Citation2018; Levy et al., Citation2009; Meisner, Citation2012; Pikhartova et al., Citation2016).

In line with these findings, the experience of undergoing negative ageism in society in response to the COVID-19 pandemic has been shown to negatively impact older adults’ self-perceptions of aging (Kornadt et al., Citation2021), which has in turn been associated with increased loneliness and psychological distress (Andrés Losada-Baltar et al., Citation2021). Hence, COVID-19 related ageism has justifiably been considered as a major public (mental) health concern (Kessler & Bowen, Citation2020).

As the pandemic faced multiple waves, ageism continued to operate, with strong discussions on lockdown policies and vaccination campaigns (i.e. older adults received their shots first, resulting in major, sometimes heated public debates). Consequently, ageism as perceived and experienced by older adults during the pandemic may not only have shifted since its onset, but may potentially also have exacerbated as the pandemic lingered. Improving our understanding of these changes is of considerable importance, as increased perceptions of negative ageism could have (even more) detrimental implications for older adults’ quality of life (QoL) and mental well-being (MWB) in the long-run. To clarify, QoL refers to the cognitive appraisal of one’s life and is affected by factors like interpersonal relationships, health, and social and physical environment, whereas MWB refers to the emotional response to what life is like and reflects the presence of positive emotions and contentment (Upton & Upton, Citation2015).

In the current study, we aimed to examine whether the level of perceived negative ageism (PNA)—as assessed with the 5-item PNA subscale of the Perceived Ageism Questionnaire (PAQ-8; Brinkhof et al., Citation2022)—changed during the pandemic, and what the implications for such changes are for individuals’ QoL and MWB. In addition, to obtain a more detailed picture of the role and effects of ageism during the pandemic, we also looked at the level of perceived positive ageism (PPA), using the 3-item PPA subscale of the PAQ-8. Although studied less thoroughly, positive and appreciated forms of ageism (e.g. the stereotype that older adults are wise) have been associated with more positive health outcomes (e.g. Fasel et al., Citation2021; Hausdorff et al., Citation1999; Levy, Citation1996), and positive beliefs about the impact of ageing have been associated with improved cognitive functioning (Brown et al., Citation2021). Potential decreases in PPA could have negative implications for QoL and MWB as well.

A large sample of adults aged 55 or older (N = 500)Footnote1 completed two measurement waves during the pandemic (T1 and T2), both holding the two-dimensional PAQ-8 and measures of QoL and MWB. In addition, self-perceptions of aging were assessed at T1. Using a series of analyses, we aimed to scrutinize the underlying relationships between (changes in) these variables. We first examined the short-term relationships between perceived ageism and both QoL and MWB to confirm some previously established phenomena in our sample (aim 1). To this end, we evaluated the direct, as well as indirect (via self-perceptions of aging) effects of PNA/PPA on QoL/MWB (all assessed at T1; i.e. cross-sectional; aim 1a). Subsequently, the association between PNA/PPA and QoL/MWB at T2, irrespective of previous ageism experiences (T1), was determined (aim 1b). We hypothesized that in both analyses PNA would be negatively related to QoL and MWB, at least partially through its unfavorable effects on individuals’ self-perceptions of aging (aim 1a). Opposite effects were anticipated for PPA.

Next, we assessed whether the level of perceived ageism had changed during the pandemic by directly comparing PNA and PPA scores at T1 and T2 (i.e. longitudinal; aim 2). We hypothesized that PNA would increase, whereas PPA would dissipate over time. Finally, we determined the consequences of (potential) changes in PNA and PPA for individuals’ QoL and MWB (aim 3). We anticipated both QoL and MWB to be negatively affected by increases in PNA and by decreases in PPA.

Methods

Participants

Participants were drawn from a larger pool of adults aged 55 or older of an ongoing online survey on successful aging and resilience in the Netherlands (Brinkhof et al., Citation2021), which consists of both questionnaires and tests. Individuals were eligible to participate if they were living in the Netherlands, and had sufficient command of the Dutch language, good vision and no dementia diagnosis. Moreover, participants had to be able to use a computer, laptop, or tablet independently. The first 500 participants (305 female, age: M 69.1, SD 6.60, 55–90) that completed both the original inventory (baseline package, T1; see Brinkhof et al., Citation2021), as well as a follow-up inventory (a newly added package of questionnaires in the same study; T2) were included here.Footnote2 As determined by using the categories of the Dutch Verhage (Citation1964) scale, 86% of the participants were highly educated (categories 6 and 7).

Design

Participants completed the original inventory (see Brinkhof et al., Citation2021) between early October 2020 and late May 2021, and the follow-up inventory between early September 2021 and mid October 2021 (see Supplement A for details on the state of the pandemic in the Netherlands during these months). The average time interval between the two measurements was 45 wk (range: 14–52 wk). As the PAQ-8 asks individuals to rate the level of perceived ageism throughout the past year (see Materials), the time windows were not entirely non-overlapping (the overlap amounting to 7 wk on average).

Materials

Perceived ageism

The 8-item Perceived Ageism Questionnaire (PAQ-8) was used to assess the level of PNA and PPA at T1 and T2 during the COVID-19 pandemic (Brinkhof et al., Citation2022). The PAQ-8 comprises five items reflecting negative forms of ageism (PNA subscale; e.g. being approached as a child because of one’s age) and 3 items reflecting positive forms of ageism (PPA subscale; e.g. ‘being considered as a wise or sensible person because of one’s age). Each subscale showed good reliability (PNA α: 0.82–0.87; PPA α: 0.77). Each of the items described different situations or attitudes that older adults may have experienced or encountered in the past year. Participants had to report how often each situation had occurred using a 5-point Likert Scale (1 = never, 2 = barely, 3 = sometimes, 4 = often, 5 = very often). PNA and PPA subscale scores were calculated by taking the sum score of their corresponding items (PNA: 5—25; PPA: 3–15).

Quality of life

The World Health Organization Quality of Life (WHOQOL)-OLD (Power et al., Citation2005; α: 0.87–0.88) was used to assess QoL (at T1 and T2) based on six subscales of four items each: (1) sensory abilities, (2) autonomy, (3) satisfaction with past, present, and future activities and achievements in life, (4) social participation, concerns, (5) worries and fears about death and dying, and (6) being able to have personal and intimate relationships (e.g. All 24 items were scored on a 5-point Likert Scale, with different wordings, and summed to a total QoL score. Some items were reverse scored prior to summation, such that higher scores were indicative of better QoL (24—100).

Mental well-being

The 14-item Warwick Edinburgh Mental Wellbeing Scale (WEMWBS; Tennant et al., Citation2007; α: 0.82–0.90) was used to measure MWB (at T1 and T2). All 14 items, all addressing positive aspects of mental health (e.g. ‘I’ve been feeling relaxed’), were scored on a 5-point Likert scale (1 = never, 2= barely, 3 = sometimes, 4 = often, 5 = always) and summed to a total of 14 to 70. Higher scores indicated better MWB.

Self-perceptions of aging

At T1, the shortened version of the Aging Perceptions Questionnaire (APQ-S) was administered to assess participants’ self-perceptions of aging (Slotman et al., Citation2015). The APQ-S consist of 21 items, with seven subscales that tap onto seven different dimensions of aging. Some dimensions point towards more negative perceptions (i.e. timeline chronic, timeline cyclical, consequences negative, emotional representations and control negative; e.g. ‘Getting older restricts the things that I can do’), whereas others contribute to more positive attitudes towards aging (i.e. consequences positive and control positive; e.g. ‘Whether I continue living life to the full depends on me’; see Slotman et al., Citation2015; α: 0.66–0.82). Answers were given on a 5-point scale, ranging from strongly disagree (1) to strongly agree (5), and items were summed to subscale scores, ranging from 3—15. Finally, the subscale scores of the positive dimensions were reverse scored, and all seven subscale scores were averaged to obtain an overall negative self-perceptions of aging measure.

Analyses

All analyses were conducted in R 4.1.2 (R Core Team, Citation2021), with alpha set at 0.05, unless specified otherwise. For all analyses, the sampling distribution of the variables was assumed to be normal due to the large sample size.

Aim 1: short-term relationships

To evaluate the cross-sectional relationships among perceived ageism, self-perceptions of aging, and the outcome variables QoL and MWB, we performed two mediation analysis, for PNA and PPA separately, on data obtained at T1. This analysis was conducted using the sem function of the Lavaan package, version 0.6-11; Rosseel, Citation2012), with Maximum Likelihood Estimation, and 5000 bootstraps. We determined the direct effect of PNA and PPA on QoL (c) and MWB (e), as well as the indirect effect (a*b and a*d), through one’s self-perceptions of aging. Both the direct and total indirect effects were expected to be negative and positive for PNA and PPA, respectively.

To better understand the short-term effects of perceived ageism, independent from the (emotional) baggage taken from the preceding stage of the pandemic, four multiple regression models were adopted. Each model included QoL or MWB as dependent variable, and either PNA or PPA scores of both T1 and T2 as independent predictors. In addition, an interaction term between PNA/PPA at T1 and T2 was included. As the time between T1 and T2 may affect the impact of PNA/PPA at T1 on MWB/QoL at T2, the factor window was also included as a potential moderator for this specific relationship. Age was added as a covariate, and potential moderating effects were also evaluated. We compared the models with and without the moderation terms (i.e. PNA/PPA T1*T2, PNA/PPA T1*Window, and or PNA/PPA T1*Age). Per default, the output of the model with only main effects (i.e. PNA/PPA T1 and T2, and Age) was interpreted, unless a more complex model showed a significantly better fit (i.e. Akaike Information Criterion of at least 2 units lower). To interpret significant moderation terms, the emtrends function of the emmeans package (Lenth, Citation2021) was used to conduct a simple slope analysis at three levels of the moderator.Footnote3 These levels were determined by splitting the data into three groups, representing the upper, middle and lower third of the distribution of the moderator, and calculating the median of each group. By default, participants were divided in bins of equal width. However, in case of a skewed distribution of the moderator, data was divided in bins of equal size (as far as possible).

Aim 2 and 3: changes in perceived ageism and its corresponding impact on QoL and MWB

Four linear mixed-effects models (with Maximum Likelihood estimation; Bates et al., Citation2020) were tested to establish whether PNA/PPA changed over time (within-subject factor: T1 and T2), and whether potential changes were moderated by age, T1 levels of QoL/MWB, and the time between T1 and T2 (window; all between-subjects factors). We adopted a two-step approach, in which we evaluated the role of time first, after which a more complex model, including one, two of three of the moderation effects, was considered. Similar model selection criteria were used as described for aim 1b. Significant moderation terms were further interpreted using the emmeans function of the emmeans package. The levels of the moderator were determined using a similar approach as described for aim 1b.

Four other linear mixed-effects models (with Maximum Likelihood estimation) were adopted to evaluate whether (potential) changes in PNA/PPA were followed by changes in QoL/MWB. This was done by calculating a PNA/PPA difference score (i.e. T2—T1, with positive scores being indicative of an increase from T1 to T2) and including the interaction term with time (as within subject factor) as predictor. The moderating effects of time between T1 and T2 (window*time) and age (age*time) were also included. For each of the four mixed-effects model, the best fitted model was selected based on previously described criteria, and post-hoc analysis was completed with the emmeans function.

Results

Aim 1: short-term relationships

The bivariate correlation coefficients of our variables of interest, at T1 and T2, are shown in . On a cross-sectional level, both PNA and PPA was found to correlate with QoL and MWB, with negative associations for PNA and positive associations for PPA, at least partially through its effects on self-perceptions of aging (). For both PNA and PPA, the relationship with QoL was the strongest. PPA was more strongly related to MWB than PNA. These results are summarized in the left column of .

Figure 1. Short-term relationships between perceived negative ageism (PNA)/perceived positive ageism (PPA), and quality of life (QoL)/mental well-being (MWB). This figure illustrates the outcomes of two separate analyses. The left column shows the results of the two mediation models (a: PNA, b: PPA). The right column depicts the outcomes of the four linear regression models, separately for QoL and MWB, and PNA (a) and PPA (b). For this analysis, moderator window (i.e. time between T1 and T2) is not included in this illustration, as it was not included in any of the selected models. In both columns, only significant (p < .05) pathways are shown. Red lines denote negative associations, and blue lines denote positive associations. SPoA: self-perceptions of aging.

Figure 1. Short-term relationships between perceived negative ageism (PNA)/perceived positive ageism (PPA), and quality of life (QoL)/mental well-being (MWB). This figure illustrates the outcomes of two separate analyses. The left column shows the results of the two mediation models (a: PNA, b: PPA). The right column depicts the outcomes of the four linear regression models, separately for QoL and MWB, and PNA (a) and PPA (b). For this analysis, moderator window (i.e. time between T1 and T2) is not included in this illustration, as it was not included in any of the selected models. In both columns, only significant (p < .05) pathways are shown. Red lines denote negative associations, and blue lines denote positive associations. SPoA: self-perceptions of aging.

Table 1. Bivariate correlation coefficients of variables of interest.

Table 2. Direct and indirect effects, via self-perceptions of aging, of perceived negative ageism (PNA) and perceived positive ageism (PPA) on quality of life and mental well-being.

When taking into account ageism experiences in the preceding stage of the pandemic, a similar, yet more refined pattern arose (see right column ; the model comparison statistics can be found in Table S1 of Supplement B). That is, PNA was most strongly associated with QoL/MWB among those reporting relatively low levels of PNA at T1 (see Table S2 of Supplement B for statistics). Similarly, PPA was most strongly associated with MWB among those reporting relatively high levels of PPA at T1. The relationships between PPA and QoL at T2 was not moderated by previous ageism experiences at T1. This suggests that short-term effects of perceived negative ageism (on QoL and MWB) and positive ageism (on MWB) seem particularly strong when previous experiences were relatively moderate.

Aim 2: changes in perceived ageism during COVID-19

The PNA scores of the PAQ-8 were found to be significantly higher at T2 (M 8.19 SD 3.09) than T1 (M 7.49, SD 2.72), B = −0.35, SE = 0.05, t(499) = −6.94, p < .001. In contrast, PPA scores were lower at T2 (M 9.29, SD 2.19) than T1 (M 9.48, SD 2.33), B = 0.10, SE = 0.04, t(499) = 2.24, p = .026. Increases in PNA, but not PPA, were moderated by T1 levels of QoL and MWB (see ). Those who reported relatively low levels of QoL/MWB at T1 had the highest PNA levels at T1, and also showed the steepest increase, resulting in even larger differences in PNA at T2 (see Table S3 and Table S4 of Supplement B for the statistics). Increases in both PNA and PPA were not moderated by age, nor by the time elapsed between T1 and T2. These results suggest that perceived ageism was indeed altered during the COVID-19 pandemic, with increased PNA and decreased PPA, and that unfavorable levels of QoL/MWB in previous waves can increase susceptibility/vulnerability to PNA in the future.

Figure 2. Changes in perceived negative ageism (PNA) and perceived positive ageism (PPA) during the COVID-19 pandemic, and the consequences for quality of life (QoL) and mental well-being (MWB). T1: first measurement wave; T2: second measurement wave. (a) Red bars represent PNA scores and blue bars represent PPA scores. Only significant moderators (MWB/QoL) are included. (b) The moderating effect of changes in PNA are shown, for QoL and MWB, separately. (c) The moderating effect of changes in PPA for QoL is shown. (b and c) Standard deviations are not shown to improve visibility. +p < .07, *p < .05, **p < .01, ***p < .001.

Figure 2. Changes in perceived negative ageism (PNA) and perceived positive ageism (PPA) during the COVID-19 pandemic, and the consequences for quality of life (QoL) and mental well-being (MWB). T1: first measurement wave; T2: second measurement wave. (a) Red bars represent PNA scores and blue bars represent PPA scores. Only significant moderators (MWB/QoL) are included. (b) The moderating effect of changes in PNA are shown, for QoL and MWB, separately. (c) The moderating effect of changes in PPA for QoL is shown. (b and c) Standard deviations are not shown to improve visibility. +p < .07, *p < .05, **p < .01, ***p < .001.

Aim 3: health impact of changes in perceived ageism

Changes in PNA (and individual differences therein) were followed by corresponding changes in QoL and MWB (see Tables S5 and S6 of Supplement B for the statistics). That is, while at the group level we did not observe a decrease from T1 to T2 in either QoL, B = −0.09, SE = 0.14, t = −0.63, p = 0.532, or MWB, B = −0.06, SE = 0.10, t = −0.60, p = 0.542, increases in PNA were associated with a steep decrease in QoL/MWB, whereas decreases in PNA were associated with improved QoL/MWB (). Changes in PPA (and individual differences therein) were followed by corresponding changes in QoL (although the post-hoc contrast analysis showed only a marginally significant increase in QoL for those showing an increase in PPA; see ), but not by corresponding changes in MWB. This indicates that changes in both PNA and PPA had implications for QoL and MWB, albeit most strongly and consistently for changes in PNA. Finally, the time elapsed between T1 and T2 seemed to moderate the change in MWB, but none of the post-hoc contrasts were significant.

Discussion

The COVID-19 pandemic was accompanied by an outbreak of ageism (Ayalon et al., Citation2021). In the current study, we evaluated whether ageism as perceived and experienced by adults aged 55 or older has changed during the pandemic, and what the implications of such changes are for their quality of life (QoL) and mental well-being (MWB). The observed patterns reveal that perceived negative ageism (PNA) worsened during the pandemic, with corresponding declines in QoL and MWB. Opposite effects were found for perceived positive ageism (PPA), although less consistently.

Foremost, we confirmed some widely established relationships between PNA and PPA on the one hand, and (mental) health outcome variables on the other (e.g. Han & Richardson, Citation2015; Lyons et al., Citation2018). Both PNA and PPA were associated with QoL and MWB at the cross-sectional level at T1, at least partially through their relationships with self-perceptions of aging (aim 1a; cf. Han & Richardson, Citation2015; H. Kim et al., Citation2019; Kotter-Grühn & Hess, Citation2012). Hence, a more positive outlook on one’s own aging, was associated with better QoL and MWB. Being able to take into account previous ageism experiences, we also showed that the short-term relationships of both PNA (with QoL and MWB) and PPA (with MWB) were particularly strong when previous perceptions were relatively favorable (aim 1b). In the case of PNA, this may suggest that those individuals who have less prior experience with ageism were less capable of dealing with the negative stereotyping, prejudices and discrimination, and perhaps did not (yet) exhibit or develop adaptive coping strategies to cope with these experiences (I. H. Kim et al., Citation2016). This concurs well with the stress-inoculation hypothesis, which states that intermittent exposure to mild stressors can promote resilience to subsequent stressful episodes (Ashokan et al., Citation2016), and suggests that prior exposure to negative ageism may help to alleviate negative consequences for QoL/MWB in future situations. More negative self-perceptions of aging (i.e. associating aging with physical losses) can be adaptive, as these can help to mentally prepare individuals for serious age-related health problems, but such negative perceptions can be detrimental for functional health in the long-run (Wolff et al., Citation2015), which also conforms to our own findings (aim 3; see discussion below).

As for PPA, a potential mechanism through which earlier experiences exert their favorable influence on MWB is that PPA leads to more positive self-perceptions of aging, which subsequently leads individuals to more strongly invest in coping efforts immediately after a negative experience (cf. Wolff et al., Citation2015; Levy & Myers, Citation2004). Moreover, PPA may also have a general mood-lifting effect (Disabato et al., Citation2016; Fasel et al., Citation2021). Both favor MWB as well as a more optimistic and positive view of the world. Subsequently, new positive ageism experiences may be identified and assimilated faster.

As anticipated, we found that PNA increased, whereas PPA decreased during the pandemic (aim 2). This is in line with the predominantly negative portrayal of older adults and public discourse during the pandemic (e.g. Silva et al., Citation2021), and shows that ageism has not only exacerbated since the onset of the pandemic, but that the perception of ageism has gotten progressively worse over time during the pandemic as well. Note that the time elapsed between T1 and T2 did not affect the observed changes in PNA/PPA. While both PNA and PPA was positively related to age at both time points, consistent with previous findings (e.g. Ayalon et al., Citation2019), even individuals aged 65 or younger sometimes reported relatively high levels of PNA and PPA. This confirms that individuals can experience ageism directed towards them at a relatively young age (Brinkhof et al., Citation2022), and that ageism may even commence before the age of 55. Interestingly, the changes in PNA and PPA were not moderated by age, which suggest that age does not explain any of the heterogeneity in changes in PNA/PPA among individuals. Moreover, although it seems plausible to assume that effects would be more negative among older participants to whom old age is more self-relevant, we found that the impact of changes in PNA/PPA for individuals’ QoL/MWB was also independent of age, as discussed further below.

Interestingly, previous levels of QoL/MWB were found to moderate the increase in PNA (but not PPA) over time, with stronger increases for those who reported low levels of QoL/MWB at T1. This supports the idea that individuals with an unfavorable (mental) health status are biased and inclined to perceive the world more negatively (see e.g. Ayalon, Citation2018; Brinkhof et al., Citation2022), thereby showing higher—and in the current study increased—levels of PNA (over time). This may give rise to a negative feedback loop, where increased exposure to negative ageism leads to more negative self-perceptions of aging and thereby QoL/MWB, which in turn evokes even more negative thoughts about aging (Kornadt et al., Citation2021) and thereby increases the likelihood of negative ageism being perceived. Note that self-perceptions of aging have been shown to grow more negatively during the pandemic (Kornadt et al., Citation2021), and that individuals that reported more negative self-perceptions of aging also showed stronger increases in psychological distress irrespective of age (Losada-Baltar et al., Citation2022).

Direct support for the detrimental implications of increases in PNA and decreases in PPA for individuals’ QoL and MWB was found in our final series of analysis. At the group level, the levels of QoL and MWB were similar at T1 and T2, suggesting no overall change over time (in accordance with previous findings; Brinkhof et al., Citation2021). However, we cannot exclude the possibility that QoL and MWB did fluctuate in between our two measurements (e.g. in a U-shaped curve; see Losada-Baltar et al., 2022) and returned to their initial T1 state at T2. Be that as it may, we found that individual differences in changes in perceived ageism from T1 to T2 were predictive for individual differences in changes in QoL and MWB (independent of age; aim 3). That is, individuals showing an increase in PNA, showed an corresponding decrease in QoL and MWB, and vice versa. The patterns corresponding to changes in PPA were less consistent: only individuals showing increased PPA showed marginally significant increases in QoL, but not MWB. This adds to the existing literature on the detrimental (long-term) impacts of negative ageism (e.g. Pikhartova et al., Citation2016), and adds new insights about the potential beneficial effects of positive ageism on functioning and mental health in the long-run (Dionigi, Citation2015). Future research is, however, needed to unravel the exact mechanisms through which PPA exerts its benefits on QoL/MWB, and explain why the effects of PPA are somewhat inconsistent and relatively moderate as compared to PNA. A possible explanation may be that positive forms of ageism can also be unwanted, or even be perceived as patronizing, thereby having negative impacts for some, but not all, individuals. Indeed, although the items of the PPA subscale of the PAQ-8 include ageist actions/behaviors that are less likely to be perceived as patronizing or uncomforting, these may still give some people the feeling of being implicitly and undesirably attributed/assigned to a group (i.e. being stereotyped). In contrast, negative forms of ageism are expected to be evaluated more consistently as negative, as these are less ambiguous and often more explicitly conveyed.

Although our study does now allow for predictions beyond the study scope, these findings do provide evidence to suggest that perceived ageism can have prolonged consequences for (older) adults’ QoL and MWB that may sustain after the pandemic. The extent to which current patterns remain in the prolonged stages or aftermath of the pandemic, may be evaluated in future research.

Interestingly, our results also suggest that over the course of the pandemic, PNA affects both MWB and QoL, although the latter more strongly so. At both T1 and T2, PNA was also more strongly related to QoL than MWB (aim 1), and the moderating effect of changes in PNA was stronger for QoL than MWB (aim 3). The differential impacts of PNA on QoL and MWB may well be related to the differential framings of the WHOQOL-OLD and WEMWBS. The WHOQOL-OLD aims to capture one’s cognitive appraisal of life, and focuses on certain issues that are particularly relevant for the social/daily life functioning of older individuals (Upton & Upton, Citation2015). The WEMWBS, on the other hand, reflects individuals’ emotional appraisal of their life and includes examples of experienced emotions that are less evidently associated with aging (Upton & Upton, Citation2015). Consequently, older adults’ QoL may be more prone to be affected by negative age-related stereotypes, prejudices, and discriminations (also see: Brothers et al., Citation2021). Moreover, the negative emotions that arise from negative ageism experiences may keep individuals from engaging in social and health-promoting behaviors (cf. Wolff et al., Citation2015), leading to decreased QoL in the long-run.

An important limitation to consider is that our sample consisted of predominantly mid-to highly educated individuals. Less educated individuals are known to perceive more ageism than their counterparts (Ayalon, Citation2018; Maurya et al., Citation2022; Palmore, Citation2001; Rippon et al., Citation2014), which may be exacerbated during the pandemic. Additionally, in the current study, we did not shed light on the role of gender, while men and women may respond differently to changes in people’s attitudes towards older individuals and the overall narrative during the pandemic. Other demographic characteristics, including marital or employment status, environment (e.g. urban versus rural), and culture may also have impacted the perceived ageism course during the COVID-19 pandemic, and might therefore provide an interesting avenue for future research as well (Bergeron & Lagacé, Citation2021; Rippon et al., Citation2014; Stokes & Moorman, Citation2016). The influence of these characteristics, as well as other potential risk and protective factors, on the adverse or mitigating effects of PNA and PPA on QoL/MWB, also warrant more in-depth investigation. For instance, woman may experience more negative consequences because they are also subjected to sexism (e.g. Rochon et al., Citation2021), whereas highly educated individuals may have greater access to resources and support to cope with the negative effects of PNA, which thereby alleviates the negative consequences for QoL/MWB. Certain personality characteristics (e.g. optimism) may also affect one’s level of resilience to the consequences of ageism (Ribeiro-Gonçalves et al., Citation2023; Wurm & Benyamini, Citation2014). Finally, in order to gain deeper insights into the potential protective and harmful effects of both forms of ageism, it is important to extend our prospective line of research by evaluating other mental and physical health outcomes.

One final aspect that is relevant to discuss, is the use and applicability of the WHOQOL-OLD for the current sample. The WHOQOL-OLD is often only applied for populations aged 60 or older (although it has been evaluated among adults aged 50—60 as well; Gobbens & Remmen, Citation2019), but we have chosen to use this instrument for all individuals to ensure consistency and comparability in our results. Importantly, we believe it is justifiable to use this instrument for all people aged 55 or older, as a person’s age can only provide a rough estimate of the life stage. That is, chronological age overlooks important factors such as physical and mental health, lifestyle and life experiences, which can all impact when and whether individuals experience certain age-related changes that may directly affect one’s QoL.

In sum, ageism as perceived and experienced by adults aged 55 or older became stronger and more negative during the COVID-19 pandemic, which had detrimental implications for their QoL and MWB. It is important to consider that the levels of ageism in our society will not simply disappear once the pandemic ends. The discourse and negative portrayal of older adults have most likely affected how they are perceived, and thereby also how the general public will approach older individuals. This could serve to sustain high levels of negative ageism. We therefore signal the urgency of taking actions to combat ageism and break the downward spiral to prevent negative consequences of ageism in the long run. It is important to improve awareness about often unrecognized and deeply ingrained ageism at the individual, inter-personal, and institutional levels, and educate the public about the far-reaching consequences for the physical and mental health of older individuals. Ultimately, combatting ageism may require a social awakening, and given the changed dynamics during the COVID-19 pandemic, now may be the right time to initiate this.

Copyright statement

The Warwick–Edinburgh Mental Well-being Scale was funded by the Scottish Government National Programme for Improving Mental Health and Wellbeing, commissioned by NHS Health Scotland, developed by the University of Warwick and the University of Edinburgh, and is jointly owned by NHS Health Scotland, the University of Warwick and the University of Edinburgh.

Data and code availability statement

The datasets presented in this article are not readily available because the datasets used and/or analyzed for the current study will only be made publicly available after completion of the overarching project, and will until that time only be available from the corresponding author on collaboration basis upon reasonable request. Requests to access the datasets should be directed to LB, [email protected].

Ethics approval and consent to participate

The study is approved by the local ethics committee of the University of Amsterdam (T1: 2020-CP-12556, T2: 2021-DP-13798) and is conducted in accord with relevant laws and institutional guidelines. Informed consent from study participants was obtained online. Participants provided consent by clicking an acceptance box at the bottom of the agreement.

Supplemental material

Supplemental Material

Download Zip (567.1 KB)

Acknowledgements

We would like to thank our participants who took part in this study.

Disclosure statement

The authors report no conflict of interest.

Additional information

Funding

This research was funded by the Centre for Urban Mental Health, a Research Priority Area at the University of Amsterdam.

Notes

1 The minimum age of 55 years is relatively lower than usually considered for older adults (i.e. 60 or 65 years old).

2 Other participants were used to validate the Perceived Ageism Questionnaire, as used in the current study (see Brinkhof et al., Citation2022).

3 The estimated trends reflect local behavior at three values of the moderator.

References

  • Aronson, L. (2020). Ageism is making the coronavirus pandemic worse. The Atlantic. https://www.theatlantic.com/culture/archive/2020/03/americas-ageism-crisis-is-helping-the-coronavirus/608905/
  • Ashokan, A., Sivasubramanian, M., & Mitra, R. (2016). Seeding stress resilience through inoculation. Neural Plasticity, 2016, 4928081. https://doi.org/10.1155/2016/4928081
  • Ayalon, L. (2018). Perceived age discrimination: A precipitator or a consequence of depressive symptoms? The Journals of Gerontology. Series B, Psychological Sciences and Social Sciences, 73(5), 860–869. https://doi.org/10.1093/geronb/gbw101
  • Ayalon, L., Chasteen, A., Diehl, M., Levy, B. R., Neupert, S. D., Rothermund, K., Tesch-Römer, C., & Wahl, H. W. (2021). Aging in times of the COVID-19 pandemic: Avoiding ageism and fostering intergenerational solidarity. The Journals of Gerontology. Series B, Psychological Sciences and Social Sciences, 76(2), e49–e52. https://doi.org/10.1093/GERONB/GBAA051
  • Ayalon, L., Dolberg, P., Mikulionienė, S., Perek-Białas, J., Rapolienė, G., Stypinska, J., Willińska, M., & de la Fuente-Núñez, V. (2019). A systematic review of existing ageism scales. Ageing Research Reviews, 54, 100919. https://doi.org/10.1016/j.arr.2019.100919
  • Bates, D., Maechler, M., Bolker, B., Walker, S. (2020). lme4: Linear mixed-effects models using “Eigen” and S4. https://cran.r-project.org/package=lme4
  • Bellingtier, J. A., & Neupert, S. D. (2018). Negative aging attitudes predict greater reactivity to daily stressors in older adults. The Journals of Gerontology. Series B, Psychological Sciences and Social Sciences, 73(7), 1155–1159. https://doi.org/10.1093/GERONB/GBW086
  • Bergeron, C. D., & Lagacé, M. (2021). On the meaning of aging and ageism: Why culture matters. University of Toronto Quarterly, 90(2), 140–154. https://doi.org/10.3138/utq.90.2.06
  • Brinkhof, L. P., De Wit, S., Murre, J. M. J., Krugers, H. J., & Ridderinkhof, K. R. (2022). The subjective experience of ageism: The Perceived Ageism Questionnaire (PAQ). International Journal of Environmental Research and Public Health, 19(14), 8792. https://doi.org/10.3390/ijerph19148792
  • Brinkhof, L. P., Huth, K. B. S., Murre, J. M. J., de Wit, S., Krugers, H. J., & Ridderinkhof, K. R. (2021). The interplay between quality of life and resilience factors in later life: A network analysis. Frontiers in Psychology, 12, 752564. https://doi.org/10.3389/fpsyg.2021.752564
  • Brinkhof, L. P., Ridderinkhof, K. R., de Wit, S., Krugers, H. J., & Murre, J. M. J. (2021). A Network Perspective on Successful Aging and Resilience in Later Life: A Protocol Paper. https://doi.org/10.31234/osf.io/bu7vq
  • Brothers, A., Kornadt, A. E., Nehrkorn-Bailey, A., Wahl, H. W., & Diehl, M. (2021). The effects of age stereotypes on physical and mental health are mediated by self-perceptions of aging. The Journals of Gerontology. Series B, Psychological Sciences and Social Sciences, 76(5), 845–857. https://doi.org/10.1093/GERONB/GBAA176
  • Brown, K. E., Kim, J., Stewart, T., Fulton, E., & McCarrey, A. C. (2021). Positive, but not negative, self-perceptions of aging predict cognitive function among older adults. International Journal of Aging & Human Development, 93(1), 543–561. https://doi.org/10.1177/0091415020917681
  • Cohen, J. (1988). Statistical power analysis for the behavioural sciences. Academic Press.
  • Dionigi, R. A. (2015). Stereotypes of aging: Their effects on the health of older adults. Journal of Geriatrics, 2015, 1–9. https://doi.org/10.1155/2015/954027
  • Disabato, D. J., Goodman, F. R., Kashdan, T. B., Short, J. L., & Jarden, A. (2016). Different types of well-being? A cross-cultural examination of hedonic and eudaimonic well-being. Psychological Assessment, 28(5), 471–482. https://doi.org/10.1037/pas0000209
  • Fasel, N., Vauclair, C. M., Lima, M. L., & Abrams, D. (2021). The relative importance of personal beliefs, meta-stereotypes and societal stereotypes of age for the wellbeing of older people. Ageing and Society, 41(12), 2768–2791. https://doi.org/10.1017/S0144686X20000537
  • Gobbens, R. J. J., & Remmen, R. (2019). The effects of sociodemographic factors on quality of life among people aged 50 years or older are not unequivocal: Comparing SF-12, WHOQOL-BREF, and WHOQOL-OLD. Clinical Interventions in Aging, 14, 231–239. https://doi.org/10.2147/CIA.S189560
  • Han, J., & Richardson, V. E. (2015). The relationships among perceived discrimination, self-perceptions of aging, and depressive symptoms: A longitudinal examination of age discrimination. Aging & Mental Health, 19(8), 747–755. https://doi.org/10.1080/13607863.2014.962007
  • Hausdorff, J. M., Levy, B. R., & Wei, J. Y. (1999). The power of ageism on physical function of older persons: Reversibility of age-related gait changes. Journal of the American Geriatrics Society, 47(11), 1346–1349. https://doi.org/10.1111/j.1532-5415.1999.tb07437.x
  • Kessler, E. M., & Bowen, C. E. (2020). COVID ageism as a public mental health concern. In the Lancet Healthy Longevity, 1(1), e12. https://doi.org/10.1016/S2666-7568(20)30002-7
  • Kim, H., Thyer, B. A., & Munn, J. C. (2019). The relationship between perceived ageism and depressive symptoms in later life: Understanding the mediating effects of self-perception of aging and purpose in life, using structural equation modeling. Educational Gerontology, 45(2), 105–119. https://doi.org/10.1080/03601277.2019.1583403
  • Kim, I. H., Noh, S., & Chun, H. (2016). Mediating and moderating effects in ageism and depression among the Korean elderly: The roles of emotional reactions and coping responses. Osong Public Health and Research Perspectives, 7(1), 3–11. https://doi.org/10.1016/j.phrp.2015.11.012
  • Kornadt, A. E., Albert, I., Hoffmann, M., Murdock, E., & Nell, J. (2021). Perceived ageism during the COVID-19-crisis is longitudinally related to subjective perceptions of aging. Frontiers in Public Health, 9, 679711. https://doi.org/10.3389/FPUBH.2021.679711
  • Kotter-Grühn, D., & Hess, T. M. (2012). The impact of age stereotypes on self-perceptions of aging across the adult lifespan. The Journals of Gerontology. Series B, Psychological Sciences and Social Sciences, 67(5), 563–571. https://doi.org/10.1093/geronb/gbr153
  • Lagacé, M., Doucet, A., Dangoisse, P., & Bergeron, C. D. (2021). The “vulnerability” discourse in times of COVID-19: Between abandonment and protection of Canadian Francophone older adults. Frontiers in Public Health, 9, 662231. https://doi.org/10.3389/fpubh.2021.662231
  • Lenth, R. V. (2021). emmeans: Estimated marginal means, aka least-squares means. https://cran.r-project.org/package=emmeans
  • Levy, B. R. (1996). Improving memory in old age through implicit self-stereotyping. Journal of Personality and Social Psychology, 71(6), 1092–1107. https://doi.org/10.1037/0022-3514.71.6.1092
  • Levy, B. R. (2009). Stereotype embodiment: A psychosocial approach to aging. Current Directions in Psychological Science, 18(6), 332–336. https://doi.org/10.1111/J.1467-8721.2009.01662.X
  • Levy, B. R., & Myers, L. M. (2004). Preventive health behaviors influenced by self-perceptions of aging. Preventive Medicine, 39(3), 625–629. https://doi.org/10.1016/J.YPMED.2004.02.029
  • Levy, B. R., Slade, M. D., Kunkel, S. R., & Kasl, S. V. (2002). Longevity increased by positive self-perceptions of aging. Journal of Personality and Social Psychology, 83(2), 261–270. https://doi.org/10.1037/0022-3514.83.2.261
  • Levy, B. R., Zonderman, A. B., Slade, M. D., & Ferrucci, L. (2009). Age stereotypes held earlier in life predict cardiovascular events in later life. Psychological Science, 20(3), 296–298. https://doi.org/10.1111/J.1467-9280.2009.02298.X
  • Losada-Baltar, A., Jiménez-Gonzalo, L., Gallego-Alberto, L., Pedroso-Chaparro, M. D. S., Fernandes-Pires, J., & Márquez-González, M. (2021). “We are staying at home.” Association of self-perceptions of aging, personal and family resources, and loneliness with psychological distress during the lock-down period of COVID-19. The Journals of Gerontology. Series B, Psychological Sciences and Social Sciences, 76(2), E10–E16. https://doi.org/10.1093/geronb/gbaa048
  • Losada-Baltar, A., Martínez-Huertas, J. Á., Jimenez-Gonzalo, L., Pedroso-Chaparro, M. D. S., Gallego-Alberto, L., Fernandes-Pires, J., & Márquez-González, M. (2022). Longitudinal correlates of loneliness and psychological distress during the lockdown situation due to COVID-19. Effects of age and self-perceptions of aging. The Journals of Gerontology. Series B, Psychological Sciences and Social Sciences, 77(4), 652–660. https://doi.org/10.1093/geronb/gbab012
  • Lyons, A., Alba, B., Heywood, W., Fileborn, B., Minichiello, V., Barrett, C., Hinchliff, S., Malta, S., & Dow, B. (2018). Experiences of ageism and the mental health of older adults. Aging & Mental Health, 22(11), 1456–1464. https://doi.org/10.1080/13607863.2017.1364347
  • Marques, S., Mariano, J., Mendonça, J., De Tavernier, W., Hess, M., Naegele, L., Peixeiro, F., & Martins, D. (2020). Determinants of ageism against older adults: A systematic review. International Journal of Environmental Research and Public Health, 17 (7), 2560. https://doi.org/10.3390/ijerph17072560
  • Maurya, P., Sharma, P., & Muhammad, T. (2022). Prevalence and correlates of perceived age-related discrimination among older adults in India. BMC Public Health, 22(1), 1–10. https://doi.org/10.1186/s12889-022-13002-5
  • Meisner, B. A. (2012). A meta-analysis of positive and negative age stereotype priming effects on behavior among older adults. The Journals of Gerontology. Series B, Psychological Sciences and Social Sciences, 67(1), 13–17. https://doi.org/10.1093/geronb/gbr062
  • Naughton, L., Padeiro, M., & Santana, P. (2021). The twin faces of ageism, glorification and abjection: A content analysis of age advocacy in the midst of the COVID-19 pandemic. Journal of Aging Studies, 57, 100938. https://doi.org/10.1016/j.jaging.2021.100938
  • Palmore, E. (2001). The ageism survey: First findings. Gerontologist, 41(5), 572–575. https://doi.org/10.1093/geront/41.5.572
  • Pikhartova, J., Bowling, A., & Victor, C. (2016). Is loneliness in later life a self-fulfilling prophecy? Aging & Mental Health, 20(5), 543–549. https://doi.org/10.1080/13607863.2015.1023767
  • Power, M., Quinn, K., Schmidt, S, & WHOQOL-OLD Group. (2005). Development of the WHOQOL-Old module. Quality of Life Research: An International Journal of Quality of Life Aspects of Treatment, Care and Rehabilitation, 14(10), 2197–2214. https://doi.org/10.1007/s11136-005-7380-9
  • Previtali, F., Allen, L. D., & Varlamova, M. (2020). Not only virus spread: The diffusion of ageism during the outbreak of COVID-19. Journal of Aging & Social Policy, 32(4–5), 506–514. https://doi.org/10.1080/08959420.2020.1772002
  • R Core Team. (2021). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.r-project.org/
  • Remuzzi, A., & Remuzzi, G. (2020). COVID-19 and Italy: What next? Lancet (London, England), 395(10231), 1225–1228. https://doi.org/10.1016/S0140-6736(20)30627-9
  • Ribeiro-Gonçalves, J. A., Costa, P. A., & Leal, I. (2023). Loneliness, ageism, and mental health: The buffering role of resilience in seniors. International Journal of Clinical and Health Psychology: IJCHP, 23(1), 100339. https://doi.org/10.1016/J.IJCHP.2022.100339
  • Rippon, I., Kneale, D., Oliveira, C., De Demakakos, P., & Steptoe, A. (2014). Perceived age discrimination in older adults. Age and Ageing, 43(3), 379–386. https://doi.org/10.1093/AGEING/AFT146
  • Rochon, P. A., Kalia, S., & Higgs, P. (2021). Gendered ageism: Addressing discrimination based on age and sex. Lancet (London, England), 398(10301), 648–649. https://doi.org/10.1016/S0140-6736(21)01636-6
  • Rosseel, Y. (2012). lavaan: An (R) package for structural equation modelling. Journal of Statistical Software, 48(2), 1–36. https://doi.org/10.18637/jss.v048.i02
  • Silva, M. F., da Silva, D. S. M., de Macedo Bacurau, A. G., Francisco, P. M. S. B., de Assumpção, D., Neri, A. L., & Borim, F. S. A. (2021). Ageism against older adults in the context of the COVID-19 pandemic: An integrative review. In Revista de Saude Publica, 55, 1–13. https://doi.org/10.11606/s1518-8787.2021055003082
  • Slotman, A., Cramm, J. M., & Nieboer, A. P. (2015). Validation of the Dutch aging perceptions questionnaire and development of a short version. Health and Quality of Life Outcomes, 13(1), 1–13. https://doi.org/10.1186/s12955-015-0248-y
  • Stokes, J. E., & Moorman, S. M. (2016). Who are the people in your neighborhood? Neighborhood age composition and age discrimination. Social Psychology Quarterly, 79(1), 68–80. https://doi.org/10.1177/0190272515626569
  • Swift, H. J., Abrams, D., Lamont, R. A., & Drury, L. (2017). The risks of ageism model: How ageism and negative attitudes toward age can be a barrier to active aging. Social Issues and Policy Review, 11(1), 195–231. https://doi.org/10.1111/sipr.12031
  • Swift, H. J., & Chasteen, A. L. (2021). Ageism in the time of COVID-19. Group Processes & Intergroup Relations: GPIR, 24(2), 246–252. https://doi.org/10.1177/1368430220983452
  • Tennant, R., Hiller, L., Fishwick, R., Platt, S., Joseph, S., Weich, S., Parkinson, J., Secker, J., & Stewart-Brown, S. (2007). The Warwick-Edinburgh mental well-being scale (WEMWBS): Development and UK validation. Health and Quality of Life Outcomes, 5(1), 63. https://doi.org/10.1186/1477-7525-5-63
  • Upton, D., & Upton, P. (2015). Quality of life and well-being. In D. Upton & P. Upton (Eds.), Psychology of wounds and wound care in clinical practice (pp. 85–111). Springer International Publishing. https://doi.org/10.1007/978-3-319-09653-7_4
  • Verhage, F. (1964). Intelligentie en leeftijd: Onderzoek bij Nederlanders van twaalf tot zevenenzeventig jaar. [Intelligence and age: Investigations on Dutch persons from twelve to seventy-seven years] [Koninklijke van Gorcum]. https://research.rug.nl/en/publications/intelligentie-en-leeftijd-bij-volwassenen-en-bejaarden
  • WHO. (2021a). Ageing: Ageism. https://www.who.int/news-room/questions-and-answers/item/ageing-ageism
  • WHO (2021b). Global report on ageism.
  • Wolff, J. K., Schüz, B., Ziegelmann, J. P., Warner, L. M., & Wurm, S. (2015). Short-term buffers, but long-term suffers? Differential effects of negative self-perceptions of aging following serious health events. The Journals of Gerontology Series B: Psychological Sciences and Social Sciences, 72(3), gbv058. https://doi.org/10.1093/geronb/gbv058
  • Wurm, S., & Benyamini, Y. (2014). Optimism buffers the detrimental effect of negative self-perceptions of ageing on physical and mental health. Psychology & Health, 29(7), 832–848. https://doi.org/10.1080/08870446.2014.891737