Development and Validation of a Knowledge Scale About the Behavioral and Psychological Symptoms of Dementia (KS-BPSD) Among Chinese Formal Caregivers

ABSTRACT

Objectives

The current study aimed to develop a scale assessing knowledge about behavioral and psychological symptoms of dementia (KS-BPSD) among Chinese formal caregivers and to investigate its psychometric properties and factorial structure.

Methods

The scale was generated with a systematic development process, and 229 formal caregivers working at nursing homes were recruited to construct and assess the psychometric properties of the scale. The preliminary scale was reviewed by an expert panel and items were selected based on item discrimination, difficulty, and item-total correlation.

Results

The final KS-BPSD version consisted of 12 items, loaded into three factors (i.e., Disease Characteristics, Care and Risks, and Treatment Needs) following principal component analysis (PCA). The KS-BPSD showed good test-retest reliability, internal consistency, as well as construct and concurrent validity.

Conclusions

The 12-item KS-BPSD was found to have high reliability and preliminary validity in assessing the level of knowledge about patient’s BPSD among formal Chinese caregivers in nursing homes.

Clinical Implications

KS-BPSD is a reliable tool to address the knowledge discrepancies and support needs among dementia caregivers, helping to develop and evaluate educational programs in the management of patient’s BPSD.

KEYWORDS:

• Dementia
• knowledge
• caregiver
• behavioral and psychological symptoms of dementia (BPSD)
• KS-BPSD

Introduction

According to the latest report of global prevalence (Guerchet, Prince, & Prina, 2020), there are over 50 million people suffering from dementia in 2020, with this number predicted to double in the next 20 years. China, with the largest population of people living with dementia (accounting for about 25% of the entire dementia population worldwide; Prince et al.), is inevitably facing a heavy burden in dementia healthcare.

Although behavioral and psychological symptoms of dementia (BPSD) are present in up to 90% of patients with dementia (Gauthier et al., 2010) and the frequency of those symptoms increase with the progression of dementia (Hessler et al., 2018), BPSD has received less attention compared with cognitive symptoms in dementia care research. BPSD refers to the symptoms of disturbed perception, thought content, mood, or behavior that frequently occur in patients with dementia, such as agitation, aggression, sleep disturbance, depression, anxiety, and apathy, etc. (Finkel, E Silva, Cohen, Miller, & Sartorius, 1997). BPSD could be a result of complex interactions between factors at the individual level (e.g., dementia severity) and contextual level (e.g., relationship with the caregivers (Gauthier et al., 2010; Sampson et al., 2014). Importantly, BPSD is associated with the breakdown of home care, premature institutionalization, increased hospitalization, greater healthcare costs, and significant loss of quality of life for patients and their families (Cerejeira & Lagarto, 2012; Finkel et al., 1997). It is also a major source for and contributor to burden and distress of formal caregivers (Miyamoto, Tachimori, & Ito, 2010). Despite the significant impact of BPSD, awareness and knowledge about BPSD may have been relatively overlooked, probably due to a lack of validated instruments assessing the knowledge about BPSD.

The World Health Organization has called for greater dementia awareness and education in response to increasing global prevalence of this syndrome (World Health Organization, 2006). As a result, a variety of measures have been developed to assess dementia knowledge, such as the Dementia Knowledge Assessment Scale (DKAS; Annear et al., 2015), the Alzheimer’s Disease Knowledge Scale (ADKS; Carpenter, Balsis, Otilingam, Hanson, & Gatz, 2009), the Knowledge of Memory Loss and Care Test (Kuhn, King, & Fulton, 2005), and the Dementia Knowledge Assessment Tool (DKAT-2; Toye et al., 2014). Abovementioned measures have been validated (Annear et al., 2015; Carpenter et al., 2009; Spector, Orrell, Schepers, & Shanahan, 2012) and adopted in various education programs for dementia (Annear et al., 2016; Eccleston et al., 2019). However, in the existing literature, most knowledge assessments focus on cognitive symptoms of dementia, especially memory loss, while non-cognitive symptoms are relatively neglected (Schindler, Engel, & Rupprecht, 2012).

Understanding BPSD is essential for appropriate management of BPSD, affecting caregiver responses to BPSD. For instance, caregivers who are well-informed about BPSD symptoms can better cope with the distress resulting from BPSD (Bessey & Walaszek, 2019; Devor & Renvall, 2008; Trivedi et al., 2019). For professional caregivers, understanding patient’s BPSD could be particularly important yet challenging. For one reason, as dementia progresses, patient’s BPSD could become more common. The Cache County study has reported that the prevalence of BPSD has significantly increased across a five-year period (Steinberg et al., 2008). In addition, patients at nursing homes usually suffer with moderate to severe dementia and with more BPSD across a larger variety population(Selbæk, Kirkevold, & Engedal, 2007; Zuidema, Derksen, Verhey, & Koopmans, 2007). Therefore, it is necessary to address the level of formal caregiver’s knowledge about BPSD to better understand how caregiver’s caregiving experiences are affected and contribute to their needs in caregiving training.

In summary, BPSD is very common among patients with dementia (Brodaty et al., 2001; Selbæk et al., 2007; Steinberg et al., 2008; Zuidema et al., 2007)and have significant impacts on caregiving burden for formal caregivers. Caregiver’s knowledge of BPSD may affect the caregiving experience and efficacy. However, little attention has been given to the level and role of BPSD knowledge in caregiving, probably because no specific assessment of BPSD knowledge has been developed. To fill this research gap, the present study develops a knowledge scale assessing the understanding of BPSD (KS-BPSD) among formal caregivers. By collecting data from 229 formal caregivers at nursing homes, the present study also evaluates the psychometric properties of the KS-BPSD.

Methods

Participants

Formal caregivers were recruited from four nursing homes in Guangzhou and Nanchang, China. The recruitment criteria included: 1) over 18 years old; 2) having provided care for patients with dementia for more than one month; and 3) no difficulties in communicating, understanding, and administering the questionnaire. The study was approved by the Ethics Committee of Jinan University and all participants provided written consent before taking part in the study.

Scale development

Six experts, including: three clinicians, two researchers in the field of psychology and rehabilitation, and one researcher in the field of aging and psychology, were invited to form an expert panel for scale development. All experts had rich experience in scale development, validation, and culture adaption. Using previous scales assessing knowledge of dementia (or Alzheimer’s disease; Sullivan & Mullan, 2017), guidelines for dementia care, and opinions and knowledge of the expert panel as reference, five graduate students, each with a nursing or gerontology background, generated a set of 15 preliminary items each. The items covered knowledge of causes, characteristics, care, treatment, risk, and health promotion etc. of BPSD. All items were reviewed and discussed by three experts in the panel and redundant items were removed. Items with ambiguity, confusion, or using jargon were revised to obtain clarity. The goal of this process was to enhance content validity of the scale (Trochim & Donnelly, 2007) and to ensure the coverage and relevance of the content (Streiner, Norman, & Cairney, 2015). After the graduate students generated items and experts reviewed, the preliminary version included 27 items. All discrepancies were resolved. Adopting the response style from existing instruments (Annear et al., 2017; Annear et al., 2015), a 5-point Likert-type scale was used (1 = false, 2 = probably false, 3 = probably true, 4 = true, and 5 = I do not know).

Three other experts in the panel were invited to participate in a three-round Delphi study to evaluate the 27 items. In the first round, they were asked to evaluate the items based on: 1) how important the item is for understanding BPSD using a 6-point scale (1 = not important at all to 5 = very important); 2) whether the wording was appropriate; and 3) whether the items covered important knowledge about BPSD. Items were considered as having high consensus if they were scored with a 5 on importance at least once, and a median score greater than or equal to 4 among expert ratings of importance. Items with high ratings and levels of consensus were retained. A total of 20 items were entered into the second round of review, in which the experts re-rated the importance of the remaining items. Again, items with high consensuses and high importance were selected for the third round of review (Annear et al., 2015). The median for each item in the previous round was provided to the experts to increase their consensus in the second and third round. With all discrepancies resolved, 14 items were included in the scale. A graduate student translated the Chinese version of the scale into English with the help of two bilingual speakers. For details, see Table 1.

Table 1. Expert’s ratings on the 14 items.

Item	Importance score
	Expert 1	Expert 2	Expert 3
(1) Behavioral and Psychological Symptoms of Dementia (BPSD) occur frequently on patients with dementia. [True]	4	5	5
(2) Psychological symptoms (e.g., Apathy, depression, anxiety) are common BPSD among Chinese patients. [True]	2	5	5
(3) BPSD have different clinical manifestations in different stages of dementia. [True]	4	4	5
(4) BPSD could be a result of a myriad of biopsychosocial factors. [True]	4	4	5
(5) BPSD cannot be cured. [False]	4	4	5
(6) Most BPSD are related with emotions. [True]	4	4	5
(7) Early intervention does not help caregivers manage patients’ BPSD. [False]	4	4	5
(8) Poor interactions between patients with dementia and the caregiver can exacerbate BPSD. [True]	4	4	5
(9) Medication should be the first choice for treating BPSD. [False]	3	5	5
(10) BPSD are the major source for caregiving burden. [True]	4	5	5
(11) Supporting programs for caregivers can help them manage BPSD more effectively. [True]	4	4	5
(12) The daily care of patient with BPSD should only focus on meeting their practical needs instead of psychological needs. [False]	4	4	5
(13) Compared with other symptoms of dementia, treatment on BPSD can reduce caregiving burden more effectively. [True]	4	4	5
(14) BPSD have a significant impact on patients’ physical and mental health. [True]	3	5	5

BPSD: behavioral and psychological symptoms of dementia.

Measurements

Participants’ demographic information was collected, including gender, age, marital status, education, and care experience. Care experience was measured using the questions: “How long you have been providing care for people with dementia?” and “Have you participated in any education programs related to dementia?.”

The 25-item Dementia Knowledge Assessment Scale (DKAS, version 2.0) was used to assess knowledge related to dementia (Annear et al., 2015). Four domains were included, namely: Causes and Characteristics (7 items), Communication and Behavior (6 items), Care Considerations (6 items), and Risk and Health Promotion (6 items). By summing the number of correct responses, sum scores of each sub-scale and the overall scale were generated. The overall score for DKAS 2.0 ranged from 0–50. This version had acceptable internal consistency and Cronbach’s alpha of the overall scale was 0.85 (Annear et al., 2017). Significant differences between different cohorts supported discriminative validity (Annear et al., 2017). At the same time, the Chinese version of DKAS 2.0 has also showed acceptable reliability and validity in healthcare providers (Zhao et al., 2020).

Data analytic methods

Correct answers (e.g., “true” or “probably true” to a true statement) on the KS-BPSD were scored 2 or 1 respectively. Incorrect answers (i.e., “false,” “probably false,” or “I don’t know” to a true statement) were scored “0.” After three rounds of review, the initial scale consisted of 14 items with a total score of 0–28. The higher the score, the higher the knowledge level of participants about BPSD.

Data analysis was conducted in three steps with IBM SPSS Statistics V22.0. Firstly, individual items were analyzed by calculating the discrimination indexes, item difficulty indexes, and item-total correlations. Secondly, an exploratory factor analysis (EFA) using principal component analysis (PCA) with varimax rotation (Chan & Idris, 2017; Costello & Osborne, 2005) was performed to ascertain the underlying structure of the scale. According to the recommendations of Trochim and Donnelly (2007), once the final scale was obtained, the psychometric properties were evaluated including: test-retest reliability, internal consistency, concurrent validity, discrimination between groups, and convergent and discriminant validity. For test-retest reliability, intra-class correlation coefficients (ICCs) were calculated. Cronbach’s alpha was used as a measure of internal consistency. Concurrent validity is reflected by the correlation between DKAS 2.0 and KS-BPSD. A t-test was used to compare whether there is a significant, statistical difference in KS-BPSD scores with or without a history of attending an education workshop on dementia. Convergent validity was supported by a satisfactory item-subscale correlation (r ≥0.40). Discriminant validity was indicated by item-subscale correlations higher than correlations between the items and other subscales.

Results

In total, 229 valid responses were collected (22.3% male). About half of the participants were aged older than 45 years (48%), and a vast majority of participants were married (81.7%). Almost half of the participants have completed college (47.2%; for details, see Table 2). Regarding caregiving experience, over half of the participants have been a caregiver for over 3 years (54.6%). In addition, the vast majority of participants have taken part in dementia-related education programs (80.8%).

Table 2. Demographic information (N = 229).

Variables	n (%)
Sex
Males	51(22.3)
Females	178(77.7)
Age
18–31	43(18.8)
32–44	76(33.2)
45–57	86(37.6)
58–70	24(10.5)
Marital status
Married	187(81.7)
Unmarried	30(13.1)
Divorced	6(2.6)
Widowed	6(2.6)
Education
Junior high school and below	72(31.4)
High school/ Technical secondary school	49(21.4)
University/ Junior college	105(45.9)
Postgraduate and above	3(1.3)
How long you have been caring for people with dementia.
1–6 months	24(10.5)
6–12 months	33(14.4)
1–3 years	47(20.5)
3–10 years	59(25.8)
Over 10 years	66(28.8)
Whether you have participated in education programs related to dementia.
Yes	185(80.8)
No	44(19.2)

The number of people who indicated I don’t know (5) ranged from 1 (0.4%) to 28 (12.2%) for each item on the KS-BPSD. The number of people who indicated I don’t know (5) ranged from 3 (1.7%) to 50 (21.8%) for each item on the DKAS 2.0.

Item analysis

In the first step to screen items, the discrimination index, difficulty index, and item-total correlation of each item was analyzed. The purpose of discrimination index analysis was to eliminate items that were ineffective in discriminating participants with high and low overall scores. Using the overall score of the 14 items, a group with high scores (top 27%, n = 62) and a group with low scores (bottom 27%, n = 62) were identified. For each item, the percentage of participants from the above two groups who answered the item correctly, was calculated. The difference between these two percentages indicated the discrimination index (D), how sensitive the item is in differentiating formal caregiver’s level of BPSD knowledge in their patients, for each item (D scores between 0 and 0.19 indicated poor discrimination; D scores between 0.2 and 0.29 indicated acceptable discrimination; D scores between 0.3 and 0.39 indicated good discrimination; D >0.4 indicated excellent discrimination; Pande, Pande, Parate, Nikam, & Agrekar, 2013; Rao, Prasad, Sajitha, Permi, & Shetty, 2016; Sharma, 2021). The D scores of the 14 items ranged from 0.19 to 0.74, indicating that the discrimination capability was acceptable.

A difficulty index was also generated for each item, representing the percentage of people who answered the item correctly. If the rate of correctness is too high or low, it may be suggested that the item’s difficulty level is not appropriate. For example, a difficulty index of 0.95 indicates that 95% participants answered correctly, thus providing little useful information and detracts from the validity of the scale (Streiner et al., 2015). Therefore, items were retained only if they had a difficulty index lower than 0.95 and higher than 0.05.

The item-total correlation, indicated by the correlation between each item and the total score (omitting that item), was calculated for each item. It is recommended that this correlation be above 0.20 for item retention (Tabachnick & Fidell, 2013) and item 9 was removed as a result (for details, see Table 3).

Table 3. The properties of individual item.

Item	Discrimination Index (D)	Difficulty index	Item-total correlation
Item 1	0.66	0.63	0.47
Item 2	0.59	0.68	0.45
Item 3	0.57	0.72	0.55
Item 4	0.74	0.57	0.52
Item 5	0.57	0.39	0.28
Item 6	0.44	0.35	0.30
Item 7	0.72	0.58	0.49
Item 8	0.62	0.53	0.45
Item 9	0.19	0.20	0.06
Item 10	0.54	0.47	0.29
Item 11	0.64	0.71	0.48
Item 12	0.67	0.66	0.42
Item 13	0.74	0.56	0.52
Item 14	0.68	0.61	0.56

Principal component analysis

The EFA was conducted using PCA with varimax rotation. Preliminary analysis confirmed that the data was suitable for factor analysis (KMO = 0.79, Bartlett test of Sphericity P < .001). The cutoff for each factor loading was 0.4 (Peterson, 2000). Since item 4 had strong cross loadings (cross-loadings >75%) and the lowest absolute maximum loading of all the factors, item 4 was removed and the PCA was re-run (Samuels, 2017). Finally, PCA with the 12-item scale revealed a three-component structure (eigenvalues >1.0) explaining 56.9% of the cumulative variance. Varimax rotation showed that three to six variables were loaded significantly on each component. The items in each component were inspected and interpreted, resulting in three labels: Disease Characteristics (3 items), Care and Risks (6 items), and Treatment Needs (3 items). The loadings and interpretations indicated that the factorial validity of the 12-item scale was acceptable (for details, see Table 4). The final scale consisted of 12 items with a total score of 0–24. The mean (SD) of the total scale scores was 16.15 (5.29). The means (SD) of the subscale scores were 4.45 (2.00), 7.97 (3.08), and 3.73 (1.96) respectively in Disease Characteristics, Care and Risks, and Treatment Needs. The correlation coefficients of the three subscales were between 0.39 and 0.28 (p <.01).

Table 4. EFA item loadings.

Item	Factor 1 Care and Risks	Factor 2 Disease Characteristics	Factor 3 Treatment Needs
Item 1: BPSD occur frequently on patients with dementia	0.08	0.84	0.13
Item 2: Psychological symptoms are common BPSD among Chinese patients.	0.15	0.84	−0.01
Item 3: BPSD have different clinical manifestations in different stages of dementia.	0.29	0.74	0.09
Item 5: BPSD cannot be cured.	0.02	−0.06	0.78
Item 6: Most BPSD are related with emotions.	0.70	0.07	−0.13
Item 7: Early intervention does not help caregivers manage patients’ BPSD.	0.05	0.45	0.62
Item 8: Poor interactions between patients with dementia and the caregiver can exacerbate BPSD.	0.61	0.17	0.11
Item 10: BPSD are the major source for caregiving burden.	0.60	0.19	−0.19
Item 11: Supporting programs for caregivers can help them manage BPSD more effectively.	0.60	0.13	0.30
Item 12: The daily care of patient with BPSD should only focus on meeting their practical needs instead of psychological needs.	0.16	0.10	0.77
Item 13: Compared with other symptoms of dementia, treatment on BPSD can reduce caregiving burden more effectively.	0.71	0.01	0.32
Item 14: BPSD have a significant impact on patients’ physical and mental health.	0.58	0.19	0.36

Factor 1 (Care and Risks) includes: item 6, item 8, item 10, item 11, item 13, and item 14.

Factor 2 (Disease Characteristics) includes: item 1, item 2, and item 3.

Factor 3 (Treatment Needs) includes: item 5, item 7, and item 12.

Reliability and validity

The test-retest reliability of the 12-item scale was obtained with an interval of one week. The minimum retest sample size of 30 was acceptable when ICC was used to measure the test-retest reliability (Bujang & Baharum, 2017). The “Randbetween” function in Excel was used to randomly select an institution out of the four nursing homes, from which 34 caregivers were randomly invited to take part in the retest. The test-retest reliability coefficient was 0.86 (p < .001), suggesting good test-retest reliability. Cronbach’ alpha of the 12-item scale was 0.79, indicating the internal consistency was acceptable. The Cronbach’s alphas of the subscales were 0.79, 0.74, and 0.65 respectively in Disease Characteristics, Care and Risks, and Treatment Needs.

As a recent tool for measuring dementia knowledge, DKAS 2.0 was found to have a high correlation with well-established knowledge scales. Good reliability and validity were reported of DKAS 2.0 and it also includes three items on patient’s psycho-behavioral symptoms. Therefore, DKAS 2.0 was considered appropriate for conducting concurrent validity. In our sample, the overall score of the 12-item KS-BPSD was highly correlated with the total score of DKAS 2.0 (r= 0.56, p < .001). As expected, participants who have taken a education workshop on dementia showed higher scores (16.62 ± 4.91) than those without attending such workshops (14.18 ± 6.34, t = 2.38, p < .05). Scores of each item was significantly correlated with its corresponding subscale (r > 0.4, P < .01). Moreover, the correlation between the item and the subscale was higher than that between the item and other subscales. The results showed that the 12-item KS-BPSD has good convergent and discriminant validity.

Discussion

The present study has reported the development and psychometric properties of a 12-item scale assessing formal caregiver’s knowledge about patient’s BPSD (KS-BPSD). To develop the scale, a three-round Delphi method was adopted, and 15 items were removed according to the exclusion criteria at each stage. The PCA results showed that the scale fit a three-factor solution: Disease Characteristics, Care and Risks, and Treatment Needs. In specific, Disease Characteristics referred to the understanding about key information of BPSD clinical manifestation, progress, and correlates; Care and Risks focused on the caregiving factors that influence the development of BPSD; and Treatment Needs captured the diagnosis, treatment, and prognosis aspects of BPSD. The three factors covered the fundamental information about BPSD and showed a good factorial validity.

The reliability and validity of the 12-item KS-BPSD were good, with decent content, concurrent, and construct validity. In addition, by comparing the score of those who have participated in the dementia-related education programs with those who haven’t, it was suggested that the dementia-related education programs could increase formal caregiver’s knowledge about their patient’s BPSD as the former group’s score was significantly higher than the latter.

The current study has developed a scale assessing formal caregiver’s knowledge of patient’s BPSD, which offers a critical complement for the existing measures on BPSD. Despite the correlation and overlap with existing dementia knowledge scales, such as the DKAS 2.0, the focuses are different. Actually, DKAS only includes two items that are related with depressive symptoms: “People with dementia are unlikely to experience depression” and “symptoms of depression can be mistaken for symptoms of dementia,” despite that depressive symptoms are an important cluster of BPSD. Moreover, clusters of behavioral and psychological symptoms other than depression could also have significant impact on caregiving burden. In addition, KS-BPSD captures an important yet often neglected component in dementia care. Previous studies showed that nurse’s knowledge plays an important role in their competence of providing care, care outcomes (Josefsson, Sonde, & Wahlin, 2008), as well as caregiver’s mental health and well-being (Evripidou, Charalambous, Middleton, & Papastavrou, 2019; Marx et al., 2014). Therefore, a knowledge scale covering more comprehensive aspects of BPSD is necessary for understanding the relationship between caregiver’s knowledge, attitude, behavior and care outcomes.

The development of this scale has important implications for designing dementia care training and education for formal caregivers. Knowledge assessments of nursing staff can help us understand their knowledge gaps and training needs, and develop tailored training programs to improve the care quality, thus benefiting both patients and caregivers. It is noteworthy that although our sample was made up of formal caregivers from nursing homes, the level of BPSD knowledge was not close to the full possible score (the average score was 16.15, SD = 5.29), suggesting no ceiling effect was found. This is probably because the educational level of our sample was relatively low and the trainings programs on BPSD offered to caregivers has remained limited. On the other hand, it is assumed that when applied to family caregivers, lay public, or the patients, KS-BPSD could also serve as an effective tool in assessing their knowledge of BPSD. Moreover, it is possible that different populations may show discrepancies across the three dimensions of KS-BPSD (i.e., Disease Characteristics, Care and Risks, and Treatment Needs) which can help diagnose the specific education needs for each group. It may also draw attention to BPSD and prompt early preparations for better managing the symptoms along with the deterioration of dementia.

Limitations and future directions

A major limitation of the current study is that the sample used to develop this scale was only formal caregivers working at nursing homes. Therefore, their level of knowledge about BPSD could be potentially higher than other populations, restraining the representativeness of the sample and the generalizability of the scale. Further work is warranted to test the scale’s generalizability with family caregivers, the public, or dementia patients to evaluate the psychometrics and norms across different populations. In addition, with more responses, confirmatory factor analysis (CFA) should be conducted to evaluate the validity and scale structure. At last, future studies should also look at the role of knowledge about BPSD in caregiver burden and care outcomes, to provide more evidence for the scale’s capacity of assessing BPSD knowledge, as well as to better understand its clinical relevance.

Clinical implications

• KS-BPSD is a reliable measurement to assess formal caregiver’s knowledge about patient’s BPSD and three factors were included: Disease Characteristics, Care and Risks, and Treatment Needs.

• Adopting KS-BPSD could address caregivers’ knowledge gaps about BPSD and develop tailored training programs to help them provide better care.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

The data is available upon reasonable request addressing to the correspondence author (http://dx.doi.org/10.17632/f38m748pz9.1).

Additional information

Funding

Financial support: This study was supported by the National Key R&D Program of China under Grant [2020YFC2005802; 2021ZD0203103]; and the Science and Technology Program of Guangzhou under Grant [202007030012].