1,533
Views
0
CrossRef citations to date
0
Altmetric
Research Article

Development and preliminary validation of a self-rating anxiety inventory for maintenance haemodialysis patients

ORCID Icon, , , , &
Pages 1482-1494 | Received 10 Jun 2020, Accepted 02 Feb 2021, Published online: 18 Feb 2021

ABSTRACT

This study aimed to develop a self-rating anxiety inventory for maintenance haemodialysis patients (AI-MHD) and perform preliminary validation to provide a simple, effective, and highly specific practical tool for effective anxiety disorder screening in haemodialysis patients. Based on existing general anxiety disorder screening scales and common symptoms of MHD patients as a reference and after expert discussions and preliminary validation at a single dialysis centre, a self-rating AI-MHD containing 12 items was developed. Subsequently, the AI-MHD was applied in 4 dialysis centres and compared with GAD-7 and HADS-A. Further multicentre validation showed that Cronbach's alpha for the scale was 0.918; the AI-MHD score not only significantly differed between the anxiety disorders group and the non-anxiety disorders group (p<0.001) but also correlated with GAD-7 and HADS-A scores (p<0.001). In addition, the Kaiser-Meyer-Olkin (KMO) score was 0.847, and Bartlett’s test of sphericity was significant (x2=849.45, p<0.001). The anxiety disorder detection rate was 93%, and the specificity was 90%, which were significantly better than the screening results using the GAD-7 and HADS-A scales in the same groups. Although there were limitations, such as the sample size and regionality, the AI-MHD showed good efficacy and reliability in rating anxiety in MHD patients.

Introduction

Anxiety disorders are diseases with a high prevalence among mental disorders; some studies have shown that the prevalence rate is approximately 2.1–13.1% in the general population depending on the definition (Kosic et al., Citation2020; Peedicayil, Citation2020). The incidence of anxiety disorders is higher in some people with underlying diseases; in maintenance haemodialysis (MHD) patients, the prevalence of anxiety disorders is 20% −55% (Brito et al., Citation2019; Gerogianni et al., Citation2018; Kopple et al., Citation2017). More importantly, existing studies have found that anxiety disorders are significantly related to decreases in MHD patients’ quality of life and sleep as well as increases in adherence to medical management and mortality (Alosaimi et al., Citation2016; Isik Ulusoy & Kal, Citation2020; Li et al., Citation2016). Therefore, convenient and effective screening for anxiety disorders in MHD can not only lay the foundation for timely targeted nursing interventions and clinical treatments but also play an important role in improving dialysis quality and the quality of life of dialysis patients.

Reliable evaluations and measurements are necessary and critical processes for the rapid diagnosis and detection of anxiety disorders in MHD patients. The current gold standard for the detection of anxiety disorders is based on the Diagnostic and Statistical Manual of Mental Disorders (5th edition) (DSM-5); diagnosis is conducted through clinical interviews between patients and specialists in psychiatry or psychology (Roehr, Citation2013). However, such interviews must be implemented by trained health personnel; these interviews can be time consuming and difficult to apply in routine clinical practice, and early detection of anxiety disorders in MHD patients is therefore challenging. Currently, commonly used anxiety scales include the Hospital Anxiety and Depression Scale (HADS), Generalized Anxiety Disorder-7 (GAD-7), et al (Rose & Devine, Citation2014; Smith et al., Citation2019). These scales are relatively simple and fast and do not require psychiatry and psychology professionals. However, these scales have the following limitations: they are not disease-specific scales and are commonly used for general populations; some scales have overlapping anxiety and depression items (Ziebold et al., Citation2019); and, there are different criteria, a lack of consensus on items, a lack of race specificity, and regional differences (Rose & Devine, Citation2014). These factors may be responsible for the large incidence range for anxiety in clinical practice. More importantly, there are currently some physical component items in commonly used anxiety self-rating scales, and these physical components may have similarities to related disease manifestations. For example, fatigue, loss of appetite, and sleep disorders may overlap with the physical symptoms and side effects of dialysis and drug treatments, thus complicating the identification of symptoms in dialysis patients (Burton et al., Citation2019; Pifer & Segal, Citation2020; Ziebold et al., Citation2019). Therefore, currently, screening MHD patients for anxiety disorders is challenging, and a specific, more concise and effective anxiety self-rating scale for haemodialysis patients is urgently needed.

With research on anxiety disorders, increasingly more specific anxiety scales have emerged, especially disease-specific scales (discipline specific), such as the dental patient anxiety scale, burn-specific anxiety scale, cardiac anxiety scale, et al (Rose & Devine, Citation2014). The recently developed long-term care geriatric anxiety scale (Pifer & Segal, Citation2020), self-rating surgical anxiety scale (Burton et al., Citation2019), and Magnetic Resonance Imaging-Anxiety Questionnaire (Ahlander et al., Citation2016) and the previously developed anxiety inventory for respiratory disease (Willgoss et al., Citation2013) do not include related items caused by specific diseases, but more disease-specific items are explored to achieve better reliability and validity than those of non-specific self-rating scales such as the HADS and Beck Anxiety Inventory. These scales also explain from another perspective that in this new era of personalized disease treatment, a more specific system is needed for more comprehensive diagnosis to effectively promote patients’ recovery and improve their quality of life.

Therefore, based on our previously developed disease-specific depression scale for MHD patients (Wang et al., Citation2019), this study developed a disease-specific self-rating anxiety scale for screening MHD patients (discipline specific) using a similar method. The developed scale was then preliminarily validated.

Materials and methods

The method was similar to that used for our previously developed depression scale for haemodialysis patients (Wang et al., Citation2019). This study was approved by the Ethics Committee of the Third Affiliated Hospital of Chongqing Medical University. All participants provided written informed consent.

Preliminary development of the I-AI-MHD

First, the 20-item I-AI-MHD for MHD patients was compiled based on a literature review and the existing items of commonly used anxiety scales, the clinical manifestations of MHD, and characteristics of Chinese culture. Subsequently, the opinions of psychiatrists (2 experts) and experts on psychology (2 experts) and haemodialysis (4 experts) were consulted. Combined with their clinical diagnosis and treatment experience and research experience regarding the core symptoms of anxiety in haemodialysis patients, the developed self-rating I-AI-MHD was discussed and modified to the 15-item I-AI-MHD. Each item was scored as follows: 0 points for ‘completely absent’, 1 point for ‘occasional’, 2 points for ‘frequent’, and 3 points for ‘almost all the time’.

Improvements to the I-AI-MHD

From January to June 2019, a total of 110 MHD patients in our hospital’s haemodialysis centre were selected to complete the questionnaire survey. The inclusion criteria were as follows: (1) older than 18 years; (2) maintenance history of haemodialysis >3 months; and (3) no cognitive impairment, with a certain education level, able to understand the scale content, and willing to cooperate voluntarily. The exclusion criteria were as follows: (1) previous mental illness; (2) other than renal diseases, no other chronic diseases (e.g., chronic heart failure, cancer, and hyperthyroidism), no craniocerebral injury, and no other major injury to the body and organs; and (3) no recent significant emotional distress (such as death of a spouse, relationship issues, unemployment, etc.).

Trained investigators used the I-AI-MHD and standardized questionnaires to conduct on-site questionnaire surveys. The general information, basic treatment and treatment status of the patients were collected. Patients signed an informed consent form.

Diagnosis of anxiety disorders in MHD patients: One week after the questionnaire survey, based on the DSM-5, anxiety disorders were diagnosed through information provided in clinical interviews involving patients and psychiatrists or psychologists (professionally qualified physicians).

Statistical analysis of the I-AI-MHD

Data analysis was conducted using the Statistical Package for the Social Sciences for Windows (IBM, SPSS, 21). To determine the disease differences in the enrolled patients, demographic variables and disease-related variables were compared using the chi-square test or t-test. The Mann-Whitney U test was used to compare the scores for I-AI-MHD items, and the item scores for the anxiety and non-anxiety groups were compared. If P > 0.05, the difference was not significant, and the item was deleted. Subsequently, Kaiser-Meyer-Olkin (KMO) values were determined, and Bartlett’s test of sphericity was performed. Furthermore, exploratory factor analysis (EFA) of a single-factor model was used to evaluate the internal consistency and calculate the total item correlation and factor loading. Based on past studies (Wang et al., Citation2019; Willgoss et al., Citation2013), if the unloaded factor loading of the item was <0.5, then the item was deleted.

Preliminary validation of the AI-MHD

From July to December 2019, 4 haemodialysis centres were selected to validate the instrument: (1) The Third Affiliated Hospital of Chongqing Medical University; (2) The Second Affiliated Hospital of Chongqing Medical University; and (3) The Third Affiliated Hospital of Army Medical University; and (4) The Third People’s Hospital of Chongqing. A total of 346 MHD patients at the 4 centres completed all questionnaires (AI-MHD, GAD-7, and HADS-A), and general information about the patients was collected. Patients were diagnosed for anxiety disorders by specialists within 1 week. The inclusion criteria and procedures for the diagnosis of anxiety disorders in patients with MHD were the same as those described in the Improvements to the I-AI-MHD section. After a 2-week interval, 321 patients completed all questionnaires again, 25 of whom were excluded for the following reasons: refusal to participate without specific reasons (n = 15), transfer to another centre for treatment (n = 6), other causes (n = 3), and death (n = 1).

We used the GAD-7 and HADS-A (Esser et al., Citation2018; Rose & Devine, Citation2014), which are widely applied and have fewer items and good reliability and validity, as the control scales for this study. Both the GAD-7 and the HADS-A contain 7 items and yield scores ranging from 0 to 21, with higher scores representing greater anxiety symptoms (Esser et al., Citation2018).

Statistical methods for the AI-MHD

The data were processed as described in the statistical analysis section of the I-AI-MHD, and the Statistical Package for the Social Sciences (IBM, SPSS, 21) was used for data analysis. Similar to a previous study (Wang et al., Citation2019), the internal reliability of the self-rating anxiety scale was evaluated using Cronbach’s α and the Guttman correlation coefficient. To determine discriminant validity, the AI-MHD scores for patients with or without anxiety disorders were compared; convergent validity was determined. Correlations among the AI-MHD and GAD-7 and HADS-A were determined by McNemar’s test; structural validity was analysed by confirmatory factor analysis (CFA) and a factor analysis matrix; receiver operating characteristic (ROC) curves were generated with clinical diagnosis as the standard. The sensitivities, specificities, positive predictive values (PPVs), and negative predictive values (NPVs) were calculated.

Results

Demographic characteristics of patients who participated in the I-AI-MHD improvement studies

A total of 110 MHD patients participated in the first phase of the study, 13 of whom were excluded for the following reasons: failed eligibility screening (n = 3), withdrew during the interview (n = 3), refused to sign the consent form (n = 1), did not obtain a consistent clinical diagnosis (n = 2), and had an incomplete I-DI-MHD questionnaire (n = 4). Ninety-seven patients completed the test and were included in the depression group (n = 33) or the non-depression group (n = 64). No significant differences in age, gender, body mass index (BMI), dialysis duration, employment status, income, education level, and activities of daily living (ADL) were noted (, p > 0.05).

Table 1. Sociodemographic and clinical characteristics of the participants.

Development and improvement of the I-AI-MHD

The original self-rating scale with 20 items was discussed by experts, and 5 items were discarded according to suggestions made by the experts. Subsequently, a comparative analysis revealed no significant differences in 3 items (), and the 3 items were excluded. Finally, a self-rating scale with 12 items was generated, and the KMO value was 0.865. Bartlett’s test of sphericity was significant (x2 = 852.6, p < 0.001). The EFA results for the univariate model showed that one-way variance accounted for 67.14% of the total variance. All of the items loaded strongly onto a single factor, with a mean loading factor of 0.64 ± 0.05, and the item-total correlation was 0.60 ± 0.04 ().

Table 2. Comparison of I-AI-MHD scores for patients with anxiety and with non-anxiety disorders.

Table 3. Analysis of the final 12-item AI-MHD.

Demographic characteristics of patients at the AI-MHD preliminary validation stage

A total of 321 MHD patients participated in this study. After excluding patients with considerable differences in answers between the 2 surveys, 310 patients were included the anxiety disorders group (n = 92) and non-anxiety disorders group (n = 218). No significant differences in age, BMI, haemodialysis time, employment status, income, social support, education level, or ADLs were noted (; p > 0.05).

Reliability of the AI-MHD

The Cronbach’s α value for the scale was 0.918.

Validation of the AI-MHD

Regarding discriminant validity, similar to the GAD-7 and HADS-A, there was a significant difference in the AI-MHD scores between the anxiety disorders group and the non-anxiety disorders group (p < 0.001, ). Regarding convergent validity, the AI-MHD scores were significantly correlated with the GAD-7 and HADS-A scores (p < 0.001, ). Regarding structural validity, the KMO value was 0.847, and Bartlett’s test of sphericity was statistically significant (x2 = 849.45, p < 0.001), indicating that the data were suitable for factor analysis. According to the variance contribution rate analysis (), 3 factors were extracted: disease-related, affective, and physical factors. The average factor load of the three-factor model was 0.73 ± 0.09 (). Compared with the single-factor model with an average factor load of 0.60 ± 0.04 (), the model’s degree of fit significantly improved (p = 0.018). shows that there was a positive correlation among the 3 factors (p ≤ 0.01).

Table 4. Comparison of GAD-7, HADS-A, and AI-MHD mean values for patients with anxiety disorders and non-anxiety disorders.

Table 5. Correlation analysis of AI-MHD scores and GAD-7 and HADS-A scores.

Table 6. Exploratory factor analysis of the promax rotation of the AI-MHD scale.

Table 7. Correlation analysis of AI-MHD subscale scores.

Clinical cut-off, sensitivity and specificity of the AI-MHD

shows that the area under the ROC curve (AUC) of the AI-MHD was 0.93 (95% CI, 0.86–0.99), which was higher than the AUCs for the GAD-7 and HADS-A. In addition, for the cut-off score (23.0), the sensitivity was 93%, the specificity was 90%, the PPV was 93%, the NPV was 95%, and the Youden index was 0.89, which were slightly better than those for the GAD-7 and HADS-A (). Moreover, using their respective optimal cut-off scores, the incidence rates of anxiety disorders in MHD patients calculated using the AI-MHD, GAD-7, and HADS-A were 40.8%, 35.3%, and 36.4%, respectively, whereas the incidence rate of anxiety disorders determined using the DSM-5 and interviews with specialized physicians was 43.8%.

Table 8. Cut-off values for the AI-MHD, GAD-7, and HADS-A.

Figure 1. ROC curve for the AI-MHD, GAD-7, and HADS-A. Note: ROC: receiver operating characteristic; AUC: area under the curve; CI: confidence interval.

Figure 1. ROC curve for the AI-MHD, GAD-7, and HADS-A. Note: ROC: receiver operating characteristic; AUC: area under the curve; CI: confidence interval.

Discussion

To the best of our knowledge, the AI-MHD is the first disease-specific self-rating anxiety scale for MHD patients. To achieve specificity for the disease, we focussed on 2 aspects. First, when developing the scale items, we kept some commonly used items with high specificity for anxiety detection while removing some of the physical symptoms, such as fatigue, loss of appetite, and sleep disorders, which may overlap with the physical symptoms experienced by MHD patients and the side effects of drugs taken by dialysis patients (Burton et al., Citation2019; Pifer & Segal, Citation2020; Ziebold et al., Citation2019). Second, through a comparative analysis, some items with no significant difference between patients in the anxiety disorders group and those in the non-anxiety disorders group were removed. However, issues such as concurrent physical symptoms and diseases, the unique conditions of disease-related populations (such as the inability to work due to the disease), and the coexistence of a variety of factors should be considered when developing disease-specific scales.

Our results confirmed that the AI-MHD performed well in screening anxiety disorders in MHD patients. First, the multicentre test showed that the AI-MHD had good reliability, and Cronbach’s α was 0.918, which was higher than the previous report on an anxiety screening scale for MHD patients (Collister et al., Citation2019). Second, in terms of validity, the AI-MHD score not only significantly differed between the clinical anxiety disorders group and the non-anxiety disorders group (p < 0.001) but was also significantly correlated with the GAD-7 and HADS-A scores (p < 0.001). Furthermore, in terms of structural validity, the average factor load of the 3-factor model was 0.73 ± 0.09, and there was a good correlation among the 3 factors (p ≤ 0.01), suggesting good validity. Third, the sensitivity (0.93) and specificity (0.90) of the AI-MHD cut-off scores determined by ROC curve analysis were good ( and ). Lastly, we are encouraged by the fact that the sensitivity and specificity of the AI-MHD were slightly better than those for the GAD-7 and HADS-A used in this study and scales previously used in studies on dialysis patient anxiety (Collister et al., Citation2019; Ripamonti et al., Citation2014), indicating that the AI-MHD may be a high-precision tool for the detection of anxiety disorders. However, this scale requires more samples and more regions to better test its application effect.

Although the AI-MHD has the same problems as other screening scales or questionnaires, it is used as a screening tool rather than a diagnostic tool. It can be seen as a means of detecting prevalence. Our data showed that using the AI-MHD, the prevalence of anxiety disorders among patients was 40.5%, which is closer to the anxiety outcomes determined using the DSM-5 and interviews with physician specialists than using the GAD-7 and HADS-A. Data from this study are also similar to the results of other studies (Semaan et al., Citation2018) and are consistent with many studies suggesting that the prevalence of anxiety disorders among MHD patients is 20%-55% (Brito et al., Citation2019; Gerogianni et al., Citation2018; Kopple et al., Citation2017). Because the occurrence of anxiety disorders in MHD patients fluctuates, in addition to using different detection scales, cut-off values, populations and regions, a non-specific detection scale is an important cause of inconsistent results and our motivation for developing the AI-MHD. Furthermore, the incidence rates in this study indicated a high prevalence, possibly related to the high percentage of female MHD patients in our study (). Studies have shown that female MHD patients are more prone to anxiety disorders than are male patients (Hou et al., Citation2014; Untas et al., Citation2009). Lastly, the 12-item AI-MHD can be completed in approximately 4 minutes, which makes its management quick and convenient, indicating that it is very suitable as a screening scale.

This study has certain limitations. First, the study results were limited by the sample size and the 4 centres. Second, this study selected patients over 18 years of age because the cognitive and thinking modes of children and adolescents are at an undefined stage. Third, the results cannot be extrapolated to the family mode or peritoneal dialysis. Finally, the diagnosis of anxiety disorders was confirmed by a professional psychologist. Therefore, more psychologists or more comprehensive measures might be more appropriate.

Conclusion

The AI-MHD is a new self-rating screening scale for anxiety disorders in MHD patients. The statistical analysis showed that the AI-MHD had excellent reliability and effectiveness. More importantly, preliminary results showed that the sensitivity and specificity of the AI-MHD were slightly better than those of the GAD-7 and HADS-A used in this study. Nevertheless, the AI-MHD cannot solve all detection problems, and more subjects and more multicentre research are needed to improve and assess reliability, effectiveness and cut-off scores.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

The study was supported by the Research incubation project of the Third Affiliated Hospital of Chongqing Medical University (No. KY08042).

References