Psychometric evaluation of the Swedish self-rated 36-item version of WHODAS 2.0 for use in psychiatric populations – using classical test theory

Abstract

Aim

The aim of this study was to evaluate the reliability and validity of the Swedish version of the self-rated 36-item WHODAS 2.0 in patients from Swedish psychiatric outpatient settings, using classical test theory.

Methods

The 36-item WHODAS 2.0, together with the Sheehan Disability Scale (SDS), was filled in by a sample of 780 participating psychiatric patients: 512 (65.6%) women, 263 (33.7%) men, and 5 (0.6%) who did not report any sex.

Results

The internal consistency, measured by Cronbach’s alpha, for the different domains of functioning were between 0.70 and 0.94, and interpreted as good. The confirmatory factor analysis (CFA) revealed two levels: the first level consisted of a general disability factor, while the second level consisted of the six domains of the scale, respectively. The model had borderline fit. There was a significant correlation between WHODAS 2.0 36-item and SDS (n = 395). The WHODAS 2.0 differed significantly between diagnostic groups.

Conclusion

The present study demonstrates that the Swedish self-rated 36-item version of WHODAS 2.0, within a psychiatric outpatient population, showed good reliability and convergent validity. We conclude that the self-rated 36-item Swedish version of WHODAS 2.0 can be used for valid interpretations of disability in patients with psychiatric health conditions.

Keywords:

• Reliability
• validity
• confirmatory factor analysis
• Sheehan Disability Scale
• disability

Introduction

Early efforts to measure disability resulted in variable estimates due to the lack of common definitions. However, since 2002, the International Classification of Functioning, Disability and Health (ICF) framework, based on the biosocial model of disability, has been used increasingly [1]. The World Health Organization Disability Assessment Schedule (WHODAS 2.0) [2,3] is an instrument for measuring disability, based on the ICF framework. It has been translated into several languages, and its psychometric properties have been examined in numerous studies and validated in several studies spanning different languages, ranging from different health conditions (for a review see [4,5–7]). The use of valid instruments for measuring disability across diverse cultural populations, settings, and health conditions has implications for the estimation of disability on several levels, including from clinic to nationwide and international levels.

One of the updates in the fifth edition of the Diagnostic and Statistical manual of Mental Disorders (DSM-5) [8] was the introduction of the WHODAS 2.0 as a replacement for the formerly recommended Global Assessment of Functioning (GAF) scale. There were several reasons; a key element was that the scale conflated psychiatric symptoms with disability. In the World Health Organization system, diseases and disabilities are addressed and classified using different frameworks [9,10].

There is a need for psychometrically validated translations of WHODAS 2.0 evaluated in different settings. The Swedish translation was made in accordance with WHO guidelines by a working group under the Swedish National Board of Health and Welfare, that now provides a formal Swedish translation of WHODAS on their web-page. There is so far one publication about the psychometric properties of the self-reported 12-item version administered online to individuals with anxiety and stress disorders [11]. They found high internal consistency (Cronbach alpha 0.83–0.92), high test–retest reliability (ICC = 0.83), and adequate construct validity. There is no publication about the psychometric properties of the Swedish translation of the self-rated 36-item version. The aim of this study was to evaluate the reliability and validity of the Swedish version of the self-rated 36-item WHODAS 2.0 in patients from Swedish psychiatric outpatient settings using classical test theory.

Ethics

All participants were given written and oral information about the study, and those participating signed an informed consent form. The regional ethics board in Uppsala and in Stockholm approved all procedures, Reg.no. 2015/339 and Reg. no. 2014/1489-31/4, respectively.

Material and methods

Design and setting

The study was based on a large sample (N = 837) of psychiatric outpatients recruited during 2016 and 2017 from four selected regions of Sweden: Uppsala, Örebro, Dalarna, and Stockholm. All participants were recruited in the setting of their regular visits to the psychiatric outpatient clinic. The sample is a cross-sectional convenience sample, since no control over the recruitment process was established. The patients filled in the WHODAS 2.0 form, 36-item version. The attending clinician provided demographical and clinical information in the context of the current visit.

Participants

See Figure 1 for a description of the recruitment process and the diagnostic characteristic of the different samples used in this study. Of the collected forms, 315 (37.6%) came from two clinics in Stockholm, 303 (36.2%) came from a clinic in Uppsala, 159 (19.0%) came from a clinic in Örebro, and 60 (7.2%) came from a clinic in Dalarna. Among the patients, 544 (65.7%) were women, 284 (33.9%) were men, and 9 (1.1%) did not report any sex information. Data were missing in some of the 36-item WHODAS 2.0 forms, and 57 (6.8%) had missing data exceeding the suggested limit for imputation (see below) and were, therefore, excluded and constituted the internal dropouts.

Figure 1. Description of study sample, the recruitment process and diagnostic characteristics of the final sample and the test–retest sample, n (%).

The final sample therefore consisted of 780 participants: 512 (65.6%) women, 263 (33.7%) men, and 5 (0.6%) who did not report any sex. The main diagnoses were as reported by the clinicians. In the 98 participants (12.6%) where two or more diagnoses were reported, the following rules were used for selecting the main diagnosis. Mainly, it was inferred from the type of clinic from where the participants were recruited. For general psychiatric clinics that treated several disorders, the following rules were used based on severity: psychotic disorders judged as most severe, followed by mood disorders, and thereafter anxiety disorders. Patients reported to have both an eating disorder and ADHD/ASD were assigned an eating disorder diagnosis. For the main diagnoses, see Figure 1. The mean age of the sample was 39.5 ± 15.7 (range 19–80) years. Neither sex, main diagnosis, nor age differed significantly between the 780 participants and the 57 internal dropouts. The Sheehan Disability Scale (SDS) [12] was distributed together with the 36-item version of the WHODAS 2.0 to the 485 participants from Dalarna, Uppsala, and Örebro. The purpose was to compare two different methods for self-assessment of functional disability as a measure of convergent validity. Since 90 patients either did not report at all or had missing data on the SDS, only those with complete answers were included (n = 395). There were no significant differences between the SDS participants and dropouts according to sex or clinical diagnosis; however, the SDS dropouts were significantly younger, 34.8 years versus 46.2 years (Z = 5.138, p ≤ 0.001). There were 37 participants who reported on the WHODAS twice in order to test stability. The time between the two occasions varied, but only those who reported within one month were included in the test–retest analysis, excluding nine forms.

Instruments

World Health Organization Disability Assessment Schedule 2.0 (WHODAS 2.0)

The WHODAS 2.0 form measures functioning in activity and participation in daily activities during the previous 30 days. The form consists of 36 items, assessing a wide range of abilities, each corresponding to an ICF code [2]. Respondents are instructed to report any difficulties in activities or participation due to health conditions that have been encountered during the last 30 days. The items are grouped into the following six domains (D1–D6): (D1) Cognition – understanding & communication (items D1.1–D1.6); (D2) Mobility – moving & getting around (items D2.1–D2.5); (D3) Self-care – hygiene, dressing, eating & staying alone (items D3.1–D3.4); (D4) Getting along – interacting with other people (items D4.1–D4.5); (D5) Life activities – domestic responsibilities, leisure, work & school (items D5.1–D5.8); and (D6) Participation – joining in community activities (items D6.1–D6.8). Respondents are instructed to answer all 36 items, except for those unemployed or no longer undergoing academic studies, who only answered 32 of the items, leaving out items D5.5 through D5.8, since these items ask questions about functioning in work and/or studies.

The original English version of the WHODAS was validated as part of the development process and was demonstrated to have good psychometric properties, including concurrent validity to other established instruments measuring functioning level [2].

There are three modes of administering the scale: one self-reported, one where it is administered as a structured interview by a clinician, and one where it is administered by a proxy that is well acquainted with the respondent e.g. caretaker, close friend, or spouse. There is also a short, 12-item, and a hybrid, 12 + 24 item, besides the long 36-item, version of the WHODAS 2.0. In this study, the respondents filled in the 36-item self-report version. Items are scored using a 5-point scale. The items are rated “none,” “mild,” “moderate,” “severe,” and “extreme,” and on the questionnaire these correspond to 0–4. There are two options for computing the summary scores for the WHODAS 2.0: one simple and one complex. The simple way is to sum up all scores without recoding or collapsing of response categories, where there is no weighting of individual items. The sum of the 36-items scored 0–4 ranges from 0 to 144. This approach is practical for use as a hand-scoring approach, and may be the method of choice in busy clinical settings, and it is easily calculated by using the scoring sheet on the WHO website. Simple scoring of WHODAS 2.0 is specific to the sample at hand and should not be assumed to be comparable across populations. The more complex method of scoring is based on item-response-theory (IRT), and it takes into account multiple levels of difficulty for each WHODAS 2.0 item. For both methods, the resulting score is transformed into a 0–100 scale, with higher numbers indicating a higher level of disability. The simple method for computing the summary scores of the WHODAS 2.0 was used for this study.

Sheehan Disability Scale (SDS)

SDS was developed in 1983 [12] and measures functioning in work/school, social and family life during the last month. The psychometric examinations have found high internal consistency and one factor structure [13–15]. This was also replicated in the Swedish translation [16]. In addition, SDS was shown to be reliable and valid in patients with different psychiatric disorders [13,14,17]. The SDS is self-reported and rated between zero and 10 on each of the three subscales for functioning in work/school, social and family life. Zero means “the symptoms do not affect function at all,” and 10 means “symptoms have an extreme effect on functioning.” The total score adds the ratings from all three areas into a sum ranging from zero to 30.

Data and statistical analysis

Imputation and data editing

In accordance with guidelines from WHO [2], missing data in WHODAS 2.0 forms were imputed if there were no more than two missing items from the form, and no more than one missing in the same domain. Imputed scores from the WHODAS 2.0 form were calculated as a whole: the mean of the answered items in the same domain was calculated, rounded off to the nearest integer, and entered into the missing item in the domain [2].

Statistical analysis

Dropout analyses were analyzed with Chi-square for dichotomous data and Mann–Whitney U test for dimensional data. Internal consistency of the six WHODAS 2.0 domains was measured with Cronbach’s alpha, and interpreted as good if 0.7 ≤ α ≤ 0.9 [18]. A confirmatory factor analysis with maximum likelihood estimation was run using R software version 3. 5.1. Using the combination strategy of [19], values of Comparative Fit Index (CFI) ≥0.95, Tucker Lewis Index (TLI) ≥ 0.95, Root Mean Square Error of Approximation (RMSEA) ≤0.06, and Standardized Root Mean Residual (SRMR) ≤0.08 were interpreted as indicators of good model fit. Convergent validity between WHODAS 2.0 and SDS was assessed using Pearson’s correlation in the patients who reported on both questionnaires. Pearson was selected due to the relatively big sample that was approximately normally distributed. Mean differences in the WHODAS total score between the different diagnostic groups, those with no diagnosis (n = 25) excluded, were compared with Kruskal–Wallis test with Bonferroni correction. Test–retest reliability was analyzed using intra-class correlation coefficient (ICC). Intra-class correlation coefficients were considered excellent if greater than 0.74, good if ranging from 0.60 to 0.74, and fair if ranging from 0.40 to 0.59 [20]. Statistical analyses, except for the confirmatory factor analysis, were performed using IBM SPSS Software version 25 (SPSS Inc., Chicago, IL).

Results

Internal consistency

The correlation between items in the scale, the internal consistency, measured by Cronbach’s alpha is presented for all items in each domain in Table 1. Using the seven-factor model splitting the eight items in domain (D5) into two domains, consisting of the first four (D5a) and last (D5b) four items, yielded a Cronbach alpha coefficient of 0.94 and 0.95, respectively.

Table 1. Ratings on the 36-item WHODAS 2.0 in Swedish psychiatric outpatients (N = 780), range 0–4.

Item	Mean	SD	Correlated Item-Total Correlation	Cronbach’s alpha if item deleted
Domain 1 – Cognition, Cronbach item deleted
D11	1.28	1.06	0.70	0.84
D12	1.46	1.07	0.66	0.84
D13	1.34	1.08	0.71	0.84
D14	1.06	1.06	0.64	0.85
D15	0.87	0.95	0.65	0.85
D16	1.07	1.08	0.62	0.85
Domain 2 – Mobility, Cronbachh item delete
D21	1.00	1.12	0.66	0.82
D22	0.63	0.90	0.69	0.82
D23	0.44	0.79	0.72	0.82
D24	0.93	1.12	0.62	0.82
D25	0.78	1.12	0.69	0.81
Domain 3 Cronbachh item deletedn in Swedis
D31	0.46	0.86	0.52	0.63
D32	0.36	0.74	0.58	0.62
D33	0.87	1.13	0.50	0.64
D34	0.97	1.28	0.46	0.68
Domain 4 Cronbachh item deletedn in Swedish ps
D41	1.14	1.15	0.63	0.77
D42	1.16	1.14	0.72	0.75
D43	0.98	0.99	0.54	0.80
D44	1.52	1.36	0.71	0.75
D45	1.40	1.43	0.48	0.83
Domain 5 – Life activities (n = 648), Cronbach (hh item del
D51	1.29	1.18	0.77	0.94
D52	1.18	1.14	0.74	0.94
D53	1.54	1.25	0.80	0.93
D54	1.60	1.30	0.77	0.93
D55	1.66	1.33	0.79	0.93
D56	1.44	1.30	0.78	0.93
D57	1.56	1.37	0.83	0.93
D58	1.55	1.40	0.83	0.93
Domain 6 onbach (hh item deletedn in Swedish ps
D61	1.48	1.28	0.67	0.86
D62	1.01	1.08	0.59	0.87
D63	1.02	1.14	0.63	0.87
D64	1.80	1.10	0.53	0.88
D65	2.25	1.18	0.72	0.87
D66	1.41	1.35	0.64	0.86
D67	1.44	1.18	0.70	0.86
D68	1.53	1.31	0.67	0.86

High scores indicate high disability.

Confirmatory factor analysis (CFA)

Including 32 items, as in the original validation study [2] leaving out items 5.5 through 5.8, in the CFA, the model had borderline fit: χ² (458, N = 780) = 2057.3, p ≤ 0.001; CFI = 0.892; TLI = 0.884; RMSEA = 0.067; and SRMR = 0.062. The factor structure was revealed to consist of two levels: the first level consisted of a general disability factor, while the second level consisted of the six domains of the scale, respectively, see Figure 2.

Figure 2. Confirmatory Factor analysis, using 32-items of the original WHODAS 2.0 scale, leaving out items D5.5 through D5.8, Domains 1–6 corresponds to: cognition, mobility, self-care, getting along, life activities, and participation.

Convergent validity

There was a significant correlation between the WHODAS 2.0 36-item and SDS (n = 395), with a Pearson’s correlation coefficient of 0.77; p ≤ 0.001, for scatter plot and regression line, see Figure 3.

Figure 3. Scatter plot and regression line of correlations between the total sum of the 36-item version of WHODAS 2.0 and the Sheehan Disability Scale (SDS), (n = 395).

Predictive validity

Ratings on WHODAS 2.0 for each diagnostic group is presented in Figure 4. The median (25^th percentile; 75^th percentile) for each diagnostic group was as follows: anxiety disorders 91 (75;104), mood disorders 69 (50;94), eating disorders 80 (63;99), psychotic disorders 73 (58;88), substance related disorders 73 (57;97), ADHD/ASD 81 (66;98), and other diagnoses 92 (61;114). The WHODAS 2.0 differed significantly between the diagnostic groups (H = 39.31; p ≤ 0.001). Mood disorders and psychotic disorders had significantly lower WHODAS-scores when Bonferroni post hoc test was applied, e.g. less disability, as compared to anxiety disorders and other diagnoses.

Figure 4. Results on the 36-item self-administered version of the WHODAS 2.0 (range 0–144) in psychiatric outpatients presented for different groups based on their main diagnosis (n = 755). Patients without a known diagnosis are not presented in the table (n = 25, 3.2%).

Test–retest reliability

There were 37 patients who reported twice on the WHODAS 2.0. The mean time between the ratings was 12.8 (7.9) days. The ratings at the first and second time correlated significantly (p ≤ 0.001) for all six domains as well as for the total sum. ICC for the total scale was 0.83. For domains 1–6, the ICC was 0.84, 0.86, 0.88, 0.80, 0.79, and 0.81, respectively.

Discussion

This study examined the reliability and validity of the Swedish version of the self-administered 36-item version of the WHODAS 2.0 by using classical test theory. The results demonstrate that the translated form, as applied to a psychiatric outpatient population who assessed their daily functioning, has satisfactory psychometric properties. As the process of translating the WHODAS 2.0 scale into more languages proceeds, validation studies such as this one explore properties that are shared across languages as well as properties that are inherent to the local translation.

For the Swedish translation, the Cronbach’s alpha values in the different domains were between 0.70 and 0.94, and therefore internal consistency was interpreted as good. The Swedish version performed similarly to the original [2], and to and to a Norwegian version evaluated in specialized rehabilitation services in Norway [21], as well as to the Swedish 12-item self-report version [11]. WHODAS 2.0 has been validated in many languages and in different psychiatric populations, for example in people with severe mental disorders in rural Ethiopia [22], in patients with autism spectrum disorders in Australia [23], and in Canadian psychiatric emergency patients [6], and always maintained its’ psychometric properties.

In a review of studies performed before 2017 [4], the Cronbach’s alpha was sometimes found to be lower in the domain of self-care [24,25]. This is in line with the results from this study, where Cronbach’s alpha for this domain was the lowest, 0.70. A high value of alpha, over 0.90, as for D5-life activities in this study, may suggest redundancies [18], meaning that some items are in fact unnecessary, and could be removed. Therefore, the values between 0.82 and 0.88 for the four domains: D1-cognition, D2-mobility, D4-getting along, and D6-participation, could be considered most optimal. When the WHODAS 3.0 is evolved in the future, it might be possible to make improvements to the domains D3-self-care and D5-life-activities, which should allow for new psychometric studies.

To compare the factor structure of the Swedish translation with the original version, confirmatory factor analysis was performed only on the 32-item version of the scale (even in cases where the respondent had provided a valid 36-item form). Using the combination strategy of [19] to evaluate model fit, CFI and TLI were somewhat low, but RMSEA and SRMR indicated good model fit. Factor loadings were psychometrically acceptable although inferior to the original scale [2]. However, factor loadings were more similar to those found in an adolescent Chinese in-patient sample [26], in people with severe mental disorders in Ethiopia [22], in Norwegian somtaic rehabilitation patients [21], and in patients with Multiple Sclerosis in Iran [27]. Most studies have been performed in older age groups [4], and this study was performed within a large age span (19–80 years). It is not known whether the psychometric properties of the WHODAS is influenced by age, but age is likely to influence the individual’s expectations of functioning.

Convergent validity was examined by comparisons with another instrument for self-assessment, the SDS, and showed high correlations. However, there may be differences between the self-administered and expert rated levels of functioning. This supports WHODAS as being a valid measure of patient experienced functioning. However, validity does not automatically presume reliability, such that the experienced level of functioning is comparable to a more objective assessment of functioning. To shed light on whether the self-assessment is reliable or not, it has to be compared with an expert rated measure with high inter-rater reliability. The participating patients were in all stages of disease, ranging from acute illness to partial or total remission. The strong correlation between the SDS and WHODAS ratings supports its validity as a self-report instrument in all stages of disease. The correlations with SDS was higher than in the study by von Korff et al. [25], but the von Korff study used population-based samples with less functional impairment, which could explain the difference. Convergent validity with other measures of functioning has shown similar results [22,27,28].

Comparing ratings between different diagnostic groups as a measure of predictive validity revealed that mood disorders and psychotic disorders had significantly lower WHODAS scores, implicating less impairment, compared to anxiety disorders and the unspecified group called other disorders (a group where many patients were diagnosed with a borderline personality disorder). This was unexpected, since patients with psychotic disorders and mood disorders, especially bipolar disorders, are expected to be severely impaired, at least during active illness. However, the study sample included about 200 patients with bipolar disorder predominantly in a stable phase at their annual follow-up. Furthermore, it has previously been shown that more symptoms in patients with bipolar disorder were related to higher ratings on WHODAS [29], while less symptoms were related to lower ratings. The same relationship between symptom severity and WHODAS ratings has been found in patients with traumatic brain injury and stroke [30,31]. Patients’ own reports might also be biased due to unawareness of their difficulties [32], something that could have influenced the ratings by patients with schizophrenia. Using the WHODAS by the combination proxy- and self-assessment option might be preferred when cognitive functioning is severely affected [30]. However, unawareness of difficulties seems to be more related to severe mental disorders then to cognitive impairment by traumatic brain injury [33], since this later group seems to rate WHODAS in a reliable way.

Test–retest reliability was excellent. This has been shown previously, both when the time between the two tests were short as two days in a Polish study with patients with low back-pain [28] or have been longer, as with the inter-test interval of 12 weeks in a study with older patients with schizophrenia [34]. The one-month limitation interval in this study was based on the time interval rated on the WHODAS, the previous month. The test–retest sample was constituted by a relatively large group of patients with eating disorders, with symptoms expected to be more stable over a one-month period, which might have increased the stability.

The study has some limitations. Since this study was conducted on a convenience sample, there was no control over the number of respondents who declined to enter the study. It is conceivable that the participating respondents differ from those who declined, but we do not know how. Also, recruitment practices varied among the participating clinics. There were no ratings of current symptoms, which makes the comparisons between the different diagnostic groups difficult to interpret. It is likely that patients with current severe symptoms were less likely to fill out the form. The diagnoses were clinical, and the diagnostic procedures probably differed between different clinics. The influence of comorbidity on functioning was not explored.

To sum up, the present study demonstrates that within a psychiatric outpatient population, the Swedish self-administered 36-item version of WHODAS 2.0 showed good reliability and convergent validity. We conclude that the self-administered 36-item Swedish version of WHODAS 2.0 can be used for valid interpretations of disability in patients with psychiatric health conditions.

Author contributions

All authors took part in the conceptualization of the study design, methods and aims. All authors contributed to resources, included participants, collected data, and took part in the analyses. Robin Midhagen and Mia Ramklint wrote the initial manuscript, that was drafted and reviewed by all the other authors. Liselotte Hermansson was the project manager and provided the financial support.

Acknowledgments

The authors express their gratitude to all participating patients and staff.

Disclosure statement

RM have no potential conflict of interest. LH have no potential conflict of interest. PS have no potential conflict of interest. ST have no potential conflict of interest. AN have no potential conflict of interest. CS have no potential conflict of interest. YG has over the past 5 years received speaker fees, royalties, reimbursement for travel costs and/or served as a consultant in the field of ADHD for Novartis, Shire, Boardman Clarke Partners, Medscape, Medibas, Studentlitteratur, and Natur & Kultur, all outside the submitted work. MR have no potential conflict of interest.

Additional information

Funding

The research was supported by grants from Uppsala-Örebro Regional Research Council [grant ID RFR 473401], the Stockholm County Council (ALF project), and Söderström-Königska Foundation.