Investigating the reliability and validity of the Dutch versions of the illness management and recovery scales among clients with mental disorders

Abstract

Background: The Illness Management and Recovery scales (IMRS) can measure the progress of clients’ illness self-management and recovery. Previous studies have examined the psychometric properties of the IMRS.

Aims: This study examined the reliability and validity of the Dutch version of the IMRS.

Method: Clients (n = 111) and clinicians (n = 40) completed the client and clinician versions of the IMRS, respectively. The scales were administered again 2 weeks later to assess stability over time. Validity was assessed with the Utrecht Coping List (UCL), Dutch Empowerment Scale (DES), and Brief Symptom Inventory (BSI).

Results: The client and clinician versions of the IMRS had moderate internal reliability, with α = 0.69 and 0.71, respectively. The scales showed strong test–retest reliability, r = 0.79, for the client version and r = 0.86 for the clinician version. Correlations between client and clinician versions ranged from r = 0.37 to 0.69 for the total and subscales. We also found relationships in expected directions between the client IMRS and UCL, DES and BSI, which supports validity of the Dutch version of the IMRS.

Conclusions: The Dutch version of the IMRS demonstrated good reliability and validity. The IMRS could be useful for Dutch-speaking programs interested in evaluating client progress on illness self-management and recovery.

Keywords:

• Illness Management and Recovery Scale
• mental health
• psychometric properties
• reliability
• validity

Introduction

The Illness Management and Recovery program was developed by integrating effective psychosocial interventions to teach clients with severe mental illness techniques to manage mental illness and work toward recovery (Mueser, 2002; Mueser et al., 2006). The Illness Management and Recovery program has been proven effective in randomized controlled trials (Färdig et al., 2011b; Hasson-Ohayon et al., 2007; Levitt et al., 2009) and in quasi-experimental trials (Fujita et al., 2010; Garber-Epstein et al., 2013; Salyers et al., 2010, 2011; Whitley et al., 2009).

Along with the program, scales were developed to evaluate the effectiveness of the program. The Illness Management and Recovery Scales (IMRS; Mueser & Gingerich, 2005) were designed to measure progress in activities and outcomes considered to be related to the program, including knowledge about the illness, social support, medication adherence, relapse prevention, coping and substance abuse. These domains can be monitored from client and clinician viewpoints using parallel versions of the same scale. Overall, the IMRS have shown moderate to good internal reliability, test–retest reliability and reasonable support for convergent validity (Färdig et al., 2011a; Hasson-Ohayon et al., 2008; Salyers et al., 2007).

Although each scale is used as a one-dimensional measure, the IMRS were initially developed to measure recovery progress in multiple domains. In their psychometric study Hasson-Ohayon et al. (2008) identified three dimensions in the IMRS: (1) “Coping” – the extent to which the clients’ coping reduces symptom relapse, symptom distress and impairment of functioning; (2) “Knowledge and Goals” – the extent to which the client is skilled in obtaining knowledge about the illness and is able to set and strive for personal goals and (3) “Medication” – the extent to which the client is using medication effectively and is able to reduce the abuse of alcohol and drugs. A recent study supported the existence and reliability of three factors, but only examined the clinician version. Sklar et al. (2012) showed a similar factor structure to that of Hasson-Ohayon et al. (2008) on the IMRS clinician version. However, a few items loaded differently and they changed the naming of the factor structure to “Management”, “Recovery” and “Substance”, corresponding to the factors “Coping”, “Knowledge and Goals” and “Medication” of the Hasson-Ohayon et al. (2008) study, respectively.

In addition to factor structure, some studies have examined the correspondence between client and clinician report. Three studies (Färdig et al., 2011a; Hasson-Ohayon et al., 2008; Salyers et al., 2007) found low to moderate correlations between client and clinician perspectives (ranging between 0.23 and 0.58), suggesting that the scales may be tapping different dimensions or perspectives of recovery. For example, Hasson-Ohayon et al. (2008) found subtle differences which suggest that the client perspective might be more related to coping, whereas the clinician perspective is more focused on the self-management aspect of recovery.

In terms of convergent validity, IMRS have been linked to a variety of constructs in the expected direction, including symptoms (rated by self or others), perception of recovery, quality of life and coping (Färdig et al., 2011a; Hasson-Ohayon et al., 2008; Salyers et al., 2007; Sklar et al., 2012). Given the focus of the Illness Management and Recovery program on managing illness, determining the relationship with empowerment, symptom distress and coping would appear particularly salient. Hasson-Ohayon et al. (2008) assessed coping as a one-dimensional construct with the Coping Efficacy Scale (CES; Mueser et al., 1997) and showed that more efficacious coping was related to higher scores on the IMRS. However, coping was assessed specifically in terms of how clients were dealing with symptoms. Previous research suggests that coping can be a cognitive style or trait of how people deal with problems beyond symptoms, and different coping styles have been identified (Schreurs et. al., 1993). Despite criticism to the approach versus avoidance dichotomy (Skinner et al., 2003), a passive avoidant coping style is most strongly associated with severity of mental health problems, across a range of mental health conditions (Goossens et al., 2008; Holahan et al., 2005; McAuliffe et al., 2006; Schnider et al., 2007; Sherbourne et al., 1995). Given the strong link between coping and illness management and recovery skills, one would expect that efficacious coping with symptoms and less passive avoidant coping would be associated with better scores on the IMRS.

The Illness Management and Recovery program is currently being implemented in the Netherlands, and although English, Swedish and Hebrew versions of the IMRS have been developed and evaluated (Färdig et al., 2011a; Hasson-Ohayon et al., 2008; Salyers et al., 2007; Sklar et al., 2012), a Dutch psychometric study is currently lacking. Before using the IMRS in a Dutch population, its psychometric qualities should first be established, including how the total IMRS and underlying dimensions relate to measures of self-management and coping. The aim of this study was to investigate the psychometric properties of the Dutch version of the IMRS from perspectives of both clients and clinicians. Due to the emphasis of the Illness Management and Recovery program on coping skills and self-management skills, and prior differences between the client and clinician versions of the IMRS, we examined the relationship between the client and clinician IMRS and explored the relationship of the IMRS with measures of coping and self-management.

Methods

Participants

Participants were 111 adults recruited from one of two psychiatric institutes in the Netherlands providing regular supportive housing or outpatient care. Participants were not attending an Illness Management and Recovery program. Clients were recruited by posters and information provided by clinicians. Inclusion criteria were: age 18–65 years, a good understanding of the Dutch language and a diagnosis of a mental disorder made by a psychiatrist according to the DSM-IV criteria (Table 1 gives an overview of participants’ demographic characteristics). Also, participants had to be able to provide informed consent, and psychiatrically stable enough to participate in the study as assessed by the case manager.

Table 1. Demographic characteristics of the participants (N = 111).

Variables	n (%)
Psychiatric diagnosis (N)	111
Pervasive development disorder	21 (18.9)
Attention disorder	2 (1.8)
Personality disorder	13 (11.7)
Anxiety disorder	5 (4.5)
Mood disorder	8 (7.2)
Psychotic disorder	11 (9.9)
Mental retardation	1 (0.9)
Addiction	3 (2.7)
Axis II and Axis I combined	25 (22.5)
Multiple Axis I diagnoses	11 (9.9)
Other/Not specified	11 (9.9)
Gender	108
Male	55 (48.7)
Female	53 (46.9)
Education	101
Primary School	11 (10.9)
Lower Secondary	43 (42.6)
Upper Secondary	11 (10.9)
Post-secondary non-tertiary	22 (21.8)
Bachelor-Master or equivalent	14 (13.9)
Employment	103
Paid job	21 (20.4)
Volunteer	22 (21.4)
Education	4 (3.5)
Adult Day Care program	21 (18.6)
None	35 (34.0)
Income	102
Salary	22 (21.6)
Paid benefits	78 (76.5)
Scholarly benefits	2 (2.0)
Born in the Netherlands	105
Yes	98 (93.3)
No	7 (6.7)
Independent living	105
Yes	88 (83.8)
No	16 (15.2)

Clinicians

Forty case managers working in the same psychiatric institutes were asked to fill out the clinician version of the IMRS for his or her participating client (ranging from 1 to 8 clients per clinician). No personal data were collected from the clinicians.

Instruments

IMRS

The IMRS include parallel client and clinician versions (Mueser & Gingerich, 2005). Each version contains 15 items, rated on a five-point scale. The client and clinician version of the IMRS were double translated into Dutch and back translated by a native speaker into English to control for translation accuracy and validity. Differences between the translations were discussed with two experts in the field of IMR. All changes were made involving the original author Kim Mueser in the decision making process.

The designers of the original IMRS used a one-dimensional scale summing or averaging items into a total score. In accordance with Salyers et al. (2007), we used a mean instead of sum for our analysis. Previous psychometric studies found a strong to moderate internal reliability ranging from 0.82 to 0.55 for the client version of the IMRS (Hasson-Ohayon et al., 2008; Salyers et al., 2007; Sklar et al., 2012), and 0.73 to 0.80 for the clinician version (Hasson-Ohayon et al., 2008; Salyers et al., 2007). We based our calculations on the factor analysis from Sklar et al. (2012), using the factor Recovery (items 1, 2, 4, 8 and 12), Management (items 6, 7, 9 and 11) and Substance (items 14 and 15). We chose the Sklar et al. (2012) factors because of good fit in their study, and we agreed with the rationale for adding item 12 to the Recovery factor and excluding item 13 from the Substance factor.

Coping

The Utrecht Coping List (UCL; Schreurs et al., 1993) was administered to measure client coping style. The UCL is a 47-item questionnaire measuring seven different coping styles: active approach, palliative reactions, avoidance/abide, searching for social support, passive coping, expression of emotions and using reassuring and comforting thoughts. Cronbach’s alpha levels indicate good internal reliability for the subscales of the UCL ranging from 0.67 to 0.82, and previous research has shown support for adequate concurrent validity (Schreurs et al., 1993). In our sample the Cronbach’s alpha of the subscales of the UCL ranged from 0.67 to 0.86.

Recovery

The Dutch Empowerment Scale (DES) (Nederlandse Empowerment Lijst; Boevink et al., 2008) is a self-report questionnaire to determine the degree of empowerment in psychiatric clients. The DES consists of 40 items tapping six dimensions: professional support, social support, headstrong, belonging, self-management and involved community. Each item is rated from 1 = strongly disagree to 5 = strongly agree. A psychometric study of the NEL showed good internal consistency (Cronbach’s α = 0.93) and correlations with other measures indicating acceptable construct validity (Boevink et al., 2008). In our sample the Cronbach’s alpha of the DES was 0.86.

Symptom list

The Brief Symptom Inventory (BSI) is a self-report questionnaire to assess psychopathology (Derogatis & Melisaratos, 1983). The BSI consists of 53 items covering a broad range of psychosocial problems. Each item is rated on a five-point scale ranging from 0 (“not at all”) to 4 (“extremely”). Additionally, the BSI also measures three global indices of distress, the Global Severity Index, measuring the overall psychological distress level, Positive Symptom Distress Index, measuring the intensity of the experienced symptoms and the Positive Symptom Total, measuring the total number of symptoms. Previous studies show good internal reliability for the subscales of the BSI ranging from 0.63 to 0.89 and good support for concurrent validity (Coelho et al., 1998; Derogatis & Melisaratos, 1983). The focus of the BSI is on experienced stress and not on actual number of symptoms. In our sample the Cronbach’s alpha of the subscales of the BSI ranged from 0.77 to 0.91.

Procedure

The study was conducted in accordance with the declaration of Helsinki. However, according to Dutch legislation full ethical approval was not obliged because participants did not receive treatment or asked to behave in a particular way (www.ccmo.nl/en/). Data collection was not related to an intervention study. Only questionnaires were administered, and those “would not in principle come within the scope of the Act, unless either the frequency with which a subject was asked to complete a questionnaire were sufficient to bring about a temporary change in the subject’s lifestyle or the (psychologically probing) nature of the questions were such that the subject could be regarded as having received a particular treatment or having been asked to behave in a particular way”. Approval for this study was obtained via the management boards and the client council of the institutes.

Clients were recruited for participation in the study by their case managers and with information leaflets and posters that were presented at the different health care locations. Client and their case managers both completed the IMRS at a 2-week interval. Clients also completed the other measures at Time 1. Case managers and researcher coordinators were present for instruction and questions concerning the study.

Analysis

We used SPSS software version 20 (IBM Corp., Armonk, NY) to examine the IMRS on internal reliability, test–retest reliability and concurrent validity. Confirmatory factor analysis was not appropriate because of the small sample size. First, internal reliability was examined by calculating Cronbach’s alpha on the total IMRS score and for each of the subscales. Test–retest reliability was examined by Pearson’s correlations calculated between the two administrations. We also examined Pearson’s correlation between the clinician and client versions of the IMRS to examine the level of correspondence between them.

Pearson’s correlations were also calculated between all measures (BSI, UCL and DES) and the total IMRS score and the IMRS subscales to examine construct validity.

As passive coping and avoidance style have been associated with mental health problems and decrease of self-efficacy (Goossens et al., 2008; Holahan et al., 2005; Schaufeli & van Dierendonk, 1992) we expected negative correlations with passive coping and avoidance coping style and positive correlations with active coping style for the IMRS-Total and the IMRS-Management factor. We also expected high correlations between the IMRS total score and the DES dimensions of headstrong, belonging and self-management. At the subscale level, we expected self-management to be strongly related to the IMRS-Management factor, and we expected social support, headstrong and belonging to be related to the IMRS-Recovery factor. As professional support is aimed at the caregiver perspective on recovery and involved community is aimed at the perspective of society, we expected close to zero correlations between the IMRS total score and the DES’ dimensions professional support and involved community, which will demonstrate divergent validity. We expected that clients showing more recovery and self-management would score lower on experienced distress (i.e. negative correlations between IMRS total score and the BSI total and subscales and the UCL avoidance abide and passive coping.

Results

Internal reliability

Cronbach’s alpha using all the items was α = 0.61 for the client version and α = 0.69 for the clinician version of the IMRS. When excluding item 13 (regarding medication use) from the reliability analysis, internal reliability improved, with Cronbach’s α = 0.69 for the client version and Cronbach’s α = 0.71 for the clinician version. We omitted item 13 from the remainder of the analyses. As 18% of respondents omitted item 13 we examined diagnoses to better understand the omission: nine had a pervasive development disorder, four had a personality disorder, one had attention deficit disorder, one had a diagnoses deferred on Axis II and two had “other conditions that may be a focus of clinical attention”.

Of the three subscales, only IMRS-Management showed acceptable Cronbach’s alpha levels, with α = 0.77 and α = 0.74 for the client and the clinician versions, respectively. All other alpha levels were questionable to poor for all subscales (Table 2 shows exact values).

Table 2. Pearson correlations between the clients and clinician total IMRS and factors.

	Client IMRS-Total	Client IMRS-Management	Client IMRS-Recovery	Client IMRS-Substance	Clinician IMRS-Total	Clinician IMRS-Management	Clinician IMRS-Recovery	Clinician IMRS-Substance
Client IMRS-Total	(0.69)
Client IMRS-Management	0.74**	(0.77)
Client IMRS-Recovery	0.78**	0.35**	(0.53)
Client IMRS-Substance	0.33**	0.04	0.12	(0.57)
Clinician IMRS-Total	0.63**	0.51**	0.56**	0.11	(0.71)
Clinician IMRS-Management	0.61**	0.66**	0.46**	0.04	0.78**	(0.74)
Clinician IMRS-Recovery	0.45**	0.32**	0.51**	−0.07	0.80**	0.46**	(0.61)
Clinician IMRS-Substance	0.08	−0.10	0.09	0.37**	0.40**	0.15	0.10	(0.28)

**p < 0.01.

Diagonals between parentheses represent internal reliability coefficients.

Test–retest reliability

Correlations between 2-week administrations showed strong test–retest reliability with r = 0.79, p < 0.001 (n = 92) for the client version of the IMRS and r = 0.86, p < 0.001 (n = 89) for the clinician version of the IMRS. Test–retest reliability was also strong for all subscales except for the client subscale Recovery and Substance with r = 0.64, p < 0.001 (n = 92) and r = 0.67, p < 0.001 (n = 85), respectively.

Correlations between clinicians and clients on the total and subscale scores of the IMRS

Table 2 shows the correlations between clients and clinicians on the total and subscale scores. Overall, client and clinician versions were significantly correlated for the total and subscale scores. The client and clinicians ratings on Management (r = 0.66) correlated higher than their ratings on Recovery (r = 0.51). The magnitude was much lower for the subscale Substance (r = 0.37).

Construct validity

Correlation with UCL

As shown in Table 3, as hypothesized, there were moderate to strong correlations between the client-rated IMRS total scale, IMRS-Management, and IMRS-Recovery scales and the UCL coping dimensions of active approach, avoidance and passive coping. However, the clinician scales performed differently. The clinician-rated IMRS total scale was associated with client reports of active approach, but not with avoidance and passive coping. The clinician subscale IMRS-Management was negatively associated with passive coping, but no association was found with active approach and avoidance coping.

Table 3. Pearson correlations between the UCL and the IMRS.

	UCL
	Active approach	Palliative reactions	Avoidance abide	Searching for social support	Passive coping	Expression of emotions	Reassuring comforting thoughts	UCL Total
Client IMRS−Total	0.39**	0.08	−0.37**	0.20*	−0.46**	−0.01	0.20*	−0.01
Client IMRS−Management	0.22*	−0.05	−0.37**	0.03	−0.66**	−0.06	0.18	−0.22*
Client IMRS−Recovery	0.33**	0.19	−0.18	0.28**	−0.17	0.12	0.12	0.17
Client IMRS−Substance	0.20*	0.04	−0.10	0.14	−0.16	−0.09	0.19	0.06
Clinician IMRS−Total	0.21*	0.04	−0.17	0.09	−0.19	−0.00	0.06	0.01
Clinician IMRS−Recovery	0.19	0.09	−0.11	0.11	−0.15	0.03	0.06	0.04
Clinician IMRS−Management	0.17	0.04	−0.17	0.07	−0.34**	0.02	0.02	−0.07
Clinician IMRS−Substance	−0.06	−0.14	0.02	0.07	0.12	−0.00	−0.07	−0.01

*p < 0.05; **p < 0.01.

Correlation with DES

As expected, and shown in Table 4, the client IMRS total score correlated significantly with the DES dimensions Belonging, Self-management and Total score. Unexpectedly, a moderate significant correlation was also found with Involved Community (r = 0.37; p < 0.01). The Client IMRS-Management subscale correlated highest with DES Self-management, and the Client IMRS-Recovery subscale correlated highest with the DES dimension Belonging.

Table 4. Pearson’s correlations between the DES and the IMRS.

	DES
	Professional support	Social support	Headstrong	Belonging	Self−management	Involved community	NEL Total
Client IMRS−Total	0.03	0.23*	0.06	0.52**	0.50**	0.37**	0.54**
Client IMRS−Management	−0.06	0.19	0.16	0.35**	0.49**	0.40**	0.50**
Client IMRS−Recovery	0.01	0.13	−0.01	0.39**	0.35**	0.23*	0.36**
Client IMRS−Substance	0.07	0.00	0.00	0.17	0.12	0.08	0.08
Clinician IMRS−Total	0.07	0.23*	0.03	0.23*	0.34**	0.35**	0.36**
Clinician IMRS−Management	0.03	0.15	0.14	0.18	0.36**	0.28**	0.33**
Clinician IMRS−Recovery	0.03	0.25*	−0.06	0.24*	0.30**	0.33**	0.33**
Clinician IMRS−Substance	0.17	0.03	−0.21	−0.07	−0.07	−0.01	−0.01

*p < 0.05; **p < 0.01.

Clinician IMRS total score correlated with Self-management, Involved community and Total score. The Clinician IMRS-Management subscale correlated highest with Self-management and the Clinician IMRS-Recovery subscale correlated highest with Involved Community.

Correlation with BSI

Moderate to strong negative correlations were found between IMRS total score, both client and clinician versions, with the BSI scales. IMRS client total score showed a significant relationship with the Global Severity Index and the number and severity of symptoms (Table 5). The client IMRS-Management subscale also correlated strongly negatively with these indices, whereas the IMRS-Recovery and IMRS-Substance subscale correlations were weaker.

Table 5. Pearson’s correlations between the BSI and the IMRS.

	BSI
	Positive Symptom Total	Positive Symptom Distress Index	Global Severity Index
Client IMRS-Total	0.51**	0.58**	0.58**
Client IMRS-Management	0.54**	0.57**	0.60**
Client IMRS-Recovery	0.28*	0.34**	0.33**
Client IMRS-Substance	0.16	0.29**	0.23*
Clinician IMRS-Total	0.29**	0.33**	0.38**
Clinician IMRS-Recovery	0.24*	0.29*	0.32*
Clinician IMRS-Management	0.37	0.35**	0.39**
Clinician IMRS-Substance	0.01	0.05	0.04

*p < 0.05; **p < 0.01.

IMRS clinician total score also showed a significant, albeit weaker, relationship with Global Severity Index and the number and severity of symptoms. Clinician IMRS-Management subscale also correlated strongly negatively with these indices, whereas IMRS-clinician Recovery subscale correlations were weaker. Clinician-rated IMRS-Substance was not significantly associated with any of the BSI scales.

Discussion

The aim of this study was to evaluate the psychometric properties of the Dutch translation of the IMRS. Our results support the reliability and validity of the Dutch version of the IMRS, similar to previous studies on the psychometric properties of the English (Salyers et al., 2007; Sklar et al., 2012), Hebrew (Hasson-Ohayon et al., 2008) and Swedish (Färdig et al., 2011a) versions of the IMRS.

The Dutch translations of both the client and clinician version of the total IMRS scores had strong test–retest reliability and reasonable internal consistency, somewhat similar to previous psychometric studies. The internal consistency varied between client and clinician and also differed because of frequent omission of item 13 on medication use in our study sample. Without item 13, the internal consistency for the total score increased to an acceptable level for both versions of the questionnaire. We found moderate correlations between the total scores of the client and clinician versions, similar to those found by Färdig et al. (2011a). Lower correlations between the client and clinician versions were found by Hasson-Ohayon et al. (2008) and Salyers et al. (2007). Differences in client and clinician ratings are similarly found in other areas (e.g. Kravetz et al., 2002), posing the question of which perspective is more accurate or whether both are accurate. Future research could include qualitative investigation to better understand differences between clients and clinicians on how they use the IMRS.

When considering subscales, only IMRS-Management showed adequate internal consistency, and the IMRS-Recovery and IMRS-Substance showed moderate to low internal consistency for both versions of the scale. However, Cronbach’s alpha coefficient is dependent upon the number of items used in its calculation. Due to the small number of items (respectively, 3 and 2) of the two scales we considered their internal consistency acceptable. Our results contrast with Sklar et al. (2012), who showed an internal consistency of 0.83 for the items in IMRS-Management, 0.76 for the items in the IMRS-Recovery and 0.69 for the items in the IMRS-Substance for clinicians. Hasson-Ohayon et al. (2008) found values ranging from 0.47 to 0.83 for these subscales. However, direct comparisons are difficult because of the different items that were used for the IMRS-Substance and IMRS-Recovery and the fact that Sklar et al. (2012) studied the clinician version only. Cronbach’s alpha levels, due to differences in variances between large and small samples, could also be related to the sample size of the studies (Ponterotto & Ruckdeschel, 2007). Sklar et al. (2012) included 9142 clients for the reliability analysis. Therefore, our study may underestimate internal consistency due to the smaller sample size.

Searching for another explanation for the moderate to low internal consistency of the IMRS subscales, we executed post hoc analyses. A post hoc analyses of the subscale Recovery identified item 12 on involvements with self-help activities as the most problematic. However, deleting this item resulted in little improvement (client version improved from 0.53 to 0.56 and in the clinician version improved from 0.61 to 0.66).

Another post hoc analysis showed that clients who omitted item 13 scored higher on the mean score of all remaining items of the IMRS than clients who completed all items. It is likely that clients omitted item 13 because no medications were prescribed to them given the majority of respondents’ diagnoses (e.g. pervasive development disorders, personality disorders) may not have medications prescribed. Similarly, the IMRS-Substance items 14 and 15 might also appear problematic because there is no clear option for consumers to score when they want to report no problematic use of alcohol or drugs. We suggest adding an option in items 13, 14 and 15 in which clients can report no medication is prescribed to them and that they do not use alcohol and drugs, which would be scored a 5. Adding these options might clear possible misunderstanding for respondents and improve the internal consistency of the IMRS subscales.

The IMRS-Total scale scores showed reasonable support for concurrent validity. As expected, there was a relation with coping on the UCL, showing that clients scoring high on the IMRS also reported more active coping styles. Our results are consistent with Hasson-Ohayon et al. (2008) and extend their results by showing a relationship between coping styles, particularly passive coping and avoidance style, which are associated with mental health problems and experienced burden (McFarland & McFarlane, 1997). On the other hand, the differential patterns of correlations we found between the clinician-rated IMRS (IMRS total scale and IMRS-Management) and the active, avoidance and passive coping styles support Skinner et al. (2003) criticism on the approach versus avoidance coping styles cannot be the extremes of one action category.

The IMRS total scale score correlated significantly with several DES scales as expected. However, although we predicted that IMRS would not be related to the Involved Community, this subscale actually was correlated with the IMRS. It might be that when clients have a greater feeling of acceptance from the community, they need less social support and are less dependent on actively seeking help from others. Future research of the total IMRS should focus more specifically on how social support and feeling of empowerment is related to clients’ recovery.

The medium to strong correlations between the IMRS and the BSI are promising. Self-reported overall psychological distress, as well as the number and intensity of symptoms were related to lower scores on the IMRS total scale and IMRS subscales. These findings are in line with results from Stewart & Kopache (2002) who show that patients who are recovering well from their illness also show less symptom distress.

Our study included a relatively small sample of psychiatric clients that limits generalizability and our ability to conduct confirmatory factor analyses. In addition, our study showed lower internal reliability on the subscales than in the previous sample using similar scoring (Sklar et al., 2012). Generally, low internal reliability on a scale puts an upper limit on correlations that one might find with the scale. However, even with lower reliability, some of the subscales showed significant correlations with other variables supporting validity of the constructs. Further research in a larger sample is necessary to determine the confirmatory factor structure of the Dutch translation of both the client and the clinician version of the IMRS.

Conclusion

Overall, our study shows that the Dutch version of the client and clinician IMRS has good test–retest reliability, acceptable internal reliability and medium to strong client clinician reliability, particularly when used as a one-dimensional scale. The lower reliability of the subscales and the poor functioning of the IMRS-Substance subscale suggest caution in using the subscales. This study also shows new evidence for the concurrent validity of the IMRS, extending to additional measures. Our findings support the Dutch version of the IMRS for use in evaluating recovery-orientated care in the Netherlands or other Dutch-speaking locales. However, further research is needed to determine whether subscales can be used.

Declaration of interest

The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the article.

Acknowledgements

This work was carried out at Saxion University of Applied Sciences, Faculty of Applied Psychology, Deventer.