Psychometric properties of the Ruminative Response Scale-short form in a clinical sample of patients with major depressive disorder

Abstract

Background

The Ruminative Response Scale (RRS)-short form is one of the most widely used measures of rumination, comprising ten items and two components: reflection and brooding. The aim of this study was to investigate RRS validity and reliability in a clinical sample of French patients with major depressive disorder (MDD).

Subjects and methods

Outpatients with a DSM-IV-TR diagnosis of MDD were recruited from a public academic hospital in France. Depressive symptoms were evaluated by the Beck Depression Inventory, anxiety by the State-Trait Anxiety Inventory – state scale, and quality of life by the 36-Item Short Form Health Survey (SF-36) questionnaire. Confirmatory factor analyses, item-dimension correlations, Cronbach’s α-coefficients, Rasch statistics, and external validity were tested. Differential item functioning analyses were performed for sex.

Results

A total of 109 patients participated. The final reflection–brooding two-factor model of the RRS showed a good fit (root-mean-square error of approximation 0.041, comparative fit index 0.987, standardized root-mean-square residual 0.048) after removing one item (daily diary writing). Internal item consistency and reliability were satisfactory for the two dimensions. External validity testing confirmed that RRS scores were correlated with Beck Depression Inventory, State-Trait Anxiety Inventory, and SF-36 scores. There was no differential item functioning across sexes.

Conclusion

These results demonstrated good scale reliability and validity for assessing rumination in patients with MDD.

Keywords:

• rumination
• response-style theory
• major depressive disorder
• psychometric properties
• validity
• reliability

Background

Rumination is defined as repetitive and passive self-focused thoughts on one’s negative feelings, symptoms of distress, and their causes and consequences.^1–4 Rumination is a method of coping with negative mood;⁵ however, this strategy is often ineffective and has been identified as an important factor in the onset and maintenance of depression.⁶ Rumination is also a significant predictor of anxiety, hopelessness, psychological distress, and suicidal ideation.^7–9 In addition, there is evidence that rumination is associated with less therapeutic responsiveness to both antidepressant and cognitive behavioral interventions.^10,11 However, the link between rumination and health outcome is probably more complex. Recent works have reported that rumination is not a unitary process, but rather a multidimensional construct associating adaptive and maladaptive components.^3,12–15 In consideration of the widely accepted response-style theory, there is growing evidence that brooding, ie, “a passive comparison of one’s current situation with some unachieved standard” and reflective pondering or reflection, ie, “a purposeful turning inward to engage in cognitive problem-solving to alleviate one’s depressive symptoms” are two distinct components of rumination.¹⁶ Brooding may represent a more maladaptive aspect of rumination than reflection, with stronger links to depression and suicide attempts.^13,16,17 It is thus of utmost importance to use an appropriate tool to assess the different components of rumination to improve the characterization and treatment of patients with depression.

The Ruminative Response Scale (RRS)-short form is one of the most widely used self-reported measures of rumination, comprising ten items and describing the factors of brooding and reflection.¹⁶ The short form was developed by removing more than half of the items of the initial RRS scale that overlapped with depressive symptoms.¹⁶ Some recent studies have supported the reflection–brooding two-factor model and confirmed the satisfactory psychometric properties of this scale.^18–22 However, no study so far has examined the psychometric properties of the RRS-short form in a clinical sample of patients with depression; previous studies mainly used nonclinical samples of undergraduate students. To what extent the reflection–brooding two-factor model can be generalized to clinically depressed groups remains an open question.^19,21,22 In addition, another concern is the sex invariance of the RRS, which has not been examined. Because measurement invariance reflects the degree to which the psychometric properties of a measure remain constant across groups, confirmation of measurement invariance across sex represents a necessary prerequisite for meaningful interpretation of mean differences between men and women.^23,24 Lastly, the RRS-short form has not been cross-validated in different cultural contexts.¹⁹ To date, the RRS has not been validated in French-speaking subjects. The purpose of this study was therefore to investigate the validity and reliability of the RRS-short form in a clinical sample of French patients with major depressive disorder (MDD).

Subjects and methods

Study population

All prospective outpatients (attending external consultations or daytime hospital hours) with a diagnosis of MDD (including single-episode and recurrent disorder, in current, partially, or fully remitted disorder periods) were consecutively included for a period of 12 months, from January 2011 to December 2011. The inclusion criteria were age 18–85 years, diagnosis of MDD according to the Diagnostic and Statistical Manual of Mental Disorders (DSM)-IV-TR criteria,²⁵ confirmed through the Mini-International Neuropsychiatric Interview Plus,²⁶ and French as a native language.

Exclusion criteria were: 1) any history or current illness or treatment that may affect the central nervous system or mood (eg, brain pathology, traumatic brain injury, dementia, infectious diseases); 2) any other psychiatric disorder, mood disorder (including current mania, hypomania, history of bipolar illness), psychotic disorder (current and remitted), predominant anxiety disorder, or current alcohol or drug abuse; and 3) presence of delusional ideas or hallucinations (consistent or not with mood). Data collection was approved by the French ethical committee, Commission Nationale de l’Informatique et des Libertés (1223715). This study was constructed in accordance with the Declaration of Helsinki and French good clinical practice.²⁷ All patients were informed of the study and gave informed written consent.

Data collection

The RRS-short form is a self-administered rumination questionnaire of ten items describing two dimensions: brooding and reflection. For each item, each subject indicates the frequency of each event on a 4-point scale ranging from 1 (“almost never”) to 4 (“almost always”).

Sociodemographic variables including age, sex, marital status (in couple versus alone), living arrangement (alone or living with a partner/family/collective), education level (primary/high school versus university level), and employment status (employed versus unemployed) were reported.

Clinical characteristics were duration of illness, age at illness onset, number of previous hospitalizations due to depression, number of depressive episodes, depression severity using the 13-item self-rating Beck Depression Inventory (BDI; total score range 0–39, with higher scores indicating greater depression and scores $16 corresponding to a diagnosis of severe depression),^28,29 and anxiety severity using the 20-item self-rating State-Trait Anxiety Inventory – situational/state anxiety (STAI-S; total score range 20–80, with higher scores indicating greater anxiety).³⁰

Quality of life (QoL) was evaluated using the 36-Item Short Form Health Survey (SF-36), which is a generic, self-administered QoL questionnaire of 36 items. It evaluates eight dimensions: physical functioning, social functioning, role – physical problems, role – emotional problems, mental health, vitality, bodily pain, and general health. Two composite scores can be calculated: the physical composite score and the mental composite score. Each dimension is scored within a range from 0 (low QoL) to 100 (high QoL).³¹

Statistical analysis

The validation process included construct validity, reliability, and some aspects of external validity. The structure of the RRS-short form was explored using confirmatory factor analysis (Lisrel model); previous studies have described a two-factor structure (ie, brooding and reflection).^16,22 The following indicators were required to indicate good model fit: the root-mean-square error of approximation (RMSEA) was acceptable if <0.08 and satisfactory if <0.05, the comparative fit index (CFI) was acceptable if >0.9, the standardized root-mean-square residual (SRMR) was acceptable if <0.08, and the Akaike information criterion (AIC), which is an index used to compare models, was acceptable with smaller AIC values, indicating a better model.^32–37

The unidimensionality of each dimension (ie, the items of each dimension measuring a single construct) was evaluated using Rasch analysis. In particular, Rasch analysis included inlier-sensitive fit (INFIT), which should be ranged between 0.7 and 1.3 to ensure that all items of the scale measure the same concept.³⁸

With regard to internal structural validity, we explored item-dimension correlations using two indicators: item internal consistency (IIC) and item discriminant validity (IDV). IIC was evaluated by correlating each item with its own dimension (corrected for overlap) using Pearson’s coefficient (a correlation higher than 0.4 was recommended for supporting satisfactory IIC).³⁹ IDV determined the extent to which items were more correlated with their own dimensions than with other dimensions.⁴⁰ Reliability was assessed using Cronbach’s α-coefficient, which should be higher than 0.7.³⁹ Floor and ceiling effects and missing values were also reported. A rate of missing values less than 15% was expected. Differential item functioning (DIF) was explored for sex to determine whether all items behaved the same way between women and men.⁴¹

External validity was assessed by studying the relationship between dimension scores of the RRS-short form and the scores of other instruments (BDI, STAI-S, and SF-36). Discriminant validity was determined by comparing the RRS-short form dimension mean scores across patient groups (sex, marital status, living arrangement, educational level, and employment status) and by studying the correlations of the RRS-short form dimension scores with age, age at illness onset, and number of hospitalizations. Several hypotheses were formulated: the RRS-short form scores 1) should differ according to sociodemographic characteristics (ie, age and sex),^42,43 2) should be positively correlated with the severity of disease (ie, BDI, STAI-S, number of previous hospitalizations, and age at illness onset), with stronger correlations for the brooding dimension than for the reflection dimension,^16,22,44 and 3) should be negatively correlated with the QoL of patients, especially for the mental dimensions.^45,46 Data analyses were performed using SPSS 20.0 (IBM Corporation, Armonk, NY, USA), Winsteps^® Rasch measurement computer program (Winsteps 3.91.0; Linacre JM. OR, USA) for Rasch analysis, Stata 9.0 (StataCorp LP, College Station, TX, USA) for DIF analyses, and AMOS 20.0 (IBM Corporation) for confirmatory factor analysis.

Results

Sample characteristics

A total of 109 patients were included in the present analysis. Sample characteristics are presented in Table 1. The mean age was 48 years (standard deviation [SD]=12.2). Among these 109 patients, 56.9% were women, 45% were in a couple, 36.7% had an education level below the university level, 63% were unemployed, and 35.8% were living alone. The average dis ease duration was 9.07 years (SD=8.23).

Table 1 Sample characteristics (N=109)

Characteristics	Mean ± SD or N (%)
Sociodemographic data
Age (years)	48.0±12.2
Gender
Women	62.0 (56.9)
Marital status
Couple	49.0 (45.0)
Living arrangement
Alone	39.0 (35.8)
Education level
University level	69.0 (63.3)
Employment status
Unemployed	68.0 (63.0)
Clinical data
Duration of illness (years)	9.1±8.2
Age at illness onset	37.6±12.8
Number of hospitalizations	1.8±2.1
First episode of depression
Yes	11 (13.1)
BDI score*	16.0±7.9
BDI score $16*	58 (55.8)
STAI-S score	55.6±12.7
Quality of life data
SF-36
Physical composite score	43.3±10.0
Mental composite score	30.3±11.2

Note:

* BDI score $16 corresponding to a diagnosis of severe depression.

Abbreviations: BDI, Beck depression inventory; STAI-S, state-trait anxiety inventory, situational or state anxiety; SF-36, 36-item short form health survey.

Construct validity and reliability

Psychometric analysis revealed that the brooding dimension presented satisfactory properties for all indicators. However, the reflection dimension presented unsatisfactory properties for the following indicators: Cronbach’s α-coefficient (<0.7), IIC (<0.4), IDV, and INFIT mean square above the acceptable range. One item (“Do you generally write down what you are thinking and analyze it?”, noted Item 10) was responsible for these unsatisfactory properties and was removed (Figure 1). The final reflection–brooding two-factor model of the RRS showed good fit, and all indices from the confirmatory Lisrel model were satisfactory (RMSEA=0.041, CFI=0.987, SRMR=0.048) (Figure 2). IIC was satisfactory for all dimensions: each item achieved the 0.4 standard (range 0.7–0.84). The correlation of each item with its own dimension was higher than with the other dimensions (IDV). Cronbach’s α-coefficients ranged from 0.74 to 0.83, indicating acceptable reliability. Floor effects ranged from 1% to 4.9%, and ceiling effects ranged from 2% to 3.9%. The percentage of missing data did not exceed 6.4%. According to the definition of the DIF, there was no difference in item behavior according to sex. Dimension characteristics are provided in Table 2.

Figure 1 Ruminative Response Scale-short form with ten items.

Abbreviations: RMSEA, root-mean-square error of approximation; CFI, comparative fit index; SRMR, standardized root-mean-square residual; AIC, Akaike information criterion.

Figure 2 Ruminative Response Scale-short form with nine items.

Abbreviations: RMSEA, root-mean-square error of approximation; CFI, comparative fit index; SRMR, standardized root-mean-square residual; AIC, Akaike information criterion.

Table 2 Dimension characteristics of the RRS-short form

Dimension/index (number of items)	Mean ± SD	Missing values N %	Item-internal consistency (min–max)	Item discriminant validity (min–max)	Floor %	Ceiling %	Alphaa	INFITb (min–max)
Brooding (5)	13.22±3.64	6 (5.5)	0.698–0.836	0.348–0.607	4.85	3.88	0.832	0.76–1.22
Reflection (4)	10.75±2.77	7 (6.4)	0.725–0.766	0.339–0.584	0.98	1.96	0.744	0.95–1.04
Index (9)	13.22±3.64	6 (5.5)	0.698–0.836	0.348–0.607	4.85	3.88	0.832	0.76–1.22

Notes:

a Cronbach’s alpha;

b Rasch’s statistics.

Abbreviations: RRS, Ruminative Response Scale; INFIT, inlier-sensitive fit.

External validity

As expected, RRS-short form scores were significantly correlated with age (correlation coefficients ranged from −0.22 to −0.2). Older patients had lower scores of rumination than younger patients. However, RRS-short form scores did not significantly differ according to sex.

RRS-short form scores were also associated with the severity of the disease (r=0.5–0.61 for the BDI, 0.33–0.52 for the STAI-S, and −0.28 to −0.3 for age at illness onset). There was only a trend for the number of hospitalizations (P=0.069 for the brooding dimension and P=0.076 for the index). As expected, the brooding dimension was generally associated with higher severity than the reflection dimension, except for age at illness onset.

Lastly, RRS-short form scores were negatively correlated with the QoL of patients, with larger correlation coefficients for the mental dimensions (eg, r=−0.57 for the mental component summary score and r=−0.25 for the physical component summary score). As for the severity of the disease, the brooding dimension was associated with a lower level of QoL than the reflection dimension. All details are provided in Table 3. Item details of the final French version of the RRS-short form are reported in the Supplementary material.

Table 3 External validity of the RRS-short form

Characteristics	Brooding	Reflection	RRS index
Age (years)
r	−0.197	−0.195	−0.215
P-value	0.025	0.052	0.032
Gender, mean ± SD
Females (N=62)	13.31±3.79	10.96±2.9	24.39±5.99
Males (N=47)	13.11±3.47	10.49±2.59	23.68±5.42
P-value	0.782	0.391	0.538
Number of hospitalisations
r	0.235	0.169	0.229
P-value	0.069	0.193	0.076
Age at illness onset (years)
r	−0.219	−0.303	−0.275
P-value	0.054	0.007	0.016
BDI score
r	0.594	0.501	0.611
P-value	0.000	0.000	0.000
STAI-S score
r	0.518	0.327	0.475
P-value	0.000	0.001	0.000
SF-36 score
Physical functioning
r	−0.178	−0.235	−0.224
P-value	0.072	0.018	0.025
Social functioning
r	−0.514	−0.437	−0.546
P-value	0.000	0.000	0.000
Role physical
r	−0.332	−0.245	−0.338
P-value	0.001	0.013	0.001
Role emotional
r	−0.338	−0.292	−0.366
P-value	0.001	0.003	0.000
Mental Health
r	−0.631	−0.448	−0.617
P-value	0.000	0.000	0.000
Vitality
r	−0.524	−0.378	−0.520
P-value	0.000	0.000	0.000
Bodily pain
r	−0.313	−0.285	−0.350
P-value	0.001	0.004	0.000
General health
r	−0.385	−0.362	−0.437
P-value	0.000	0.000	0.000
Physical component summary score
r	−0.207	−0.212	−0.245
P-value	0.041	0.037	0.016
Mental component summary score
r	−0.559	−0.425	−0.569
P-value	0.000	0.000	0.000

Notes: r, Pearson’s coefficient; significant associations (P<0.05) are in bold.

Abbreviations: RRS, Ruminative Response Scale; BDI, Beck depression inventory; STAI-S, state-trait anxiety inventory, situational or state anxiety; SF-36, 36-item short form health survey.

Discussion

Our study provided evidence to support the French version of the nine-item RRS-short form as a valid and reliable instrument for assessing rumination in middle-aged native French-speaking subjects with MDD.

With regard to construct validity, confirmatory factor analysis showed that the two-factor model of the RRS-short form proposed by Treynor et al¹⁶ was not entirely satisfactory (RMSEA >0.05). The reflection dimension presented unsatisfactory properties for several important indicators (ie, Cronbach’s α-coefficient, IIC, IDV, and INFIT mean square). One item (“Do you generally write down what you are thinking and analyze it?”) was responsible for these unsatisfactory properties and was removed. After removing this item, the reflection–brooding two-factor model of the RRS showed a good fit, and all indices were satisfactory (RMSEA=0.041, CFI=0.987, SRMR=0.048).

Several hypotheses may be suggested to explain why the writing item was not relevant in our MDD population. First, our population was older (mean age 48 years) than previous populations in which other validations were carried out (mainly adolescents or young adults). Adolescents or young adults with high rumination levels may be more inclined to use diary writing spontaneously as a coping strategy for rumination. For example, blogging was recently shown to be associated with better mood outcomes in adolescents with emotional difficulties.⁴⁷ A life-story review was also found to be an effective intervention to express one’s inner feelings and provide emotional catharsis in adults with MDD.^48,49 However, only 11% of French people between the ages of 35 and 49 years were reported to write versus 23%–28% of French people between the ages of 15 and 34 years.⁵⁰ It is thus probable that writing may be spontaneously used by adolescents and young adults to cope with ruminations, but not by middle-aged adults, especially those with current acute MDD. MDD is characterized by a diminution in all activities, including writing. The writing item is the only one in the reflection dimension of the RRS that implies a voluntary action associated with an effective problem-solving and adaptive comportment. It may also be suggested that our patients received cognitive and behavioral therapy for their depression, which includes diary writing about negative thoughts, regardless of the presence of ruminations. To make a long story short, our results clearly suggest that items regarding daily writing about negative thoughts is not relevant for assessing ruminations in middle-aged people with MDD.

Finally, the unidimensionality of the two dimensions was supported by satisfactory INFIT statistics. The IIC, IDV, and Cronbach’s α-coefficients were all satisfactory. The floor and ceiling effects were also acceptable.

External validity, which was explored using sociodemographic characteristics and established psychiatric and QoL measures, globally supported our hypotheses, except for sex. Older age was associated with less rumination, which was consistent with recent studies⁴³ and psychological theory on goal adjustment and age-related shifts.^51,52 This finding may be explained by a decreased emotional responsiveness with age, increased emotional control, and psychological immunization to stressful experiences. Adaptation of goals and a different weighing of personal priorities have been shown to provide the basis for the stability, resourcefulness, and resilience of aging adults.⁵²

Unexpectedly, we found no significant association between rumination and sex. Previous studies have reported higher rates of depression in women than in men and also a greater tendency in women to ruminate.^13,53 However, a recent meta-analysis reported that effect sizes for sex differences in rumination were in fact small in magnitude (,0.2) and not clinically significant.⁵⁴ In addition, the majority of previous studies on rumination concerned nonclinical samples. It is thus conceivable that a sex difference in rumination exists in nonclinical samples, explaining the sex difference in depression prevalence, but in a clinical population of patients with MDD, the levels of rumination are similar because rumination is one of the causes of depression.

As expected, rumination was associated with illness severity, as measured by depressive and anxiety-symptom levels and age at illness onset. Rumination is a well-established risk factor for the onset and maintenance of major depression and anxiety symptomatology,^15,22,55 and is described as a major vulnerability factor for depression in young individuals.^56,57

The association between rumination and QoL was also expected, and has been reported in previous research.⁵⁸ This association is also supported by recent neuroimaging studies, which have reported that QoL is closely linked to brain networks that regulate affective and emotional behaviour.^59–62

Finally, we found stronger correlations for the brooding dimension than for the reflection dimension, with greater illness severity and lower QoL. This is consistent with previous studies suggesting that brooding might indeed represent the relatively maladaptive component of rumination.^12,13,16,18

A last important finding of this study was the absence of DIF across sex. Our findings demonstrated the measurement invariance of the RRS-short form across sex, which is a prerequisite to compare men and women in surveys. No previous study has examined sex invariance in a clinical sample of MDD for the RRS-short form. Evidence of measurement invariance across men and women indicates that the RRS-short form assesses the same construct in men and women. On the contrary, a lack of invariance would have implied that the differences may reflect not only a true mean difference but a difference in the relationship between the latent rumination variable and the obtained score.

Limitations

Some limitations of this study have to be carefully considered. First, the sample may not be representative of the entire population of patients with MDD. Because our study took place in one psychiatric center and in a relatively small sample, our findings may not be generalized to all patient groups, such as inpatients or outpatients with MDD receiving ambulatory primary care. Confirmation is thus needed on larger and more diverse groups of patients.

Second, validity is confirmed when the measurement predicts an external criterion based on a gold standard. In the case of rumination, there is no gold standard, so the instrument is considered valid if it consistently fits other constructs. In our study, we performed comparisons with measurements of depression (BDI), state anxiety (STAI-S), and QoL (SF-36). Although this choice can be debatable, it can be assumed that our assumptions based on the relationships between the RRS-short form and these three scales are both reasonable and pragmatic.

Third, the sensitivity to change for the RRS-short form should be explored in future studies. This property is of particular interest for the follow-up of patients in clinical practice. Fourth, we used a self-rating scale (BDI) to measure the severity of depression. However, several studies support a satisfactory convergent validity between the BDI and more standardized objective tools, such as the Hamilton Psychiatric Rating Scale for Depression and the Montgomery–Åsberg Depression Rating Scale.^29,63,64 The Inventory of Depressive Symptomatology would have been of interest, as it contains one item (16 – “Outlook [self]”) that allows the assessment of the intensity of ruminations considering major and minor defects in self.

Finally, important data concerning external validity were not collected in our study. In particular, the cross-sectional nature of our study prevented us from making longitudinal predictions in terms of mid- and long-term change in clinical state (eg, response to treatment, remission, and relapse). Future studies should specifically address these issues.

Perspectives

This study raises the need for further investigations of rumination processes. Indeed, recent studies have highlighted the importance of understanding rumination and similar negative cognitive styles, such as perseverative cognition or automatic thoughts (like mind wandering). These processes, which are greater in depression due to executive and attentional impairments,⁶⁵ could be associated with autonomic dysfunctions and high cardiovascular risks.⁶⁶ Future work is thus needed to improve knowledge about rumination in MDD.

Conclusion

Our study demonstrated satisfactory acceptability and psychometric properties of the French-language version of the RRS-short form, thus supporting its use as a rumination measurement for patients with MDD. The French RRS differed slightly from the English version, with one item having to be removed. Moreover, our study proved the measurement invariance of the RRS-short form, which is a prerequisite to compare groups or individuals in cross-sectional or longitudinal surveys.

Author contributions

XYZ and NP were responsible for conception, design, study coordination, and inclusion and clinical data collection, MA, KB, AL, PA, and LB for analysis of data, XYZ, NP, MA, GF, FB, CL, PA, and LB for interpretation of data, and XYZ, MA, KB, GF, FB, and LB for drafting and writing the manuscript: All authors saw and approved the final version of the manuscript. The corresponding author had full access to all data and decided to submit the paper for publication.

All authors contributed toward data analysis, drafting and revising the paper and agree to be accountable for all aspects of the work.

Disclosure

The authors report no conflicts of interest in this work.