Who benefits from psychotherapies for adult depression? A meta-analytic update of the evidence

Abstract

It is not clear whether specific target groups for psychotherapies in adult depression benefit as much from these treatments as other patients. We examined target groups that have been examined in randomized trials, including women, older adults, students, minorities, patients with general medical disorders, and specific types of depression, and we examined where patients were recruited. We conducted subgroup and multivariate metaregression analyses in a sample of 256 trials (with 332 comparisons) comparing psychotherapy with an inactive control condition. Only 22% of the studies had low risk of bias (RoB), heterogeneity was high and there was a considerable risk of publication bias. A meta-regression analysis among low RoB studies showed that effect sizes found for studies among women, older adults, patients with general medical disorders, patients recruited from primary care, and patients scoring above a cut-off on a self-rating depression scale, did not differ significantly from effect sizes from other studies. For other target groups, the number of low RoB studies was too small to draw any conclusion. We found few indications that psychotherapies for adult depression are more or less effective in women, older adults, patients with comorbid general medical disorders, and primary care patients.

Introduction

It is well established that psychological interventions are effective in the treatment of adult depression, including cognitive behavior therapy (Cuijpers et al., 2013; Furukawa et al., 2014), interpersonal psychotherapy (Churchill et al., 2010; Cuijpers, Donker, Weissman, Ravitz, & Cristea, 2016), behavioral activation therapy (Ekers, Richards, & Gilbody, 2008; Shinohara et al., 2013), problem-solving therapy (Malouff, Thorsteinsson, & Schutte, 2007), and possibly non-directive counseling (Cuijpers et al., 2012), third-wave psychotherapies (Churchill et al., 2010), and psychodynamic therapy (Leichsenring & Rabung, 2008). There is also considerable evidence that the effects of these therapies are comparable or have only minimal differences (Barth et al., 2013; Palpacuer et al., 2017). Furthermore, these therapies can be delivered in an individual, group or guided self-help format, again with no or only minimal differences between the different types of format (Cuijpers, Donker, van Straten, Li, & Andersson, 2010; Richards & Richardson, 2012). In addition, there are no clear indications that number of sessions or contact time between therapist and patients are related to outcome (Nieuwsma et al., 2012).

It is less clear, however, whether the type of target population is related to the outcomes of therapy. Psychological therapies have been examined in many different target populations, including older adults, college students, patients with comorbid general medical disorders (such as cancer, diabetes, or heart disease), as well as more clinical subgroups, like patients with chronic depression, subthreshold depression, or perinatal depression. Some meta-analyses have examined whether psychotherapies in specific target populations differ from unselected target groups of adults. Such meta-analyses suggested that the effects of psychotherapies in older adults are comparable with those in younger adults (Gould, Coulson, & Howard, 2012), and that these effects are comparable in student populations compared with adults in general (Davies, Morriss, & Glazebrook, 2014). However, other meta-analyses indicated that the effects of psychotherapies in subthreshold depression are lower than those in major depression (Cuijpers et al., 2014), and that psychotherapy is less effective in primary care compared with specialized mental health care (Cuijpers, van Straten, van Schaik, & Andersson, 2009). Furthermore, the effect sizes in chronic depression seem to be lower than those in other populations with depression (Jobst et al., 2016).

At the same time, the number of randomized controlled trials has increased exponentially in the past years, with many new trials examining the effects of psychotherapies for depression in specific target groups. In the current meta-analysis, we were able to include 256 randomized controlled trials with 332 comparisons between psychotherapies and control groups, while in the last metaregression analysis examining the main target groups in 2008, we included “only” 83 trials with 135 comparisons (Cuijpers, Van Straten, Warmerdam, & Smits, 2008), which is more than three times as many trials and comparisons in 8 years’ time. This strong increase in number of comparisons makes it possible to examine subgroups of patients in more detail than was possible ever before. In this earlier metaregression analysis, we found no significant difference between studies conducted among older adults, women with postpartum depression, and other specific populations. However, because of the small number of trials this may very well have been related to the low power of several comparisons. We also could not further differentiate between target groups in this study because of the small number of trials.

We decided therefore to conduct an updated meta-analysis of studies examining the effects of psychotherapies for adult depression, with a focus on the different target groups that are examined in these trials and differences in effects between these target groups.

Method

Identification and selection of studies

A database of 1885 papers on the psychological treatment of depression was used. This database has been described in detail elsewhere (Cuijpers, van Straten, Warmerdam, & Andersson, 2008), and has been used in a series of earlier published meta-analyses (www.evidencebasedpsychotherapies.org). The database is continuously updated and was developed through a comprehensive literature search (from 1966 to 1 January 2016). Studies were identified by combining terms indicative of psychological treatment and depression (both index terms and text words). The full search string is given in Appendix 1 (supplementary material). For this database, the primary studies from earlier meta-analyses of psychological treatment for depression were also checked to ensure that no published studies had been missed. All records were read by two independent researchers independently (PC and EK) and the full-text of all papers that one of the reviewers selected were retrieved. The decision to include a study in the meta-analysis was based on consensus.

For the current meta-analysis, we included all randomized trials on the acute treatment of depression in which the effects of a psychological treatment were compared with a control group (waiting list, care-as-usual, placebo, other). Studies in which psychotherapy was compared with another active treatment (e.g. pharmacotherapy, another psychotherapy) were not included. All trials in any target group were included. Depression had to be defined according to a diagnostic interview in which a depressive disorder was established, or as a score above a cut-off on a self-rating depression scale. We excluded studies on maintenance treatments, on inpatients and on children and adolescents below 18 years of age.

Data extraction

We read all studies and categorized them according to the target group at which they were focused. Studies could be targeted at adults in general, or at adults with specific socio-demographic characteristics, including sex (only women, mixed men and women), age (college students, adults, older adults), and ethnicity (aimed specifically at minority populations; studies with a limited number of other high-risk groups next to ethnic minorities were also coded as positive), as well as clinical characteristics, including type of diagnosis, comorbid general medical disorder, and whether or not the intervention was aimed at perinatal depression. Studies among older adults were defined as studies in which an age limit (higher than 50 years; typically >55, >60, or >65) was used as an inclusion criterion. Types of diagnoses we distinguished were: subthreshold depression (depressive symptoms but no depressive disorder according to a diagnostic interview), major depressive disorder (established through a diagnostic interview), a mood disorder (that could include major depression, but also dysthymia; according to a diagnostic interview), scoring above a cut-off score on a self-rating depression scale, and chronic depression. Setting was rated according to where patients were recruited (completely or partly through the community; through primary care, outpatient settings, or other methods, such as screening in general medical settings). We considered studies to be community studies when the participants were recruited completely or partly through advertisements or mass media campaigns.

We also rated other characteristics of the studies, including treatment format (individual, group, guided self-help, mixed), number of sessions, type of control group (waiting list, care-as-usual, pill placebo, other), and where it was conducted (in a Western or non-Western country; Cuijpers, Karyotaki, Reijnders, Purgato, & Barbui, 2017). We also defined the type of psychotherapy according to operationalizations given elsewhere (Cuijpers et al., 2008), and added a category of third wave therapies (i.e. that included Mindfulness-based cognitive therapy, Acceptance and commitment therapy, Dialectic behavior therapy and Metacognitive therapy).We assessed the risk of bias of the studies according to four basic criteria suggested by the Cochrane Handbook for Systematic Reviews of Interventions (Higgins & Green, 2011): (a) adequate sequence generation (the randomization scheme was generated correctly); (b) allocation to conditions by an independent (third) party; (c) blinding of assessors of outcomes; and (d) completeness of follow-up data. All characteristics were rated by two independent assessors.

Meta-analyses

For each comparison between a therapy and a control condition, we calculated the effect size indicating the difference between the two treatments at post-test, adjusted for small sample bias (Hedges’ g) (Hedges & Olkin, 1985). Effect sizes were calculated by subtracting (at post-test) the average score of the treatment group from the average score of the control group, and dividing the result by the pooled standard deviations of the two groups. We only used those instruments that explicitly measured symptoms of depression, such as the Beck Depression Inventory (BDI; Beck, Ward, Mendelson, Mock, & Erbaugh, 1961) or the Hamilton Rating Scale for Depression (HAM-D; Hamilton, 1960). If more than one depression measure was used, the mean of the effect sizes was calculated, so that each comparison provided only one effect size.

We used the computer program Comprehensive Meta-Analysis (version 3.3.070) to calculate pooled mean effect sizes. As we expected considerable heterogeneity, we decided to calculate mean effect sizes using a random effects model. In all analyses, we calculated the I²-statistic as an indicator of heterogeneity in percentages (25% indicates low, 50% moderate, and 75% high heterogeneity) (Higgins, Thompson, Deeks, & Altman, 2003). We calculated 95% confidence intervals (CI) around I² (Ioannidis, Patsopoulos, & Evangelou, 2007), using the non-central Chi squared-based approach within the heterogi module for Stata (Orsini, Bottai, Higgins, & Buchan, 2006).

To examine differences between subgroups of studies (the target groups examined in the included trials and other subgroups), we conducted subgroup analyses according to the mixed effects model (Borenstein, Hedges, Higgins, & Rothstein, 2009), in which effect sizes within subgroups are pooled according to the random effects model and the difference between subgroups according to a fixed effects model. Bivariate and multivariate meta-regression analyses were conducted with a random effects model using the Knapp–Hartung method (Borenstein et al., 2009).

Numbers-needed-to-be-treated (NNT) were calculated using the formulae provided by Furukawa (Furukawa, 1999), in which the control group’s event rate was set at a conservative 19% (based on the pooled response rate of 50% reduction of symptoms across trials in psychotherapy for depression) (Cuijpers et al., 2014). We tested for publication bias by inspecting the funnel plot on primary outcome measures and by Duval and Tweedie’s trim and fill procedure (Duval & Tweedie, 2000), which yields an estimate of the effect size after the publication bias has been taken into account (as implemented in CMA). We also conducted Egger’s test of the intercept to quantify the bias captured by the funnel plot and test whether it was significant.

Results

Selection and inclusion of studies

After examining a total of 16,407 abstracts (13,384 after removal of duplicates), we retrieved 1885 full-text papers for further consideration. We excluded 1621 of the retrieved papers. The reasons for excluding studies are given in Figure 1. A total of 256 trials with 332 comparisons with a psychotherapy and an inactive control group met inclusion criteria. Figure 1 presents a flowchart describing the inclusion process.

Figure 1. Flowchart.

Characteristics of included studies

Selected characteristics of each of the included studies are presented in Appendix 2 (supplementary material) and references are reported in Appendix 3 (supplementary material). In the 256 included studies, a total of 23,908 patients participated (12,895 in the therapy conditions, and 11,013 in the control conditions). Participants were recruited through: (a) announcements in local newspapers and other media (114 studies), (b) referrals from health services (32 studies), (c) screening in primary care settings (26 studies), and (c) other recruitment strategies, like for example through screening in other health care sectors, or through social programs (84 studies).

In 188 of the 332 comparisons between a treatment and a control condition, cognitive behavior therapy was used as the intervention, 24 used interpersonal psychotherapy, nine used psychodynamic therapy, 17 used non-directive supportive therapy, 14 used behavioral activation, 23 used problem-solving therapy, and the remaining 57 used another type of treatment. 113 comparisons used a group treatment format, 145 studies utilized individual treatment, 57 used a guided self-help treatment format, and 17 used another treatment format. The number of treatment sessions ranged from three to 21 (median: 8). For the control group, 150 studies used a waiting list, 125 studies used care-as-usual, eleven studies used placebo, and 46 used another control group, such as discussion groups, support on request, or an educational intervention. Eighteen of the studies were targeted at students, 275 were targeted at adults and 39 were targeted at older adults. Nine studies were conducted in Africa, 23 were conducted in Asia, 20 were conducted in Australia, 109 were conducted in Europe, three were conducted in Latin America and the Caribbean, and 168 were conducted in North America and Canada.

Risk of bias

The risk of bias in most of the studies was considerable. A total of 117 of the 256 studies reported an adequate sequence generation (46%), while the other 139 did not report a sequence generation method. One hundred studies reported allocation to conditions by an independent (third) party (39%). Sixty-eight studies reported using blinded outcome assessors (27%), and 165 used only self-report outcomes, the others did not report blinding of assessors. In 127 studies intent-to-treat analyses (completeness of follow-up data) were conducted (50%). Only 57 studies (22%) met all quality criteria. 110 studies (43%) met two or three of the criteria and the 93 remaining studies met no or only one criterion.

Overall effects

The overall effect of all psychotherapies compared with non-active control groups was g = 0.71 (95% CI: 0.66–0.77) with high heterogeneity (I² = 75; 95% CI: 72–77), which corresponds with a NNT of 4.11.

When we examined the outcomes limited to the HAM-D, BDI-I and BDI-II (each of them was examined separately), the effect sizes were very comparable and heterogeneity was still high (Table 1). In 15 comparisons, the effect size was very high (g > 2.0) and these comparisons were considered to be outliers. After exclusion of these outliers, the effect size dropped to g = 0.63 (95% CI: 0.59–0.68; NNT = 4.71), with moderate to high heterogeneity (I² = 63; 95% CI: 58–67). In 61 studies, more than one psychotherapy was compared to the same control group (50 studies had two psychotherapy conditions, 8 had 3 conditions, and 3 had 4 conditions). These comparisons are not independent of each other and this may have resulted in an artificial reduction of heterogeneity and may have affected the pooled effect size. We examined the possible effects of this by conducting an analysis in which we included only one effect size per study. First, we included only the comparisons with the largest effect size from these studies and then we conducted another analysis in which we included only the smallest effect sizes. As can be seen from Table 1, the resulting effect sizes were almost the same as in the overall analyses and heterogeneity was still high in these analyses.

Table 1. Effects of psychotherapies for adult depression compared with control groups: hedges’ g^a.

		Ncomp	g	95% CI	I^2	95% CI	p^b	NNT
All studies		332	0.71	0.66–0.77	75	72–77		4.11
One effects size per study (only highest)		256	0.73	0.67–0.79	78	75–80		3.98
One effects size per study (only lowest)		256	0.66	0.60–0.71	75	72–78		4.47
Outliers excluded		317	0.63	0.59–0.68	63	58–67		4.71
Only HAMD		93	0.90	0.78–1.02	74	68–79		3.14
Only BDI-I		125	0.89	0.78–0.99	72	67–76		3.18
Only BDI-II		66	0.67	0.56–0.79	68	58–75		4.39
Only studies with low Risk of Bias (RoB)		98	0.46	0.40–0.52	58	47–67		6.76
Sociodemographic characteristics
Sex^c	Women only	62	0.64	0.54–0.75	70	61–77	0.16	4.63
	Mixed	270	0.73	0.67–0.80	76	73–78		3.98
Age group	Older adults	39	0.78	0.59–0.98	85	81–88	0.001	3.69
	Adults	275	0.68	0.63–0.74	72	69–75		4.32
	Student populations	18	1.15	0.91–1.39	42	0–65
Ethnicity	Minorities	13	0.63	0.36–0.89	73	48–83	0.52	4.71
	Mixed	319	0.72	0.66–0.77	74	72–77		4.04
Clinical characteristics
Comorbidity	General medical dis	48	0.57	0.44–0.69	71	61–78	0.02	5.29
	No comorbidity	284	0.74	0.68–0.80	75	72–78		3.92
Type of diagnosis	MDD	97	0.68	0.59–0.78	69	61–74	0.48	4.32
	Mood disorder	70	0.83	0.68–0.99	84	81–87		3.44
	Cut-off score	147	0.69	0.62–0.77	69	64–74		4.25
	Chronic depression	7	0.70	0.26–1.14	75	47–88		4.18
	Subthreshold depr	11	0.61	0.34–0.88	82	67–88		4.89
Perinatal depression	Yes	36	0.59	0.48–0.70	60	43–72	0.02	5.08
	No	296	0.74	0.68–0.80	76	73–78		3.92
Setting
Recruitment	Community	154	0.67	0.61–0.73	46	33–55	0.08	4.39
	Primary care	30	0.40	0.29–0.51	49	15–66		7.91
	Outpatients	39	0.78	0.64–0.93	52	27–66		3.69
	Other recruitment	94	0.59	0.50–0.68	75	70–79		5.08
Intervention characteristics
Therapy	CBT	182	0.73	0.65–0.80	76	72–79	0.40	3.98
	BAT	14	0.88	0.65–1.11	41	0–67		3.22
	Third wave therapies	14	0.72	0.58–0.86	27	0–61		4.04
	IPT	24	0.61	0.41–0.82	75	61–82		4.89
	Dynamic	9	0.48	0.19–0.76	65	8–81		6.44
	PST	23	0.81	0.57–1.05	82	74–87		3.53
	Supportive	17	0.63	0.45–0.81	48	0–69		4.71
	Other	49	0.70	0.53–0.86	78	71–82		4.18
Format	Individual	145	0.71	0.63–0.80	73	68–77	0.02	4.11
	Group	113	0.79	0.68–0.90	81	78–84		3.64
	Guided self-help	57	0.67	0.58–0.77	60	44–69		4.39
	Other/mixed	17	0.49	0.35–0.64	46	4–69		6.29
Other characteristics
Control group	Waiting list	150	0.88	0.79–0.97	68	62–73	≤0.001	3.22
	Care as usual	125	0.63	0.55–0.72	76	72–80		4.71
	Placebo	11	0.36	0.18–0.55	52	0–74		8.91
	Other	46	0.58	0.43–0.72	80	74–84		5.18
Country	Western	295	0.66	0.60–0.71	67	63–71	≤0.001	4.47
	Non-Western	37	1.11	0.91–1.30	89	87–91		2.49

Abbreviations: BAT: Behavioral Actiation Therapy; CBT: Cognitive Behavioral Therapy; CI: Confidence interval; Depr.: Depression; HAM-D: Hamilton Rating Scale for Depression; IPT: Interpersonal Psychotherapy; MDD: Major Depressive Disorder; N_comp: number of comparisons; NNT: Numbers-needed-to-treat; PST: Problem Solving Therapy.

^a Based on the random effects model.

^b The p-values in this column indicate whether the difference between the effect sizes in the subgroups is significant.

^c Two studies were only aimed at men and these were included in the “mixed” category.

Underlined values indicate that these are significant (p < 0.05).

When we limited the effect sizes to studies with low risk of bias, the pooled effect size dropped to g = 0.46 (95% CI: 0.40–0.52; NNT = 6.67) with moderate heterogeneity (I² = 58; 95% CI: 47–67).

According to Duval and Tweedie’s trim and fill procedure, there was considerable publication bias, with 81 missing studies and an adjusted effect size of g = 0.50 (95% CI: 0.44–0.56; NNT = 6.14). When we limited the analyses for publication bias to the studies with low risk of bias, we still found significant publication bias, with 19 missing studies and an adjusted effect size of g = 0.37 (95% CI: 0.31–0.44; NNT = 8.64).

Effects in specific subgroups

We first conducted the subgroup analyses with all 332 comparisons between a therapy and control group. In these analyses, we found that studies in student populations resulted in higher effect sizes than studies in adults and older adults (Table 1; p = 0.001), that studies in patients with general medical disorders resulted in lower effect sizes than other studies (p = 0.02), and that studies in women with perinatal depression resulted in smaller effect sizes than other studies (p = 0.02). We found no indication that sex (studies focused on women only versus mixed samples), ethnicity (studies aimed exclusively at minority groups versus all other studies), type of diagnosis (including chronic depression and subthreshold depression), and recruitment method were associated with differential effect sizes (ps > 0.05).

In these analyses, we also examined other characteristics of the studies and found that a mixed treatment format resulted in a smaller effects size (p = 0.02), that type of control group was associated with differential effect sizes (waiting list control groups had larger effect sizes than other control groups; p < 0.001), that studies in non-Western countries had larger effect sizes than studies in Western countries (p < 0.001), but type of therapy and treatment format were not associated with differential effect sizes (p > 0.05).

Because studies with low risk of bias give the best evidence, we repeated the subgroup analyses limited to the studies with low risk of bias (97 comparisons). Heterogeneity was lower in most of these analyses (with I² < 50 in most subgroups). In these analyses, none of the characteristics of the participants was significantly associated with the effect size. The only exception was that studies in women with perinatal depression differed significantly from the other studies (p = 0.04). However, in these analyses studies in women with perinatal depression resulted in larger effect sizes, while in the main analyses they resulted in smaller effect sizes (Appendix 4 in supplementary material).

We also conducted subgroup analyses in which we excluded outliers (Appendix 5 in supplementary material). Overall the effect sizes were somewhat lower than in the main analyses, and heterogeneity was also lower. The results on target groups were very similar to the main analyses, except that studies in perinatal depression did not differ significantly anymore from other studies. Furthermore, we found that studies that recruited patients in primary care resulted in significantly lower effect sizes than studies using other recruitment methods (p < 0.001).

Multivariate meta-regression analyses

We conducted a multivariate meta-regression analysis with the effect size as dependent variable and all characteristics of participants, the interventions and the studies as predictors. The results are presented in Table 2. As can be seen, none of the characteristics of the participants were significantly associated with the effect size. Only studies in which patients were recruited in primary care resulted in a significantly smaller effect size than other studies (p = 0.04).

Table 2. Standardized regression coefficients of characteristics of studies on psychological treatment of depression in different target populations: multivariate metaregression analyses (N = 332).

		Fully adjusted model				Parsimonious model
		Coeff	SE	p^a	p^b	Coeff	SE	p^a	p^b
Sociodemographic characteristics
Sex	Women only vs mixed	−0.02	0.12	0.88
Age group	Adults	Ref.
	Older adults	0.03	0.10	0.79	0.37
	Student populations	0.23	0.16	0.16
Ethnicity	Minorities vs mixed	−0.01	0.17	0.97
Clinical characteristics
Comorbidity	Yes vs no	−0.16	0.10	0.11
Diagnosis	MDD	Ref.
	Mood disorder	0.03	0.09	0.70	0.60
	Cut-off score	−0.08	0.08	0.33
	Chronic depression	0.13	0.23	0.58
	Subthreshold depr	−0.14	0.17	0.43
Perinatal depr.	Yes vs no	−0.20	0.14	0.17
Setting
Recruitment	Community	Ref.
	Primary care	−0.25	0.12	0.04	0.16
	Outpatients	−0.10	0.11	0.40
	Other recruitment	0.03	0.09	0.76
Other study characteristics
Risk of bias		−0.10	0.03	≤0.001		−0.12	0.02	≤0.001
Control	Waiting list	Ref.				Ref.
	Care as usual	−0.15	0.08	0.07	0.03	−0.19	0.07	0.01	0.01
	Placebo	−0.40	0.18	0.02		−0.31	0.16	0.05
	Other	−0.21	0.10	0.03		−0.26	0.09	0.01
Type	CBT vs other therapies	−0.03	0.07	0.68
Format	Individual	Ref.
	Group	−0.11	0.08	0.17	0.24
	Other	−0.13	0.09	0.14
Number of sessions		0.00	0.01	0.61
Country	Western vs non-Western	0.40	0.11	≤0.001		0.40	0.10	≤0.001
Intercept		1.25	0.15	≤0.001		1.10	0.07	≤0.001
R^2 analog		0.26				0.28

Abbreviations: CBT: Cognitive Behavioral Therapy; Coeff: Regression Coefficient; MDD: Major Depressive Disorder; SE: Standard Error.

^a The p-value in this column refers to each individual predictor.

^b The p-value in this column refers to a set of predictors in which one predictor is set as reference group.

Underlined values indicate that these are significant (p < 0.05).

To avoid overfit of the meta-regression models, we repeated this meta-regression analyses, with a (manual) stepwise backward elimination of the least significant predictor until only significant predictors remained in the model (Table 2). As can be seen, none of the characteristics of participants remained significant in these analyses, and only risk of bias, type of control group and whether the study was conducted in a Western country remained as significant predictors of the effect size.

Because so many studies suffered from risk of bias, we ran the multivariate metaregression analyses again, limited to the studies with low risk of bias. In these analyses, we only included the subgroup for which at least 10 studies were available. The results are presented in Table 3. As can be seen, none of the subgroups was significantly associated with a higher or lower effect size (except for type of control condition).

Table 3. Standardized regression coefficients of characteristics of studies on psychological treatment of depression in different target populations, limited to studies with low risk of bias: multivariate metaregression analyses (N = 77).^a

		Fully adjusted model				Parsimonious model
		Coeff	SE	p^b	p^c	Coeff	SE	p^b	p^c
Sociodemographic characteristics
Sex	Women only vs mixed	−0.16	0.22	0.47
Age group (younger vs older adults)		−0.05	0.11	0.67
Clinical characteristics
Comorbidity	No comorbidity	0.02	0.10	0.86
Diagnosis	MDD	Ref.			0.49
	Mood disorder	−0.01	0.10	0.95
	Cut-off score	−0.10	0.10	0.30
Perinatal depression (perinatal vs other)		0.26	0.24	0.28
Setting
Recruitment	Community	Ref.			0.11
	Primary care	−0.22	0.11	≤0.04
	Other recruitment	−0.19	0.12	0.12
Other study characteristics
Control	Waiting list	Ref.			0.16	Ref.			≤0.01
	Care as usual	−0.04	0.12	0.10		−0.19	0.08	0.02
	Other	−0.20	0.12	0.10		−0.28	0.09	≤0.01
Type	CBT vs other therapies	−0.03	0.07	0.69
Format	Individual	Ref.			0.70
	Group	−0.04	0.10	0.73
	Guided self-help	0.06	0.12	0.63
Number of sessions		0.00	0.01	0.73
Intercept		0.63	0.19	≤0.001		0.59	0.06	≤0.001
R^2 analog		0.30				0.34

Abbreviations: CBT: Cognitive Behavioral Therapy; Coeff: Regression Coefficient; Depr.: Depression; MDD: Major Depressive Disorder; SE: Standard Error.

^a Subgroups with less than 10 studies were removed from these analyses (student populations; minorities; patients with chronic depression; patients with subthreshold depression; other/mixed treatment format; studies in non-Western countries) or merged with other categories (outpatient samples were merged with samples that were recruited in “other” ways; and placebo control groups were merged with “other” control groups).

^b The p-value in this column refers to each individual predictor.

^c The p-value in this column refers to a set of predictors in which one predictor is set as reference group.

Underlined values indicate that these are significant (p < 0.05).

We also ran the same regression models, with the outliers excluded (Appendix 6 in supplementary material). In these analyses, we found that studies in primary care did result in a significantly smaller effect size than other studies (p < 0.01). However, in the parsimonious model in which we removed all non-significant predictors step by step, recruitment setting did not remain significant. In this parsimonious model again only risk of bias, type of control group and Western versus non-Western countries were significant predictors.

Discussion

We examined in a large sample of randomized trials whether specific target groups of psychotherapies for adult depression benefit more or less from these therapies than other target groups. Probably, the most important result of this study is that only 22% of 256 randomized trials had low risk of bias and that there was probably considerable publication bias, even among the studies with low risk of bias. Furthermore, heterogeneity was very high in most analyses, indicating that the resulting effect sizes varied considerably across studies and did not sufficiently point into the direction of one effect size. This means that despite the large number of randomized trials, it was not possible to examine for several of the examined target groups whether they benefit more or less from therapies than others.

The lack of studies with low risk of bias as well as the problem of publication bias have been found before (Driessen, Hollon, Bockting, Cuijpers, & Turner, 2015). And it should be noted that there has been improvement in terms of studies with low risk of bias. In 2010, we found that less than 10% of trials on psychotherapy for adult depression were considered to have low risk of bias (Cuijpers, van Straten, Bohlmeijer, Hollon, & Andersson, 2010). And although a small percentage of the studies had low risk of bias, the absolute number of studies was still considerable (57 studies with 97 comparisons), and this does allow us to draw some preliminary conclusions about the research questions we wanted to answer in this study.

The main goal of this study was to examine whether specific target groups benefit more or less from psychotherapies than other groups. However, for several of the target groups that are examined in these studies, only a small number of studies with low risk of bias was available. For these categories of studies, no definite conclusions can be drawn about whether they result in higher or lower effects. This was true for studies among student populations, among minority groups, patients with chronic depression and patients with subthreshold depression. We also did not find strong indications that studies among these populations resulted in higher or lower effect sizes compared to the full sample of studies (including those with low and high risk of bias), even though we controlled for level of risk of bias. This could be seen as preliminary support that the effects of therapies were not higher or lower than among other populations, but this has to be considered with caution because of the small number of trials with low risk of bias.

For other populations more high quality evidence was available. This included studies with only women as participants (compared with mixed populations), older adults (versus younger adults), patients with a comorbid general medical disorder (versus no comorbid disorders), studies among patients scoring above a cut-off on a self-rating depression scale (versus patient with a diagnosed major depressive disorder), and studies in which patients were recruitment from primary care (versus studies in which patients were recruited from the community). In each of these target groups, we could include 10 or more studies with low risk of bias, which provides more confidence in their results. Moreover, in the analyses with the studies with low risk of bias, the level of heterogeneity was also lower, which gives more confidence in the robustness of the outcomes. None of these target groups seemed to be associated with higher or lower effect sizes, except for studies in which patients were recruited in primary care. These studies resulted in somewhat smaller effects than other studies. However, recruitment as predictor lost its significance after using a stepwise removal of non-significant predictors in a more parsimonious model.

In the bivariate analyses in which we included all studies, we did find some significant associations between the effect size and the characteristics of the target group. Studies in student populations resulted in larger effect sizes, studies in patients with comorbid general medical disorders and studies in women with perinatal depression resulted in smaller effect sizes. However, none of these associations remained significant in multivariate meta-regression analyses, in which we adjusted for all characteristics of the studies. Furthermore, the number of studies with low risk of bias was too small for several categories to draw any conclusion.

One could assume based on these results that characteristics of patients are either not associated with the effect sizes of studies on psychotherapies for adult depression, or when they are associated, the differences with other studies are small and not consistent. It also implies that psychotherapies are effective in all these target groups, with no or only small fluctuations in effect sizes between target groups. However, because of the small number of trials with low risk of bias, the high levels of heterogeneity in the full sample of trials, and the considerable publication bias, even among the studies with low risk of bias, all results should be considered with caution. Therefore, a definite conclusion that psychotherapies seem to work in all these specific target groups cannot be drawn from these studies.

If we would accept the conclusion that none of these characteristics is associated with the outcome of psychotherapy, that would be in agreement with overviews on moderators and predictors of outcome within randomized trials. Reviews of studies on moderators and predictors provide very little evidence that sociodemographic variables such as age and sex are significant predictors of outcome (Hamilton & Dobson, 2002; Van, Schoevers, & Dekker, 2008). In a more recent “individual patient data” meta-analysis, we also found that sex is not a significant predictor or moderator of outcome of cognitive behavior therapy (Cuijpers et al., 2014).

Apart from the small number of trials with low risk of bias, the high level of heterogeneity and the considerable risk for publication bias, this study was also limited because we could only examine patient characteristics that were examined in larger groups of trials on psychotherapy for depression. Other characteristics that have been suggested to be associated with differential outcomes have not been examined sufficiently as target group for psychotherapies. For example, there is some suggestive evidence that marital status is related to outcome of psychotherapies (Van et al., 2008), and although work status and level of education have not been found to be significant predictors of outcome, there are also not enough trials focusing on these characteristics to examine them in a meta-analysis. Other more clinical predictors that have been suggested to affect outcomes, such as family history of depression (Van et al., 2008), childhood traumas (Vitriol, Ballesteros, Florenzano, Weil, & Benadof, 2009) and comorbid alcohol problems (Riper et al., 2014), have also not been examined enough in individual trials.

There are several limitations of this study that have to be acknowledged. We already mentioned the small number of trials with low risk of bias, the high level of heterogeneity and the considerable risk for publication bias. In addition, because this meta-analysis could only compare studies aimed at different target groups, the results are indirect evidence of the differential efficacy and all results should be validated in well-powered new randomized trials that directly compare these target groups. While the number of studies in several subgroups was already small, the number of studies with low risk of bias in several subgroups was too small to conduct specific analyses. Another limitation was that the characteristics of the subgroups were not defined in the same way across studies. For example, studies in older adults used different definitions of what old age means, with some studies referring to people older than 55 years of age, while other used 60 or 65 as cut-off. The recruitment method was also not defined very well in many studies and we had to lump together all studies in which (part of) the participants were recruited through advertising, and perinatal depression could refer to depression before or after giving birth. This may very well have added to the heterogeneity of our findings and may have prevented us from finding significant associations. A final limitation of the current study was that we did not examine the independence of the researchers in each study, while this is known to be related to the effects of interventions found in randomized trials (Munder, Brütsch, Leonhart, Gerger, & Barth, 2013). Because of these limitations, the results of this study have to be considered with caution. We found few indications that psychotherapies for adult depression are more or less effective in specific target groups. There was some support that specific characteristics are not related to better or worse outcomes, including women, older adults, patients with a comorbid general medical disorder and patients scoring above a cut-off on a self-rating depression scale. More rigorous, high quality research is needed to verify these results and to examine other target groups.

Contributors

PC had the idea for this paper with input from all authors. PC also did the analyses and wrote the first draft. The searches were done by PC and EK, data extraction was done by PC, EK, and MR. All authors have contributed to the next versions of the paper and have agreed that this version is submitted for publication.

Disclosure statement

The authors report no conflicts of interest.

Supplemental data

Supplemental data for this article can be accessed https://doi.org/10.1080/16506073.2017.1420098