Smaller effects matter in the psychological therapies: 25 years on from Wampold et al. (1997)

Wampold et al. (1997) reported a landmark meta-analysis concluding that “the efficacy of bona fide treatments are roughly equivalent” [p.203]. Crucially, the authors applied three stringent criteria to the selection of studies: direct comparisons between treatments, named treatments rather than general types, and bona fide treatments as opposed to alternate treatments. They showed all effects to centre around zero and yield a maximum true effect size (ES d) difference of .21, providing support for the Dodo Bird verdict of no real – or at best, small – differences between the effectiveness of bona fide psychological therapies.

Twenty-five years on, the article has achieved high impact – in the region of 1700 Google Scholar citations and 700 Web of Science (WoS) citations, twice that of the next highest-cited relevant article in WoS – and can reasonably be viewed as the high watermark of the Dodo Bird argument. Clinically, the article has done much to protect the collective body of bona fide psychological therapies, providing practitioners and patients with treatment options within the context of an ever-increasing and diverse population in need of psychological help. For research, it marked a turning point in the argument of there being a dominant single therapy modality and enabled the focus to move away from considering only differences between therapies and move towards investigating areas of variability both between and within therapies.

Of course, there are subtle differences between treatment modalities, often masked by meta-analyses (e.g., by classification categories of treatments) and by group mean comparisons taken at a single time point (e.g., Baldwin & Imel, 2020). While it might be argued that single studies do not outweigh meta-analytic findings, they may act as more sensitive indicators of differences derived from “same experiment data”. For example, a recent large (N >500) pragmatic, non-inferiority randomized trial comparing cognitive behavioural therapy (CBT) with person-centred experiential therapy (PCET), embedded in routine practice and satisfying Wampold et al.’s three inclusion criteria, found no significant difference between treatments at 6-months but at 12 months favoured CBT (ES = 0.27; Barkham et al., 2021). Effects are not static; they can change over time post-therapy and also across the course of therapy, as evidenced in a large practice-based study where patient gains in initial sessions significantly and differentially favoured non-directive supportive therapy but in later sessions favoured CBT (Pybis et al., 2017). Such subtle differences between therapies might have important implications for treatment assignment but are lost when considering results at an overall group level.

The counterpoint is that smaller differences are not important. But consider the maths. A traditional ES (d) of .20 is equivalent to an 11% difference in outcomes between therapy conditions (i.e., affecting 110 people per 1000 treated). In the English Improving Access to Psychological Therapies (IAPT) programme, hundreds of individual services treat thousands of patients each year and the national throughput of patients is >1 m. In this context, such differences will be meaningful to patients and service providers. In short, smaller effects matter because of their collective and cumulative impact.

Consider a recent large meta-analysis reporting an overall ES d of 0.15 for routine outcome monitoring, increasing marginally to 0.17 for patients not-on-track (de Jong et al., 2021). While classed as a small effect, it is additive to the existing overall effect of psychotherapy (i.e., it is an effect over and above the general effect of psychological therapies). Smaller effects have value and ESs that are equal in size are rarely equal in meaning. There is an understandable search for larger effects, especially regarding process variables. But whereas between group comparisons reflect causal effects, correlational studies represent only descriptive relations between two variables. Hence, we need to be cautious in how we read, interpret, compare, and cross reference reported effect sizes, taking note of the context and research design in which they are obtained.

Returning directly to the issue of treatment comparisons, the research agenda now moves to the question: Which of these available bona fide therapies is best suited to the characteristics of this patient? Our ability to understand variability and identify moderators provides the opportunity to harness smaller effects to greatest benefit. Current attempts at matching patients to treatments through the use of machine learning, thereby maximizing the impact of specific psychological therapies, is testimony to this endeavour. Differential (stratified) assignment of patients to low or high-intensity therapies in the IAPT programme has been shown to be more effective (d = .19) than standard stepped care, although slightly more expensive (Delgadillo et al., 2022). Again, a small effect. But there is now the real possibility of bona fide therapies being offered to patients with greater precision and impact.

Wampold et al. (1997) concluded with the question as to why researchers invest in pursuing small effects in comparison to, for example, therapist effects. The response, 25 years on, is that we need both. Therapist effects are not ubiquitous, being more apparent as patient intake severity increases (Saxon & Barkham, 2012). And considerable advances have been achieved in relation to identifying the characteristics of more effective therapists (see Wampold & Owen, 2021) as well as the potential for matching therapists with patients’ presenting problems (Constantino et al., 2021). The combination of utilizing differential effects for the purposes of therapist matching together with greater precision in treatment assignment holds promise for our field.

In conclusion, for the effective delivery of appropriate psychological therapies in response to a population health need, smaller effects matter and it may be time to re-evaluate Cohen’s litany of small, medium, and large effects and consider the impact of research findings in their specific context (see Barkham & Lambert, 2021). As such, Wampold et al.’s (1977) article stands as a watershed in determining the overall similarity of psychological therapy outcomes. It is now up to the scientific community to utilize embedded pragmatic trials, practice-based studies, meta-analyses, and other research methods to see how smaller (and other) effects between and within treatments can ensure each patient is matched with the most appropriate bona fide therapy and with the best suited therapist.

Disclosure Statement

No potential conflict of interest was reported by the author(s).